Custom compression methods
Started by Ildus Kurbangalievover 8 years ago458 messages
Hello hackers!
I've attached a patch that implements custom compression
methods. This patch is based on Nikita Glukhov's code (which he hasn't
publish in mailing lists) for jsonb compression. This is early but
working version of the patch, and there are still few fixes and features
that should be implemented (like pg_dump support and support of
compression options for types), and it requires more testing. But I'd
like to get some feedback at the current stage first.
There's been a proposal [1]/messages/by-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx+Sig4ykWzWEAfkC6ZKMDy6=Q@mail.gmail.com -- --- Ildus Kurbangaliev Postgres Professional: http://www.postgrespro.com Russian Postgres Company of Alexander Korotkov and some discussion
about custom compression methods before. This is an implementation of
per-datum compression. Syntax is similar to the one in proposal but not
the same.
Syntax:
CREATE COMPRESSION METHOD <cmname> HANDLER <compression_handler>;
DROP COMPRESSION METHOD <cmname>;
Compression handler is a function that returns a structure containing
compression routines:
- configure - function called when the compression method applied to
an attribute
- drop - called when the compression method is removed from an attribute
- compress - compress function
- decompress - decompress function
User can create compressed columns with the commands below:
CREATE TABLE t(a tsvector COMPRESSED <cmname> WITH <options>);
ALTER TABLE t ALTER COLUMN a SET COMPRESSED <cmname> WITH <options>;
ALTER TABLE t ALTER COLUMN a SET NOT COMPRESSED;
Also there is syntax of binding compression methods to types:
ALTER TYPE <type> SET COMPRESSED <cmname>;
ALTER TYPE <type> SET NOT COMPRESSED;
There are two new tables in the catalog, pg_compression and
pg_compression_opt. pg_compression is used as storage of compression
methods, and pg_compression_opt is used to store specific compression
options for particular column.
When user binds a compression method to some column a new record in
pg_compression_opt is created and all further attribute values will
contain compression options Oid while old values will remain unchanged.
And when we alter a compression method for
the attribute it won't change previous record in pg_compression_opt.
Instead it'll create a new one and new values will be stored
with new Oid. That way there is no need of recompression of the old
tuples. And also tuples containing compressed datums can be copied to
other tables so records in pg_compression_opt shouldn't be removed. In
the current patch they can be removed with DROP COMPRESSION METHOD
CASCADE, but after that decompression won't be possible on compressed
tuples. Maybe CASCADE should keep compression options.
I haven't changed the base logic of working with compressed datums. It
means that custom compressed datums behave exactly the same as current
LZ compressed datums, and the logic differs only in toast_compress_datum
and toast_decompress_datum.
This patch doesn't break backward compability and should work seamlessly
with older version of database. I used one of two free bits in
`va_rawsize` from `varattrib_4b->va_compressed` as flag of custom
compressed datums. Also I renamed it to `va_info` since it contains not
only rawsize now.
The patch also includes custom compression method for tsvector which is
used in tests.
[1]: /messages/by-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx+Sig4ykWzWEAfkC6ZKMDy6=Q@mail.gmail.com -- --- Ildus Kurbangaliev Postgres Professional: http://www.postgrespro.com Russian Postgres Company
/messages/by-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx+Sig4ykWzWEAfkC6ZKMDy6=Q@mail.gmail.com
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v1.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..145f59b949 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -92,7 +92,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ TupleDescAttrCompression(tupleDescriptor, i));
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index ec10762529..21e375ad96 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -935,11 +935,48 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
+void
+freeRelOptions(List *options)
+{
+ ListCell *cell;
+
+ Assert(options != NIL);
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ pfree(def->defname);
+ pfree(defGetString(def));
+ pfree(def->arg);
+ }
+ list_free_deep(options);
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 4436c86361..7e79b89beb 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -65,6 +67,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
desc->tdrefcount = -1; /* assume not reference-counted */
+ desc->tdcompression = NULL;
return desc;
}
@@ -91,6 +94,27 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
return desc;
}
+static AttributeCompression *
+TupleDescCopyCompression(TupleDesc tupdesc)
+{
+ AttributeCompression *src = tupdesc->tdcompression;
+ AttributeCompression *dst = (AttributeCompression *)
+ palloc0(sizeof(*dst) * tupdesc->natts);
+ int i;
+
+ for (i = 0; i < tupdesc->natts; i++)
+ if (src[i].routine)
+ {
+ dst[i].routine = palloc(sizeof(*dst[i].routine));
+ memcpy(dst[i].routine, src[i].routine, sizeof(*dst[i].routine));
+
+ dst[i].cmoptoid = src[i].cmoptoid;
+ dst[i].options = copyObject(src[i].options);
+ }
+
+ return dst;
+}
+
/*
* CreateTupleDescCopy
* This function creates a new TupleDesc by copying from an existing
@@ -119,6 +143,9 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ if (tupdesc->tdcompression)
+ desc->tdcompression = TupleDescCopyCompression(tupdesc);
+
return desc;
}
@@ -178,6 +205,9 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->constr = cpy;
}
+ if (tupdesc->tdcompression)
+ desc->tdcompression = TupleDescCopyCompression(tupdesc);
+
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
@@ -226,6 +256,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -271,6 +302,23 @@ FreeTupleDesc(TupleDesc tupdesc)
pfree(tupdesc->constr);
}
+ if (tupdesc->tdcompression)
+ {
+ for (i = 0; i < tupdesc->natts; i++)
+ {
+ AttributeCompression *ac = &tupdesc->tdcompression[i];
+
+ if (ac->routine)
+ {
+ pfree(ac->routine);
+ if (ac->options)
+ freeRelOptions(ac->options);
+ }
+ }
+
+ pfree(tupdesc->tdcompression);
+ }
+
pfree(tupdesc);
}
@@ -380,6 +428,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -442,6 +492,22 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
+ if ((tupdesc1->tdcompression == NULL) != (tupdesc2->tdcompression == NULL))
+ return false;
+
+ if (tupdesc1->tdcompression == NULL)
+ return true;
+
+ for (i = 0; i < tupdesc1->natts; i++)
+ {
+ AttributeCompression *ac1 = &tupdesc1->tdcompression[i];
+ AttributeCompression *ac2 = &tupdesc2->tdcompression[i];
+
+ if (ac1->cmoptoid != ac2->cmoptoid)
+ return false;
+ }
+
return true;
}
@@ -547,6 +613,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -659,6 +726,26 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
+void
+TupleDescInitAttrCompression(TupleDesc desc,
+ AttrNumber attnum,
+ Oid cmoptoid)
+{
+ CompressionMethodRoutine *cmr;
+ AttributeCompression *ac;
+
+ cmr = GetCompressionRoutine(cmoptoid);
+
+ /* initialize array with compression routines */
+ if (!desc->tdcompression)
+ desc->tdcompression = (AttributeCompression *)
+ palloc0(desc->natts * sizeof(AttributeCompression));
+
+ ac = TupleDescAttrCompression(desc, attnum - 1);
+ ac->routine = cmr;
+ ac->options = GetCompressionOptionsList(cmoptoid);
+ ac->cmoptoid = cmoptoid;
+}
/*
* BuildDescForRelation
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 5a8f1dab83..694e16f103 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -53,19 +55,43 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ ((toast_compress_header *) (ptr))->info &= 0xC0000000; \
+ ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -773,7 +799,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttrCompression(tupleDesc, i));
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +941,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttrCompression(tupleDesc, i));
if (DatumGetPointer(new_value) != NULL)
{
@@ -1229,7 +1257,9 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
if (VARATT_IS_EXTERNAL(new_value) ||
VARATT_IS_COMPRESSED(new_value))
{
- new_value = heap_tuple_untoast_attr(new_value);
+ struct varlena *untoasted_value = heap_tuple_untoast_attr(new_value);
+
+ new_value = untoasted_value;
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
}
@@ -1353,7 +1383,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,11 +1397,12 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, AttributeCompression * ac)
{
struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ int32 valsize,
+ len = 0;
+ bool custom = (ac != NULL);
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
@@ -1381,12 +1411,27 @@ toast_compress_datum(Datum value)
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ if (!custom && (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size))
return PointerGetDatum(NULL);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ if (custom)
+ {
+ Assert(ac->routine);
+ tmp = ac->routine->compress(ac, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
* We recheck the actual size even if pglz_compress() reports success,
@@ -1398,11 +1443,7 @@ toast_compress_datum(Datum value)
* only one header byte and no padding if the value is short enough. So
* we insist on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!custom && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,6 +1451,14 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (custom && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, ac->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
@@ -2280,15 +2329,29 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionMethodRoutine *cmr;
+ toast_compress_header_custom *hdr = \
+ (toast_compress_header_custom *) attr;
+ List *options;
+
+ /* TODO: add some cache for options and routines */
+ cmr = GetCompressionRoutine(hdr->cmoptoid);
+ options = GetCompressionOptionsList(hdr->cmoptoid);
+ result = cmr->decompress(attr, options);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 0453fd4ac1..53feb17abc 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -718,6 +718,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6fffc290fa..faaf7fb7f1 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2459,6 +2471,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 2eebb061b7..36c7fa0484 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -449,6 +449,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 45ee9ac8b9..88fe220e2e 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -707,6 +712,13 @@ AddNewAttributeTuples(Oid new_rel_oid,
referenced.objectSubId = 0;
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+
+ if (OidIsValid(attr->attcompression))
+ {
+ ObjectAddressSet(referenced, CompressionOptRelationId,
+ attr->attcompression);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
}
/*
@@ -1453,6 +1465,22 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+static void
+DropAttributeCompression(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ bool attisdropped = att->attisdropped;
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ att->attisdropped = true;
+ cmr->drop(att, options);
+ att->attisdropped = attisdropped;
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1511,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ DropAttributeCompression(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1611,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ DropAttributeCompression(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1634,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c7b2f031f0..11dc107ab7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6cac2dfd1d..d2b5285a5a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 59ffd2104d..6e806b3334 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
@@ -118,6 +119,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(-1);
values[Anum_pg_type_typndims - 1] = Int32GetDatum(0);
values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(InvalidOid);
+ values[Anum_pg_type_typdefaultcm - 1] = ObjectIdGetDatum(InvalidOid);
nulls[Anum_pg_type_typdefaultbin - 1] = true;
nulls[Anum_pg_type_typdefault - 1] = true;
nulls[Anum_pg_type_typacl - 1] = true;
@@ -362,6 +364,7 @@ TypeCreate(Oid newTypeOid,
values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(typeMod);
values[Anum_pg_type_typndims - 1] = Int32GetDatum(typNDims);
values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(typeCollation);
+ values[Anum_pg_type_typdefaultcm - 1] = ObjectIdGetDatum(InvalidOid);
/*
* initialize the default binary value for this type. Check for nulls of
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..32cbcc3efe
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,485 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/compression.h"
+#include "access/reloptions.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine *get_compression_method_routine(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+LookupCompressionHandlerFunc(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+static ObjectAddress
+CreateCompressionMethod(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = LookupCompressionHandlerFunc(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ return CreateCompressionMethod(cmName, (List *) handlerEl->arg);
+}
+
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, List *options)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Oid result,
+ cmhandler;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+ ObjectAddress myself,
+ ref1,
+ ref2,
+ ref3;
+ CompressionMethodRoutine *routine;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmhandler = ((Form_pg_compression) GETSTRUCT(tuple))->cmhandler;
+ ReleaseSysCache(tuple);
+
+ routine = get_compression_method_routine(cmhandler, attr->atttypid);
+
+ if (routine->configure && options != NIL)
+ routine->configure(attr, options);
+
+ values[Anum_pg_compression_opt_cmid - 1] = ObjectIdGetDatum(cmid);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ if (options)
+ values[Anum_pg_compression_opt_cmoptions - 1] = optionListToArray(options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+ tuple = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ result = CatalogTupleInsert(rel, tuple);
+ heap_freetuple(tuple);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, result);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmhandler);
+ ObjectAddressSubSet(ref2, RelationRelationId, attr->attrelid, attr->attnum);
+ ObjectAddressSet(ref3, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&ref2, &myself, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref3, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return result;
+}
+
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+ ReleaseSysCache(tup);
+ heap_close(relation, RowExclusiveLock);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = NULL;
+ compression->methodOid = cmopt->cmid;
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression * c1, ColumnCompression * c2,
+ const char *attributeName)
+{
+ char *cmname1 = c1->methodName ? c1->methodName :
+ get_compression_method_name(c1->methodOid);
+ char *cmname2 = c2->methodName ? c2->methodName :
+ get_compression_method_name(c2->methodOid);
+
+ if (strcmp(cmname1, cmname2))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", cmname1, cmname2)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ Oid cmid;
+ char *result = NULL;
+ Form_pg_compression cmform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmid = ((Form_pg_compression_opt) GETSTRUCT(tup))->cmid;
+ ReleaseSysCache(tup);
+
+ /* now we can get the name */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ return result;
+}
+
+/* get_compression_options */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+CompressionMethodRoutine *
+GetCompressionRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return get_compression_method_routine(cmhandler, InvalidOid);
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+get_compression_method_routine(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index cfa3f059c2..8cef3be086 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2758,8 +2758,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e60210cb24..010bdb644b 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -112,7 +112,8 @@ create_ctas_internal(List *attrList, IntoClause *into)
* Create the relation. (This will error out if there's an existing view,
* so we don't need more code to complain if "replace" is false.)
*/
- intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL);
+ intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL,
+ NULL);
/*
* If necessary, create a TOAST table for the target table. Note that
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 938133bbe4..f520c7fe7b 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 62937124ef..857eb14b64 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -212,7 +212,8 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq)
stmt->tablespacename = NULL;
stmt->if_not_exists = seq->if_not_exists;
- address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL);
+ address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL,
+ NULL);
seqoid = address.objectId;
Assert(seqoid != InvalidOid);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index c8fc9cb7fe..a715093a66 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -456,6 +461,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression * compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -501,7 +508,8 @@ static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
*/
ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString)
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt)
{
char relname[NAMEDATALEN];
Oid namespaceId;
@@ -521,6 +529,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
static char *validnsps[] = HEAP_RELOPT_NAMESPACES;
Oid ofTypeId;
ObjectAddress address;
+ AlterTableStmt *alterStmt = NULL;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -722,6 +731,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ transformColumnCompression(colDef, stmt->relation, &alterStmt);
}
/*
@@ -876,6 +887,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
relation_close(rel, NoLock);
+ if (pAlterStmt)
+ *pAlterStmt = (Node *) alterStmt;
+ else
+ Assert(!alterStmt);
+
return address;
}
@@ -1565,6 +1581,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1899,6 +1916,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1926,6 +1956,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2135,6 +2168,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3234,6 +3274,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3724,6 +3765,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4063,6 +4110,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5274,6 +5326,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
+ if (!colDef->compression)
+ {
+ /* colDef->compression is handled in subsequent ALTER TABLE statement */
+ Oid cmid = get_base_typdefaultcm(typeTuple);
+
+ if (OidIsValid(cmid))
+ attribute.attcompression = CreateCompressionOptions(&attribute,
+ cmid, NULL);
+ }
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6139,8 +6203,8 @@ ATPrepSetStatistics(Relation rel, const char *colName, int16 colNum, Node *newVa
RelationGetRelationName(rel))));
/*
- * We allow referencing columns by numbers only for indexes, since
- * table column numbers could contain gaps if columns are later dropped.
+ * We allow referencing columns by numbers only for indexes, since table
+ * column numbers could contain gaps if columns are later dropped.
*/
if (rel->rd_rel->relkind != RELKIND_INDEX && !colName)
ereport(ERROR,
@@ -6374,6 +6438,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9019,6 +9088,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
SysScanDesc scan;
HeapTuple depTup;
ObjectAddress address;
+ Oid cmid;
attrelation = heap_open(AttributeRelationId, RowExclusiveLock);
@@ -9294,6 +9364,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9343,7 +9415,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9366,6 +9440,10 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ cmid = get_base_typdefaultcm(typeTuple);
+ if (OidIsValid(cmid))
+ attTup->attcompression = CreateCompressionOptions(attTup, cmid, NULL);
+
ReleaseSysCache(typeTuple);
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
@@ -12362,6 +12440,86 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression * compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName || OidIsValid(compression->methodOid))
+ {
+ /* SET COMPRESSED */
+ Oid cmid,
+ cmoptoid;
+
+ cmid = compression->methodName
+ ? get_compression_method_oid(compression->methodName, false)
+ : compression->methodOid;
+
+ cmoptoid = CreateCompressionOptions(atttableform, cmid, compression->options);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 7ed16aeff4..2d39f57ab0 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -2110,7 +2111,7 @@ DefineCompositeType(RangeVar *typevar, List *coldeflist)
* Finally create the relation. This also creates the type.
*/
DefineRelation(createStmt, RELKIND_COMPOSITE_TYPE, InvalidOid, &address,
- NULL);
+ NULL, NULL);
return address;
}
@@ -3638,3 +3639,95 @@ AlterTypeNamespaceInternal(Oid typeOid, Oid nspOid,
return oldNspOid;
}
+
+
+/*
+ * Execute ALTER TYPE SET COMPRESSED <cm> [WITH (<option>, ...)]
+ */
+static void
+AlterTypeDefaultCompression(Oid typeid, ColumnCompression * compression)
+{
+ Oid cmoid;
+ Type oldtup = typeidType(typeid);
+ Form_pg_type oldtype = (Form_pg_type) GETSTRUCT(oldtup);
+
+ cmoid = compression->methodName
+ ? get_compression_method_oid(compression->methodName, false)
+ : InvalidOid;
+
+ if (oldtype->typdefaultcm != cmoid)
+ {
+ Relation typrel;
+ HeapTuple newtup;
+ Datum values[Natts_pg_type];
+ bool nulls[Natts_pg_type];
+ bool replace[Natts_pg_type];
+
+ typrel = heap_open(TypeRelationId, RowExclusiveLock);
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replace, 0, sizeof(replace));
+
+ values[Anum_pg_type_typdefaultcm - 1] = ObjectIdGetDatum(cmoid);
+ nulls[Anum_pg_type_typdefaultcm - 1] = false;
+ replace[Anum_pg_type_typdefaultcm - 1] = true;
+
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(typrel),
+ values, nulls, replace);
+
+ CatalogTupleUpdate(typrel, &newtup->t_self, newtup);
+
+ heap_freetuple(newtup);
+ heap_close(typrel, RowExclusiveLock);
+
+ if (OidIsValid(oldtype->typdefaultcm))
+ deleteDependencyRecordsForClass(TypeRelationId, typeid,
+ CompressionMethodRelationId,
+ DEPENDENCY_NORMAL);
+
+ if (OidIsValid(cmoid))
+ {
+ ObjectAddress myself,
+ referenced;
+
+ ObjectAddressSet(myself, TypeRelationId, typeid);
+ ObjectAddressSet(referenced, CompressionMethodRelationId, cmoid);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
+
+ InvokeObjectPostAlterHook(TypeRelationId, typeid, 0);
+ }
+
+ ReleaseSysCache(oldtup);
+}
+
+/*
+ * Execute ALTER TYPE <typeName> <command>, ...
+ */
+void
+AlterType(AlterTypeStmt * stmt)
+{
+ TypeName *typename;
+ Oid typeid;
+ ListCell *lcmd;
+
+ /* Make a TypeName so we can use standard type lookup machinery */
+ typename = makeTypeNameFromNameList(stmt->typeName);
+ typeid = typenameTypeId(NULL, typename);
+
+ foreach(lcmd, stmt->cmds)
+ {
+ AlterTypeCmd *cmd = (AlterTypeCmd *) lfirst(lcmd);
+
+ switch (cmd->cmdtype)
+ {
+ case AT_AlterTypeCompression:
+ AlterTypeDefaultCompression(typeid,
+ (ColumnCompression *) cmd->def);
+ break;
+ default:
+ elog(ERROR, "unknown ALTER TYPE command %d", cmd->cmdtype);
+ }
+ }
+}
diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c
index 076e2a3a40..9af0f11856 100644
--- a/src/backend/commands/view.c
+++ b/src/backend/commands/view.c
@@ -251,7 +251,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace,
* false).
*/
address = DefineRelation(createStmt, RELKIND_VIEW, InvalidOid, NULL,
- NULL);
+ NULL, NULL);
Assert(address.objectId != InvalidOid);
/* Make the new view relation visible */
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 47a34a044a..a58af7ee92 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -300,6 +300,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
var->vartypmod != -1))
return false; /* type mismatch */
+ if (OidIsValid(att_tup->attcompression))
+ return false;
+
tlist_item = lnext(tlist_item);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 9bae2647fd..6d2d775ed3 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2805,6 +2805,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2823,6 +2824,19 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression * from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_SCALAR_FIELD(methodOid);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -4547,6 +4561,28 @@ _copyDropSubscriptionStmt(const DropSubscriptionStmt *from)
return newnode;
}
+static AlterTypeStmt *
+_copyAlterTypeStmt(const AlterTypeStmt * from)
+{
+ AlterTypeStmt *newnode = makeNode(AlterTypeStmt);
+
+ COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(cmds);
+
+ return newnode;
+}
+
+static AlterTypeCmd *
+_copyAlterTypeCmd(const AlterTypeCmd * from)
+{
+ AlterTypeCmd *newnode = makeNode(AlterTypeCmd);
+
+ COPY_SCALAR_FIELD(cmdtype);
+ COPY_NODE_FIELD(def);
+
+ return newnode;
+}
+
/* ****************************************************************
* pg_list.h copy functions
* ****************************************************************
@@ -5455,6 +5491,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
@@ -5538,6 +5577,14 @@ copyObjectImpl(const void *from)
retval = _copyForeignKeyCacheInfo(from);
break;
+ case T_AlterTypeStmt:
+ retval = _copyAlterTypeStmt(from);
+ break;
+
+ case T_AlterTypeCmd:
+ retval = _copyAlterTypeCmd(from);
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(from));
retval = 0; /* keep compiler quiet */
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 11731da80a..9b94330db6 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2536,6 +2536,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2554,6 +2555,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression * a, const ColumnCompression * b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_SCALAR_FIELD(methodOid);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -2866,6 +2877,24 @@ _equalPartitionCmd(const PartitionCmd *a, const PartitionCmd *b)
return true;
}
+static bool
+_equalAlterTypeStmt(const AlterTypeStmt * a, const AlterTypeStmt * b)
+{
+ COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(cmds);
+
+ return true;
+}
+
+static bool
+_equalAlterTypeCmd(const AlterTypeCmd * a, const AlterTypeCmd * b)
+{
+ COMPARE_SCALAR_FIELD(cmdtype);
+ COMPARE_NODE_FIELD(def);
+
+ return true;
+}
+
/*
* Stuff from pg_list.h
*/
@@ -3601,6 +3630,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
@@ -3676,6 +3708,12 @@ equal(const void *a, const void *b)
case T_PartitionCmd:
retval = _equalPartitionCmd(a, b);
break;
+ case T_AlterTypeStmt:
+ retval = _equalAlterTypeStmt(a, b);
+ break;
+ case T_AlterTypeCmd:
+ retval = _equalAlterTypeCmd(a, b);
+ break;
default:
elog(ERROR, "unrecognized node type: %d",
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index e3eb0c5788..c1bea10eff 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 9ee3e23761..a02f4b3fda 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2792,6 +2792,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2808,6 +2809,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression * node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_OID_FIELD(methodOid);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4096,6 +4107,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 5eb398118e..d06b098650 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -282,6 +282,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
+ AlterTypeStmt
%type <node> select_no_parens select_with_parens select_clause
simple_select values_clause
@@ -290,8 +291,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> add_drop opt_asc_desc opt_nulls_order
%type <node> alter_table_cmd alter_type_cmd opt_collate_clause
- replica_identity partition_cmd
-%type <list> alter_table_cmds alter_type_cmds
+ replica_identity partition_cmd alterTypeCmd
+%type <list> alter_table_cmds alter_type_cmds alterTypeCmds
%type <list> alter_identity_column_option_list
%type <defelt> alter_identity_column_option
@@ -396,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
%type <list> group_by_list
%type <node> group_by_item empty_grouping_set rollup_clause cube_clause
@@ -579,6 +581,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> partbound_datum PartitionRangeDatum
%type <list> partbound_datum_list range_datum_list
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -611,9 +615,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -840,6 +844,7 @@ stmt :
| AlterSubscriptionStmt
| AlterTSConfigurationStmt
| AlterTSDictionaryStmt
+ | AlterTypeStmt
| AlterUserMappingStmt
| AnalyzeStmt
| CheckPointStmt
@@ -2165,6 +2170,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -2720,6 +2734,32 @@ PartitionRangeDatum:
* really variants of the ALTER TABLE subcommands with different spellings
*****************************************************************************/
+AlterTypeStmt:
+ ALTER TYPE_P any_name alterTypeCmds
+ {
+ AlterTypeStmt *n = makeNode(AlterTypeStmt);
+ n->typeName = $3;
+ n->cmds = $4;
+ $$ = (Node *) n;
+ }
+ ;
+
+alterTypeCmds:
+ alterTypeCmd { $$ = list_make1($1); }
+ | alterTypeCmds ',' alterTypeCmd { $$ = lappend($1, $3); }
+ ;
+
+alterTypeCmd:
+ /* ALTER TYPE <name> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ SET columnCompression
+ {
+ AlterTypeCmd *n = makeNode(AlterTypeCmd);
+ n->cmdtype = AT_AlterTypeCompression;
+ n->def = $2;
+ $$ = (Node *)n;
+ }
+ ;
+
AlterCompositeTypeStmt:
ALTER TYPE_P any_name alter_type_cmds
{
@@ -3245,11 +3285,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3259,8 +3300,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3307,6 +3348,39 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->methodOid = InvalidOid;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->methodOid = InvalidOid;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5667,6 +5741,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6175,6 +6258,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6238,7 +6322,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6428,6 +6512,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14636,6 +14721,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 20586797cc..0b1f0ff9aa 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -493,6 +494,49 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (!column->compression && column->typeName)
+ {
+ Type tup = typenameType(NULL, column->typeName, NULL);
+ Oid cmoid = get_base_typdefaultcm(tup);
+
+ ReleaseSysCache(tup);
+
+ if (OidIsValid(cmoid))
+ {
+ column->compression = makeNode(ColumnCompression);
+ column->compression->methodName = NULL;
+ column->compression->methodOid = cmoid;
+ column->compression->options = NIL;
+ }
+ }
+
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -793,6 +837,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1002,6 +1056,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 775477c6cf..a07327fbf7 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -217,6 +217,7 @@ check_xact_readonly(Node *parsetree)
case T_CreateSubscriptionStmt:
case T_AlterSubscriptionStmt:
case T_DropSubscriptionStmt:
+ case T_AlterTypeStmt:
PreventCommandIfReadOnly(CreateCommandTag(parsetree));
PreventCommandIfParallelMode(CreateCommandTag(parsetree));
break;
@@ -998,6 +999,7 @@ ProcessUtilitySlow(ParseState *pstate,
foreach(l, stmts)
{
Node *stmt = (Node *) lfirst(l);
+ Node *alterStmt = NULL;
if (IsA(stmt, CreateStmt))
{
@@ -1008,7 +1010,9 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_RELATION,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1042,7 +1046,8 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_FOREIGN_TABLE,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
CreateForeignTable((CreateForeignTableStmt *) stmt,
address.objectId);
EventTriggerCollectSimpleCommand(address,
@@ -1074,6 +1079,9 @@ ProcessUtilitySlow(ParseState *pstate,
NULL);
}
+ if (alterStmt)
+ lappend(stmts, alterStmt);
+
/* Need CCI between commands */
if (lnext(l) != NULL)
CommandCounterIncrement();
@@ -1283,6 +1291,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -1643,6 +1656,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterCollation((AlterCollationStmt *) parsetree);
break;
+ case T_AlterTypeStmt:
+ AlterType((AlterTypeStmt *) parsetree);
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -1696,6 +1713,11 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_FOREIGN_TABLE:
RemoveRelations(stmt);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ if (stmt->behavior == DROP_CASCADE)
+ {
+ /* TODO decompress columns instead of their deletion */
+ }
default:
RemoveObjects(stmt);
break;
@@ -2309,6 +2331,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2437,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2846,6 +2874,10 @@ CreateCommandTag(Node *parsetree)
}
break;
+ case T_AlterTypeStmt:
+ tag = "ALTER TYPE";
+ break;
+
default:
elog(WARNING, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -3291,6 +3323,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_AlterTypeStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 6f66c1f58c..ec4e89bf94 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,92 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(AttributeCompression * ac, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static void
+tsvector_configure(Form_pg_attribute attr, List *options)
+{
+ if (options != NIL)
+ elog(ERROR, "the compression method for tsvector doesn't take any options");
+}
+
+static struct varlena *
+tsvector_decompress(const struct varlena *data, List *options)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = tsvector_configure;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index b7a14dc87e..ae7320707c 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2276,16 +2276,12 @@ getBaseType(Oid typid)
}
/*
- * getBaseTypeAndTypmod
- * If the given type is a domain, return its base type and typmod;
- * otherwise return the type's own OID, and leave *typmod unchanged.
- *
* Note that the "applied typmod" should be -1 for every domain level
* above the bottommost; therefore, if the passed-in typid is indeed
* a domain, *typmod should be -1.
*/
-Oid
-getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+static inline HeapTuple
+getBaseTypeTuple(Oid *typid, int32 *typmod)
{
/*
* We loop to find the bottom base type in a stack of domains.
@@ -2295,24 +2291,33 @@ getBaseTypeAndTypmod(Oid typid, int32 *typmod)
HeapTuple tup;
Form_pg_type typTup;
- tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*typid));
if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for type %u", typid);
+ elog(ERROR, "cache lookup failed for type %u", *typid);
typTup = (Form_pg_type) GETSTRUCT(tup);
if (typTup->typtype != TYPTYPE_DOMAIN)
- {
/* Not a domain, so done */
- ReleaseSysCache(tup);
- break;
- }
+ return tup;
Assert(*typmod == -1);
- typid = typTup->typbasetype;
+ *typid = typTup->typbasetype;
*typmod = typTup->typtypmod;
ReleaseSysCache(tup);
}
+}
+/*
+ * getBaseTypeAndTypmod
+ * If the given type is a domain, return its base type and typmod;
+ * otherwise return the type's own OID, and leave *typmod unchanged.
+ */
+Oid
+getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+{
+ HeapTuple tup = getBaseTypeTuple(&typid, typmod);
+
+ ReleaseSysCache(tup);
return typid;
}
@@ -2808,6 +2813,39 @@ type_is_collatable(Oid typid)
return OidIsValid(get_typcollation(typid));
}
+/*
+ * get_base_typdefaultcm
+ *
+ * Given the type tuple, return the base type's typdefaultcm attribute.
+ */
+Oid
+get_base_typdefaultcm(HeapTuple typtup)
+{
+ Oid typid;
+ Oid base;
+ Oid cm = InvalidOid;
+
+ for (typid = (Oid) -1; !OidIsValid(cm) && OidIsValid(typid); typid = base)
+ {
+ Form_pg_type type;
+
+ if (typid != (Oid) -1)
+ {
+ typtup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ if (!HeapTupleIsValid(typtup))
+ elog(ERROR, "cache lookup failed for type %u", typid);
+ }
+
+ type = (Form_pg_type) GETSTRUCT(typtup);
+ base = type->typtype == TYPTYPE_DOMAIN ? type->typbasetype : InvalidOid;
+ cm = type->typdefaultcm;
+
+ if (typid != (Oid) -1)
+ ReleaseSysCache(typtup);
+ }
+
+ return cm;
+}
/* ---------- STATISTICS CACHE ---------- */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index b8e37809b0..5f54fa0a3e 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/nbtree.h"
@@ -76,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -538,6 +540,7 @@ RelationBuildTupleDesc(Relation relation)
while (HeapTupleIsValid(pg_attribute_tuple = systable_getnext(pg_attribute_scan)))
{
Form_pg_attribute attp;
+ Oid cmoptoid;
attp = (Form_pg_attribute) GETSTRUCT(pg_attribute_tuple);
@@ -565,6 +568,16 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ cmoptoid = attp->attcompression;
+ if (!attp->attisdropped && OidIsValid(cmoptoid))
+ {
+ MemoryContext oldctx = MemoryContextSwitchTo(CacheMemoryContext);
+
+ TupleDescInitAttrCompression(relation->rd_att, attp->attnum, cmoptoid);
+ MemoryContextSwitchTo(oldctx);
+ }
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index fcbb683a99..634482e326 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 041b5e0c87..0ded023858 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6fb9bdd063..9939ca1cdb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1620,6 +1683,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ if (pset.sversion >= 100000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT cm.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " JOIN pg_catalog.pg_compression cm ON (cm.oid = c.cmid) "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1741,6 +1820,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1840,6 +1923,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1847,7 +1935,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1858,7 +1946,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index 4d1c0ec3c6..307f8bfbe5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 2ab8809fa5..2477df82e4 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..b969bff3c1
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,69 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ CompressionMethodRoutine *routine;
+ List *options;
+ Oid cmoptoid;
+} AttributeCompression;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef void (*CompressionDropRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (AttributeCompression * ac, const struct varlena *data);
+typedef struct varlena *(*DecompressionRoutine)
+ (const struct varlena *data, List *options);
+
+/*
+ * API struct for an compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionDropRoutine drop;
+ CompressionRoutine compress;
+ DecompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine * GetCompressionRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr, Oid cmid,
+ List *options);
+extern ColumnCompression * GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression * c1,
+ ColumnCompression * c2, const char *attributeName);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 5cdaa3bff1..573512367a 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, char *name, char *desc,
extern Datum transformRelOptions(Datum oldOptions, List *defList,
char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,9 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern void freeRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 989fe738bb..e2807276d0 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -76,12 +78,16 @@ typedef struct tupleDesc
bool tdhasoid; /* tuple has oid attribute in its header */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
+ AttributeCompression *tdcompression;
/* attrs[N] is the description of Attribute Number N+1 */
FormData_pg_attribute attrs[FLEXIBLE_ARRAY_MEMBER];
} *TupleDesc;
/* Accessor for the i'th attribute of tupdesc. */
#define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)])
+#define TupleDescAttrCompression(tupdesc, i) \
+ ((tupdesc)->tdcompression? &((tupdesc)->tdcompression[i]) : NULL)
+
extern TupleDesc CreateTemplateTupleDesc(int natts, bool hasoid);
@@ -134,6 +140,10 @@ extern void TupleDescInitEntryCollation(TupleDesc desc,
AttrNumber attributeNumber,
Oid collationid);
+extern void TupleDescInitAttrCompression(TupleDesc desc,
+ AttrNumber attnum,
+ Oid cmoptoid);
+
extern TupleDesc BuildDescForRelation(List *schema);
extern TupleDesc BuildDescFromLists(List *names, List *types, List *typmods, List *collations);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..955b6620cd 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -210,7 +210,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, AttributeCompression * ac);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..b580f1971a 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(oid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..04c7c18d70 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -147,9 +147,9 @@ typedef FormData_pg_class *Form_pg_class;
* Note: "3" in the relfrozenxid column stands for FirstNormalTransactionId;
* similarly, "1" in relminmxid stands for FirstMultiXactId
*/
-DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 31 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..343d3355d9
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420)
+{
+ Oid cmid; /* compression method oid */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 3
+#define Anum_pg_compression_opt_cmid 1
+#define Anum_pg_compression_opt_cmhandler 2
+#define Anum_pg_compression_opt_cmoptions 3
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index d820b56aa1..5a2be99f0b 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3874,6 +3874,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4676,6 +4680,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index ffdb452b02..23921db534 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -199,6 +199,9 @@ CATALOG(pg_type,1247) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71) BKI_SCHEMA_MACRO
*/
Oid typcollation;
+ /* Default compression method for the datatype or InvalidOid */
+ Oid typdefaultcm;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/*
@@ -236,7 +239,7 @@ typedef FormData_pg_type *Form_pg_type;
* compiler constants for pg_type
* ----------------
*/
-#define Natts_pg_type 30
+#define Natts_pg_type 31
#define Anum_pg_type_typname 1
#define Anum_pg_type_typnamespace 2
#define Anum_pg_type_typowner 3
@@ -264,9 +267,10 @@ typedef FormData_pg_type *Form_pg_type;
#define Anum_pg_type_typtypmod 25
#define Anum_pg_type_typndims 26
#define Anum_pg_type_typcollation 27
-#define Anum_pg_type_typdefaultbin 28
-#define Anum_pg_type_typdefault 29
-#define Anum_pg_type_typacl 30
+#define Anum_pg_type_typdefaultcm 28
+#define Anum_pg_type_typdefaultbin 29
+#define Anum_pg_type_typdefault 30
+#define Anum_pg_type_typacl 31
/* ----------------
@@ -283,102 +287,102 @@ typedef FormData_pg_type *Form_pg_type;
*/
/* OIDS 1 - 99 */
-DATA(insert OID = 16 ( bool PGNSP PGUID 1 t b B t t \054 0 0 1000 boolin boolout boolrecv boolsend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 16 ( bool PGNSP PGUID 1 t b B t t \054 0 0 1000 boolin boolout boolrecv boolsend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("boolean, 'true'/'false'");
#define BOOLOID 16
-DATA(insert OID = 17 ( bytea PGNSP PGUID -1 f b U f t \054 0 0 1001 byteain byteaout bytearecv byteasend - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 17 ( bytea PGNSP PGUID -1 f b U f t \054 0 0 1001 byteain byteaout bytearecv byteasend - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("variable-length string, binary values escaped");
#define BYTEAOID 17
-DATA(insert OID = 18 ( char PGNSP PGUID 1 t b S f t \054 0 0 1002 charin charout charrecv charsend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 18 ( char PGNSP PGUID 1 t b S f t \054 0 0 1002 charin charout charrecv charsend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("single character");
#define CHAROID 18
-DATA(insert OID = 19 ( name PGNSP PGUID NAMEDATALEN f b S f t \054 0 18 1003 namein nameout namerecv namesend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 19 ( name PGNSP PGUID NAMEDATALEN f b S f t \054 0 18 1003 namein nameout namerecv namesend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("63-byte type for storing system identifiers");
#define NAMEOID 19
-DATA(insert OID = 20 ( int8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 1016 int8in int8out int8recv int8send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 20 ( int8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 1016 int8in int8out int8recv int8send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("~18 digit integer, 8-byte storage");
#define INT8OID 20
-DATA(insert OID = 21 ( int2 PGNSP PGUID 2 t b N f t \054 0 0 1005 int2in int2out int2recv int2send - - - s p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 21 ( int2 PGNSP PGUID 2 t b N f t \054 0 0 1005 int2in int2out int2recv int2send - - - s p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("-32 thousand to 32 thousand, 2-byte storage");
#define INT2OID 21
-DATA(insert OID = 22 ( int2vector PGNSP PGUID -1 f b A f t \054 0 21 1006 int2vectorin int2vectorout int2vectorrecv int2vectorsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 22 ( int2vector PGNSP PGUID -1 f b A f t \054 0 21 1006 int2vectorin int2vectorout int2vectorrecv int2vectorsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("array of int2, used in system tables");
#define INT2VECTOROID 22
-DATA(insert OID = 23 ( int4 PGNSP PGUID 4 t b N f t \054 0 0 1007 int4in int4out int4recv int4send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 23 ( int4 PGNSP PGUID 4 t b N f t \054 0 0 1007 int4in int4out int4recv int4send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("-2 billion to 2 billion integer, 4-byte storage");
#define INT4OID 23
-DATA(insert OID = 24 ( regproc PGNSP PGUID 4 t b N f t \054 0 0 1008 regprocin regprocout regprocrecv regprocsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 24 ( regproc PGNSP PGUID 4 t b N f t \054 0 0 1008 regprocin regprocout regprocrecv regprocsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered procedure");
#define REGPROCOID 24
-DATA(insert OID = 25 ( text PGNSP PGUID -1 f b S t t \054 0 0 1009 textin textout textrecv textsend - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 25 ( text PGNSP PGUID -1 f b S t t \054 0 0 1009 textin textout textrecv textsend - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("variable-length string, no limit specified");
#define TEXTOID 25
-DATA(insert OID = 26 ( oid PGNSP PGUID 4 t b N t t \054 0 0 1028 oidin oidout oidrecv oidsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 26 ( oid PGNSP PGUID 4 t b N t t \054 0 0 1028 oidin oidout oidrecv oidsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("object identifier(oid), maximum 4 billion");
#define OIDOID 26
-DATA(insert OID = 27 ( tid PGNSP PGUID 6 f b U f t \054 0 0 1010 tidin tidout tidrecv tidsend - - - s p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 27 ( tid PGNSP PGUID 6 f b U f t \054 0 0 1010 tidin tidout tidrecv tidsend - - - s p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("(block, offset), physical location of tuple");
#define TIDOID 27
-DATA(insert OID = 28 ( xid PGNSP PGUID 4 t b U f t \054 0 0 1011 xidin xidout xidrecv xidsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 28 ( xid PGNSP PGUID 4 t b U f t \054 0 0 1011 xidin xidout xidrecv xidsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("transaction id");
#define XIDOID 28
-DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b U f t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b U f t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("command identifier type, sequence in transaction id");
#define CIDOID 29
-DATA(insert OID = 30 ( oidvector PGNSP PGUID -1 f b A f t \054 0 26 1013 oidvectorin oidvectorout oidvectorrecv oidvectorsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 30 ( oidvector PGNSP PGUID -1 f b A f t \054 0 26 1013 oidvectorin oidvectorout oidvectorrecv oidvectorsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("array of oids, used in system tables");
#define OIDVECTOROID 30
/* hand-built rowtype entries for bootstrapped catalogs */
/* NB: OIDs assigned here must match the BKI_ROWTYPE_OID declarations */
-DATA(insert OID = 71 ( pg_type PGNSP PGUID -1 f c C f t \054 1247 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 75 ( pg_attribute PGNSP PGUID -1 f c C f t \054 1249 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 81 ( pg_proc PGNSP PGUID -1 f c C f t \054 1255 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 83 ( pg_class PGNSP PGUID -1 f c C f t \054 1259 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 71 ( pg_type PGNSP PGUID -1 f c C f t \054 1247 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 75 ( pg_attribute PGNSP PGUID -1 f c C f t \054 1249 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 81 ( pg_proc PGNSP PGUID -1 f c C f t \054 1255 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 83 ( pg_class PGNSP PGUID -1 f c C f t \054 1259 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 100 - 199 */
-DATA(insert OID = 114 ( json PGNSP PGUID -1 f b U f t \054 0 0 199 json_in json_out json_recv json_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 114 ( json PGNSP PGUID -1 f b U f t \054 0 0 199 json_in json_out json_recv json_send - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define JSONOID 114
-DATA(insert OID = 142 ( xml PGNSP PGUID -1 f b U f t \054 0 0 143 xml_in xml_out xml_recv xml_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 142 ( xml PGNSP PGUID -1 f b U f t \054 0 0 143 xml_in xml_out xml_recv xml_send - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("XML content");
#define XMLOID 142
-DATA(insert OID = 143 ( _xml PGNSP PGUID -1 f b A f t \054 0 142 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 199 ( _json PGNSP PGUID -1 f b A f t \054 0 114 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 143 ( _xml PGNSP PGUID -1 f b A f t \054 0 142 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 199 ( _json PGNSP PGUID -1 f b A f t \054 0 114 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 194 ( pg_node_tree PGNSP PGUID -1 f b S f t \054 0 0 0 pg_node_tree_in pg_node_tree_out pg_node_tree_recv pg_node_tree_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 194 ( pg_node_tree PGNSP PGUID -1 f b S f t \054 0 0 0 pg_node_tree_in pg_node_tree_out pg_node_tree_recv pg_node_tree_send - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("string representing an internal node tree");
#define PGNODETREEOID 194
-DATA(insert OID = 3361 ( pg_ndistinct PGNSP PGUID -1 f b S f t \054 0 0 0 pg_ndistinct_in pg_ndistinct_out pg_ndistinct_recv pg_ndistinct_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 3361 ( pg_ndistinct PGNSP PGUID -1 f b S f t \054 0 0 0 pg_ndistinct_in pg_ndistinct_out pg_ndistinct_recv pg_ndistinct_send - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("multivariate ndistinct coefficients");
#define PGNDISTINCTOID 3361
-DATA(insert OID = 3402 ( pg_dependencies PGNSP PGUID -1 f b S f t \054 0 0 0 pg_dependencies_in pg_dependencies_out pg_dependencies_recv pg_dependencies_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 3402 ( pg_dependencies PGNSP PGUID -1 f b S f t \054 0 0 0 pg_dependencies_in pg_dependencies_out pg_dependencies_recv pg_dependencies_send - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("multivariate dependencies");
#define PGDEPENDENCIESOID 3402
-DATA(insert OID = 32 ( pg_ddl_command PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 pg_ddl_command_in pg_ddl_command_out pg_ddl_command_recv pg_ddl_command_send - - - ALIGNOF_POINTER p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 32 ( pg_ddl_command PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 pg_ddl_command_in pg_ddl_command_out pg_ddl_command_recv pg_ddl_command_send - - - ALIGNOF_POINTER p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("internal type for passing CollectedCommand");
#define PGDDLCOMMANDOID 32
/* OIDS 200 - 299 */
-DATA(insert OID = 210 ( smgr PGNSP PGUID 2 t b U f t \054 0 0 0 smgrin smgrout - - - - - s p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 210 ( smgr PGNSP PGUID 2 t b U f t \054 0 0 0 smgrin smgrout - - - - - s p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("storage manager");
/* OIDS 300 - 399 */
@@ -388,280 +392,280 @@ DESCR("storage manager");
/* OIDS 500 - 599 */
/* OIDS 600 - 699 */
-DATA(insert OID = 600 ( point PGNSP PGUID 16 f b G f t \054 0 701 1017 point_in point_out point_recv point_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 600 ( point PGNSP PGUID 16 f b G f t \054 0 701 1017 point_in point_out point_recv point_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric point '(x, y)'");
#define POINTOID 600
-DATA(insert OID = 601 ( lseg PGNSP PGUID 32 f b G f t \054 0 600 1018 lseg_in lseg_out lseg_recv lseg_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 601 ( lseg PGNSP PGUID 32 f b G f t \054 0 600 1018 lseg_in lseg_out lseg_recv lseg_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric line segment '(pt1,pt2)'");
#define LSEGOID 601
-DATA(insert OID = 602 ( path PGNSP PGUID -1 f b G f t \054 0 0 1019 path_in path_out path_recv path_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 602 ( path PGNSP PGUID -1 f b G f t \054 0 0 1019 path_in path_out path_recv path_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric path '(pt1,...)'");
#define PATHOID 602
-DATA(insert OID = 603 ( box PGNSP PGUID 32 f b G f t \073 0 600 1020 box_in box_out box_recv box_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 603 ( box PGNSP PGUID 32 f b G f t \073 0 600 1020 box_in box_out box_recv box_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric box '(lower left,upper right)'");
#define BOXOID 603
-DATA(insert OID = 604 ( polygon PGNSP PGUID -1 f b G f t \054 0 0 1027 poly_in poly_out poly_recv poly_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 604 ( polygon PGNSP PGUID -1 f b G f t \054 0 0 1027 poly_in poly_out poly_recv poly_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric polygon '(pt1,...)'");
#define POLYGONOID 604
-DATA(insert OID = 628 ( line PGNSP PGUID 24 f b G f t \054 0 701 629 line_in line_out line_recv line_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 628 ( line PGNSP PGUID 24 f b G f t \054 0 701 629 line_in line_out line_recv line_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric line");
#define LINEOID 628
-DATA(insert OID = 629 ( _line PGNSP PGUID -1 f b A f t \054 0 628 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 629 ( _line PGNSP PGUID -1 f b A f t \054 0 628 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 700 - 799 */
-DATA(insert OID = 700 ( float4 PGNSP PGUID 4 FLOAT4PASSBYVAL b N f t \054 0 0 1021 float4in float4out float4recv float4send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 700 ( float4 PGNSP PGUID 4 FLOAT4PASSBYVAL b N f t \054 0 0 1021 float4in float4out float4recv float4send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("single-precision floating point number, 4-byte storage");
#define FLOAT4OID 700
-DATA(insert OID = 701 ( float8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N t t \054 0 0 1022 float8in float8out float8recv float8send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 701 ( float8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N t t \054 0 0 1022 float8in float8out float8recv float8send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("double-precision floating point number, 8-byte storage");
#define FLOAT8OID 701
-DATA(insert OID = 702 ( abstime PGNSP PGUID 4 t b D f t \054 0 0 1023 abstimein abstimeout abstimerecv abstimesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 702 ( abstime PGNSP PGUID 4 t b D f t \054 0 0 1023 abstimein abstimeout abstimerecv abstimesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("absolute, limited-range date and time (Unix system time)");
#define ABSTIMEOID 702
-DATA(insert OID = 703 ( reltime PGNSP PGUID 4 t b T f t \054 0 0 1024 reltimein reltimeout reltimerecv reltimesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 703 ( reltime PGNSP PGUID 4 t b T f t \054 0 0 1024 reltimein reltimeout reltimerecv reltimesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("relative, limited-range time interval (Unix delta time)");
#define RELTIMEOID 703
-DATA(insert OID = 704 ( tinterval PGNSP PGUID 12 f b T f t \054 0 0 1025 tintervalin tintervalout tintervalrecv tintervalsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 704 ( tinterval PGNSP PGUID 12 f b T f t \054 0 0 1025 tintervalin tintervalout tintervalrecv tintervalsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("(abstime,abstime), time interval");
#define TINTERVALOID 704
-DATA(insert OID = 705 ( unknown PGNSP PGUID -2 f p X f t \054 0 0 0 unknownin unknownout unknownrecv unknownsend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 705 ( unknown PGNSP PGUID -2 f p X f t \054 0 0 0 unknownin unknownout unknownrecv unknownsend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("");
#define UNKNOWNOID 705
-DATA(insert OID = 718 ( circle PGNSP PGUID 24 f b G f t \054 0 0 719 circle_in circle_out circle_recv circle_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 718 ( circle PGNSP PGUID 24 f b G f t \054 0 0 719 circle_in circle_out circle_recv circle_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric circle '(center,radius)'");
#define CIRCLEOID 718
-DATA(insert OID = 719 ( _circle PGNSP PGUID -1 f b A f t \054 0 718 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 790 ( money PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 791 cash_in cash_out cash_recv cash_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 719 ( _circle PGNSP PGUID -1 f b A f t \054 0 718 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 790 ( money PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 791 cash_in cash_out cash_recv cash_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("monetary amounts, $d,ddd.cc");
#define CASHOID 790
-DATA(insert OID = 791 ( _money PGNSP PGUID -1 f b A f t \054 0 790 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 791 ( _money PGNSP PGUID -1 f b A f t \054 0 790 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 800 - 899 */
-DATA(insert OID = 829 ( macaddr PGNSP PGUID 6 f b U f t \054 0 0 1040 macaddr_in macaddr_out macaddr_recv macaddr_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 829 ( macaddr PGNSP PGUID 6 f b U f t \054 0 0 1040 macaddr_in macaddr_out macaddr_recv macaddr_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("XX:XX:XX:XX:XX:XX, MAC address");
#define MACADDROID 829
-DATA(insert OID = 869 ( inet PGNSP PGUID -1 f b I t t \054 0 0 1041 inet_in inet_out inet_recv inet_send - - - i m f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 869 ( inet PGNSP PGUID -1 f b I t t \054 0 0 1041 inet_in inet_out inet_recv inet_send - - - i m f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("IP address/netmask, host address, netmask optional");
#define INETOID 869
-DATA(insert OID = 650 ( cidr PGNSP PGUID -1 f b I f t \054 0 0 651 cidr_in cidr_out cidr_recv cidr_send - - - i m f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 650 ( cidr PGNSP PGUID -1 f b I f t \054 0 0 651 cidr_in cidr_out cidr_recv cidr_send - - - i m f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("network IP address/netmask, network address");
#define CIDROID 650
-DATA(insert OID = 774 ( macaddr8 PGNSP PGUID 8 f b U f t \054 0 0 775 macaddr8_in macaddr8_out macaddr8_recv macaddr8_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 774 ( macaddr8 PGNSP PGUID 8 f b U f t \054 0 0 775 macaddr8_in macaddr8_out macaddr8_recv macaddr8_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("XX:XX:XX:XX:XX:XX:XX:XX, MAC address");
#define MACADDR8OID 774
/* OIDS 900 - 999 */
/* OIDS 1000 - 1099 */
-DATA(insert OID = 1000 ( _bool PGNSP PGUID -1 f b A f t \054 0 16 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1001 ( _bytea PGNSP PGUID -1 f b A f t \054 0 17 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1002 ( _char PGNSP PGUID -1 f b A f t \054 0 18 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1003 ( _name PGNSP PGUID -1 f b A f t \054 0 19 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1005 ( _int2 PGNSP PGUID -1 f b A f t \054 0 21 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1000 ( _bool PGNSP PGUID -1 f b A f t \054 0 16 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1001 ( _bytea PGNSP PGUID -1 f b A f t \054 0 17 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1002 ( _char PGNSP PGUID -1 f b A f t \054 0 18 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1003 ( _name PGNSP PGUID -1 f b A f t \054 0 19 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1005 ( _int2 PGNSP PGUID -1 f b A f t \054 0 21 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INT2ARRAYOID 1005
-DATA(insert OID = 1006 ( _int2vector PGNSP PGUID -1 f b A f t \054 0 22 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1007 ( _int4 PGNSP PGUID -1 f b A f t \054 0 23 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1006 ( _int2vector PGNSP PGUID -1 f b A f t \054 0 22 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1007 ( _int4 PGNSP PGUID -1 f b A f t \054 0 23 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INT4ARRAYOID 1007
-DATA(insert OID = 1008 ( _regproc PGNSP PGUID -1 f b A f t \054 0 24 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1009 ( _text PGNSP PGUID -1 f b A f t \054 0 25 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1008 ( _regproc PGNSP PGUID -1 f b A f t \054 0 24 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1009 ( _text PGNSP PGUID -1 f b A f t \054 0 25 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
#define TEXTARRAYOID 1009
-DATA(insert OID = 1028 ( _oid PGNSP PGUID -1 f b A f t \054 0 26 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1028 ( _oid PGNSP PGUID -1 f b A f t \054 0 26 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define OIDARRAYOID 1028
-DATA(insert OID = 1010 ( _tid PGNSP PGUID -1 f b A f t \054 0 27 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1011 ( _xid PGNSP PGUID -1 f b A f t \054 0 28 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1012 ( _cid PGNSP PGUID -1 f b A f t \054 0 29 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1013 ( _oidvector PGNSP PGUID -1 f b A f t \054 0 30 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1014 ( _bpchar PGNSP PGUID -1 f b A f t \054 0 1042 0 array_in array_out array_recv array_send bpchartypmodin bpchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
-DATA(insert OID = 1015 ( _varchar PGNSP PGUID -1 f b A f t \054 0 1043 0 array_in array_out array_recv array_send varchartypmodin varchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
-DATA(insert OID = 1016 ( _int8 PGNSP PGUID -1 f b A f t \054 0 20 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1017 ( _point PGNSP PGUID -1 f b A f t \054 0 600 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1018 ( _lseg PGNSP PGUID -1 f b A f t \054 0 601 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1019 ( _path PGNSP PGUID -1 f b A f t \054 0 602 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1020 ( _box PGNSP PGUID -1 f b A f t \073 0 603 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1021 ( _float4 PGNSP PGUID -1 f b A f t \054 0 700 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1010 ( _tid PGNSP PGUID -1 f b A f t \054 0 27 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1011 ( _xid PGNSP PGUID -1 f b A f t \054 0 28 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1012 ( _cid PGNSP PGUID -1 f b A f t \054 0 29 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1013 ( _oidvector PGNSP PGUID -1 f b A f t \054 0 30 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1014 ( _bpchar PGNSP PGUID -1 f b A f t \054 0 1042 0 array_in array_out array_recv array_send bpchartypmodin bpchartypmodout array_typanalyze i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1015 ( _varchar PGNSP PGUID -1 f b A f t \054 0 1043 0 array_in array_out array_recv array_send varchartypmodin varchartypmodout array_typanalyze i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1016 ( _int8 PGNSP PGUID -1 f b A f t \054 0 20 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1017 ( _point PGNSP PGUID -1 f b A f t \054 0 600 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1018 ( _lseg PGNSP PGUID -1 f b A f t \054 0 601 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1019 ( _path PGNSP PGUID -1 f b A f t \054 0 602 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1020 ( _box PGNSP PGUID -1 f b A f t \073 0 603 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1021 ( _float4 PGNSP PGUID -1 f b A f t \054 0 700 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define FLOAT4ARRAYOID 1021
-DATA(insert OID = 1022 ( _float8 PGNSP PGUID -1 f b A f t \054 0 701 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1023 ( _abstime PGNSP PGUID -1 f b A f t \054 0 702 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1024 ( _reltime PGNSP PGUID -1 f b A f t \054 0 703 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1025 ( _tinterval PGNSP PGUID -1 f b A f t \054 0 704 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1027 ( _polygon PGNSP PGUID -1 f b A f t \054 0 604 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1033 ( aclitem PGNSP PGUID 12 f b U f t \054 0 0 1034 aclitemin aclitemout - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1022 ( _float8 PGNSP PGUID -1 f b A f t \054 0 701 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1023 ( _abstime PGNSP PGUID -1 f b A f t \054 0 702 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1024 ( _reltime PGNSP PGUID -1 f b A f t \054 0 703 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1025 ( _tinterval PGNSP PGUID -1 f b A f t \054 0 704 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1027 ( _polygon PGNSP PGUID -1 f b A f t \054 0 604 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1033 ( aclitem PGNSP PGUID 12 f b U f t \054 0 0 1034 aclitemin aclitemout - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("access control list");
#define ACLITEMOID 1033
-DATA(insert OID = 1034 ( _aclitem PGNSP PGUID -1 f b A f t \054 0 1033 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1040 ( _macaddr PGNSP PGUID -1 f b A f t \054 0 829 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 775 ( _macaddr8 PGNSP PGUID -1 f b A f t \054 0 774 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1041 ( _inet PGNSP PGUID -1 f b A f t \054 0 869 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 651 ( _cidr PGNSP PGUID -1 f b A f t \054 0 650 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1263 ( _cstring PGNSP PGUID -1 f b A f t \054 0 2275 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1034 ( _aclitem PGNSP PGUID -1 f b A f t \054 0 1033 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1040 ( _macaddr PGNSP PGUID -1 f b A f t \054 0 829 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 775 ( _macaddr8 PGNSP PGUID -1 f b A f t \054 0 774 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1041 ( _inet PGNSP PGUID -1 f b A f t \054 0 869 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 651 ( _cidr PGNSP PGUID -1 f b A f t \054 0 650 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1263 ( _cstring PGNSP PGUID -1 f b A f t \054 0 2275 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define CSTRINGARRAYOID 1263
-DATA(insert OID = 1042 ( bpchar PGNSP PGUID -1 f b S f t \054 0 0 1014 bpcharin bpcharout bpcharrecv bpcharsend bpchartypmodin bpchartypmodout - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1042 ( bpchar PGNSP PGUID -1 f b S f t \054 0 0 1014 bpcharin bpcharout bpcharrecv bpcharsend bpchartypmodin bpchartypmodout - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("char(length), blank-padded string, fixed storage length");
#define BPCHAROID 1042
-DATA(insert OID = 1043 ( varchar PGNSP PGUID -1 f b S f t \054 0 0 1015 varcharin varcharout varcharrecv varcharsend varchartypmodin varchartypmodout - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1043 ( varchar PGNSP PGUID -1 f b S f t \054 0 0 1015 varcharin varcharout varcharrecv varcharsend varchartypmodin varchartypmodout - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("varchar(length), non-blank-padded string, variable storage length");
#define VARCHAROID 1043
-DATA(insert OID = 1082 ( date PGNSP PGUID 4 t b D f t \054 0 0 1182 date_in date_out date_recv date_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1082 ( date PGNSP PGUID 4 t b D f t \054 0 0 1182 date_in date_out date_recv date_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("date");
#define DATEOID 1082
-DATA(insert OID = 1083 ( time PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1183 time_in time_out time_recv time_send timetypmodin timetypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1083 ( time PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1183 time_in time_out time_recv time_send timetypmodin timetypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("time of day");
#define TIMEOID 1083
/* OIDS 1100 - 1199 */
-DATA(insert OID = 1114 ( timestamp PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1115 timestamp_in timestamp_out timestamp_recv timestamp_send timestamptypmodin timestamptypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1114 ( timestamp PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1115 timestamp_in timestamp_out timestamp_recv timestamp_send timestamptypmodin timestamptypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("date and time");
#define TIMESTAMPOID 1114
-DATA(insert OID = 1115 ( _timestamp PGNSP PGUID -1 f b A f t \054 0 1114 0 array_in array_out array_recv array_send timestamptypmodin timestamptypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1182 ( _date PGNSP PGUID -1 f b A f t \054 0 1082 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1183 ( _time PGNSP PGUID -1 f b A f t \054 0 1083 0 array_in array_out array_recv array_send timetypmodin timetypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1184 ( timestamptz PGNSP PGUID 8 FLOAT8PASSBYVAL b D t t \054 0 0 1185 timestamptz_in timestamptz_out timestamptz_recv timestamptz_send timestamptztypmodin timestamptztypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1115 ( _timestamp PGNSP PGUID -1 f b A f t \054 0 1114 0 array_in array_out array_recv array_send timestamptypmodin timestamptypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1182 ( _date PGNSP PGUID -1 f b A f t \054 0 1082 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1183 ( _time PGNSP PGUID -1 f b A f t \054 0 1083 0 array_in array_out array_recv array_send timetypmodin timetypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1184 ( timestamptz PGNSP PGUID 8 FLOAT8PASSBYVAL b D t t \054 0 0 1185 timestamptz_in timestamptz_out timestamptz_recv timestamptz_send timestamptztypmodin timestamptztypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("date and time with time zone");
#define TIMESTAMPTZOID 1184
-DATA(insert OID = 1185 ( _timestamptz PGNSP PGUID -1 f b A f t \054 0 1184 0 array_in array_out array_recv array_send timestamptztypmodin timestamptztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1186 ( interval PGNSP PGUID 16 f b T t t \054 0 0 1187 interval_in interval_out interval_recv interval_send intervaltypmodin intervaltypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1185 ( _timestamptz PGNSP PGUID -1 f b A f t \054 0 1184 0 array_in array_out array_recv array_send timestamptztypmodin timestamptztypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1186 ( interval PGNSP PGUID 16 f b T t t \054 0 0 1187 interval_in interval_out interval_recv interval_send intervaltypmodin intervaltypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("@ <number> <units>, time interval");
#define INTERVALOID 1186
-DATA(insert OID = 1187 ( _interval PGNSP PGUID -1 f b A f t \054 0 1186 0 array_in array_out array_recv array_send intervaltypmodin intervaltypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1187 ( _interval PGNSP PGUID -1 f b A f t \054 0 1186 0 array_in array_out array_recv array_send intervaltypmodin intervaltypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 1200 - 1299 */
-DATA(insert OID = 1231 ( _numeric PGNSP PGUID -1 f b A f t \054 0 1700 0 array_in array_out array_recv array_send numerictypmodin numerictypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1266 ( timetz PGNSP PGUID 12 f b D f t \054 0 0 1270 timetz_in timetz_out timetz_recv timetz_send timetztypmodin timetztypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1231 ( _numeric PGNSP PGUID -1 f b A f t \054 0 1700 0 array_in array_out array_recv array_send numerictypmodin numerictypmodout array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1266 ( timetz PGNSP PGUID 12 f b D f t \054 0 0 1270 timetz_in timetz_out timetz_recv timetz_send timetztypmodin timetztypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("time of day with time zone");
#define TIMETZOID 1266
-DATA(insert OID = 1270 ( _timetz PGNSP PGUID -1 f b A f t \054 0 1266 0 array_in array_out array_recv array_send timetztypmodin timetztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1270 ( _timetz PGNSP PGUID -1 f b A f t \054 0 1266 0 array_in array_out array_recv array_send timetztypmodin timetztypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 1500 - 1599 */
-DATA(insert OID = 1560 ( bit PGNSP PGUID -1 f b V f t \054 0 0 1561 bit_in bit_out bit_recv bit_send bittypmodin bittypmodout - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1560 ( bit PGNSP PGUID -1 f b V f t \054 0 0 1561 bit_in bit_out bit_recv bit_send bittypmodin bittypmodout - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("fixed-length bit string");
#define BITOID 1560
-DATA(insert OID = 1561 ( _bit PGNSP PGUID -1 f b A f t \054 0 1560 0 array_in array_out array_recv array_send bittypmodin bittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1562 ( varbit PGNSP PGUID -1 f b V t t \054 0 0 1563 varbit_in varbit_out varbit_recv varbit_send varbittypmodin varbittypmodout - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1561 ( _bit PGNSP PGUID -1 f b A f t \054 0 1560 0 array_in array_out array_recv array_send bittypmodin bittypmodout array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1562 ( varbit PGNSP PGUID -1 f b V t t \054 0 0 1563 varbit_in varbit_out varbit_recv varbit_send varbittypmodin varbittypmodout - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("variable-length bit string");
#define VARBITOID 1562
-DATA(insert OID = 1563 ( _varbit PGNSP PGUID -1 f b A f t \054 0 1562 0 array_in array_out array_recv array_send varbittypmodin varbittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1563 ( _varbit PGNSP PGUID -1 f b A f t \054 0 1562 0 array_in array_out array_recv array_send varbittypmodin varbittypmodout array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 1600 - 1699 */
/* OIDS 1700 - 1799 */
-DATA(insert OID = 1700 ( numeric PGNSP PGUID -1 f b N f t \054 0 0 1231 numeric_in numeric_out numeric_recv numeric_send numerictypmodin numerictypmodout - i m f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1700 ( numeric PGNSP PGUID -1 f b N f t \054 0 0 1231 numeric_in numeric_out numeric_recv numeric_send numerictypmodin numerictypmodout - i m f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("numeric(precision, decimal), arbitrary precision number");
#define NUMERICOID 1700
-DATA(insert OID = 1790 ( refcursor PGNSP PGUID -1 f b U f t \054 0 0 2201 textin textout textrecv textsend - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1790 ( refcursor PGNSP PGUID -1 f b U f t \054 0 0 2201 textin textout textrecv textsend - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("reference to cursor (portal name)");
#define REFCURSOROID 1790
/* OIDS 2200 - 2299 */
-DATA(insert OID = 2201 ( _refcursor PGNSP PGUID -1 f b A f t \054 0 1790 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2201 ( _refcursor PGNSP PGUID -1 f b A f t \054 0 1790 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2202 ( regprocedure PGNSP PGUID 4 t b N f t \054 0 0 2207 regprocedurein regprocedureout regprocedurerecv regproceduresend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2202 ( regprocedure PGNSP PGUID 4 t b N f t \054 0 0 2207 regprocedurein regprocedureout regprocedurerecv regproceduresend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered procedure (with args)");
#define REGPROCEDUREOID 2202
-DATA(insert OID = 2203 ( regoper PGNSP PGUID 4 t b N f t \054 0 0 2208 regoperin regoperout regoperrecv regopersend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2203 ( regoper PGNSP PGUID 4 t b N f t \054 0 0 2208 regoperin regoperout regoperrecv regopersend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered operator");
#define REGOPEROID 2203
-DATA(insert OID = 2204 ( regoperator PGNSP PGUID 4 t b N f t \054 0 0 2209 regoperatorin regoperatorout regoperatorrecv regoperatorsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2204 ( regoperator PGNSP PGUID 4 t b N f t \054 0 0 2209 regoperatorin regoperatorout regoperatorrecv regoperatorsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered operator (with args)");
#define REGOPERATOROID 2204
-DATA(insert OID = 2205 ( regclass PGNSP PGUID 4 t b N f t \054 0 0 2210 regclassin regclassout regclassrecv regclasssend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2205 ( regclass PGNSP PGUID 4 t b N f t \054 0 0 2210 regclassin regclassout regclassrecv regclasssend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered class");
#define REGCLASSOID 2205
-DATA(insert OID = 2206 ( regtype PGNSP PGUID 4 t b N f t \054 0 0 2211 regtypein regtypeout regtyperecv regtypesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2206 ( regtype PGNSP PGUID 4 t b N f t \054 0 0 2211 regtypein regtypeout regtyperecv regtypesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered type");
#define REGTYPEOID 2206
-DATA(insert OID = 4096 ( regrole PGNSP PGUID 4 t b N f t \054 0 0 4097 regrolein regroleout regrolerecv regrolesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4096 ( regrole PGNSP PGUID 4 t b N f t \054 0 0 4097 regrolein regroleout regrolerecv regrolesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered role");
#define REGROLEOID 4096
-DATA(insert OID = 4089 ( regnamespace PGNSP PGUID 4 t b N f t \054 0 0 4090 regnamespacein regnamespaceout regnamespacerecv regnamespacesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4089 ( regnamespace PGNSP PGUID 4 t b N f t \054 0 0 4090 regnamespacein regnamespaceout regnamespacerecv regnamespacesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered namespace");
#define REGNAMESPACEOID 4089
-DATA(insert OID = 2207 ( _regprocedure PGNSP PGUID -1 f b A f t \054 0 2202 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2208 ( _regoper PGNSP PGUID -1 f b A f t \054 0 2203 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2209 ( _regoperator PGNSP PGUID -1 f b A f t \054 0 2204 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2210 ( _regclass PGNSP PGUID -1 f b A f t \054 0 2205 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2211 ( _regtype PGNSP PGUID -1 f b A f t \054 0 2206 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2207 ( _regprocedure PGNSP PGUID -1 f b A f t \054 0 2202 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2208 ( _regoper PGNSP PGUID -1 f b A f t \054 0 2203 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2209 ( _regoperator PGNSP PGUID -1 f b A f t \054 0 2204 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2210 ( _regclass PGNSP PGUID -1 f b A f t \054 0 2205 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2211 ( _regtype PGNSP PGUID -1 f b A f t \054 0 2206 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define REGTYPEARRAYOID 2211
-DATA(insert OID = 4097 ( _regrole PGNSP PGUID -1 f b A f t \054 0 4096 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 4090 ( _regnamespace PGNSP PGUID -1 f b A f t \054 0 4089 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4097 ( _regrole PGNSP PGUID -1 f b A f t \054 0 4096 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4090 ( _regnamespace PGNSP PGUID -1 f b A f t \054 0 4089 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* uuid */
-DATA(insert OID = 2950 ( uuid PGNSP PGUID 16 f b U f t \054 0 0 2951 uuid_in uuid_out uuid_recv uuid_send - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2950 ( uuid PGNSP PGUID 16 f b U f t \054 0 0 2951 uuid_in uuid_out uuid_recv uuid_send - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("UUID datatype");
#define UUIDOID 2950
-DATA(insert OID = 2951 ( _uuid PGNSP PGUID -1 f b A f t \054 0 2950 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2951 ( _uuid PGNSP PGUID -1 f b A f t \054 0 2950 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* pg_lsn */
-DATA(insert OID = 3220 ( pg_lsn PGNSP PGUID 8 FLOAT8PASSBYVAL b U f t \054 0 0 3221 pg_lsn_in pg_lsn_out pg_lsn_recv pg_lsn_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3220 ( pg_lsn PGNSP PGUID 8 FLOAT8PASSBYVAL b U f t \054 0 0 3221 pg_lsn_in pg_lsn_out pg_lsn_recv pg_lsn_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("PostgreSQL LSN datatype");
#define LSNOID 3220
-DATA(insert OID = 3221 ( _pg_lsn PGNSP PGUID -1 f b A f t \054 0 3220 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3221 ( _pg_lsn PGNSP PGUID -1 f b A f t \054 0 3220 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* text search */
-DATA(insert OID = 3614 ( tsvector PGNSP PGUID -1 f b U f t \054 0 0 3643 tsvectorin tsvectorout tsvectorrecv tsvectorsend - - ts_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3614 ( tsvector PGNSP PGUID -1 f b U f t \054 0 0 3643 tsvectorin tsvectorout tsvectorrecv tsvectorsend - - ts_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("text representation for text search");
#define TSVECTOROID 3614
-DATA(insert OID = 3642 ( gtsvector PGNSP PGUID -1 f b U f t \054 0 0 3644 gtsvectorin gtsvectorout - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3642 ( gtsvector PGNSP PGUID -1 f b U f t \054 0 0 3644 gtsvectorin gtsvectorout - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("GiST index internal text representation for text search");
#define GTSVECTOROID 3642
-DATA(insert OID = 3615 ( tsquery PGNSP PGUID -1 f b U f t \054 0 0 3645 tsqueryin tsqueryout tsqueryrecv tsquerysend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3615 ( tsquery PGNSP PGUID -1 f b U f t \054 0 0 3645 tsqueryin tsqueryout tsqueryrecv tsquerysend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("query representation for text search");
#define TSQUERYOID 3615
-DATA(insert OID = 3734 ( regconfig PGNSP PGUID 4 t b N f t \054 0 0 3735 regconfigin regconfigout regconfigrecv regconfigsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3734 ( regconfig PGNSP PGUID 4 t b N f t \054 0 0 3735 regconfigin regconfigout regconfigrecv regconfigsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered text search configuration");
#define REGCONFIGOID 3734
-DATA(insert OID = 3769 ( regdictionary PGNSP PGUID 4 t b N f t \054 0 0 3770 regdictionaryin regdictionaryout regdictionaryrecv regdictionarysend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3769 ( regdictionary PGNSP PGUID 4 t b N f t \054 0 0 3770 regdictionaryin regdictionaryout regdictionaryrecv regdictionarysend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered text search dictionary");
#define REGDICTIONARYOID 3769
-DATA(insert OID = 3643 ( _tsvector PGNSP PGUID -1 f b A f t \054 0 3614 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3644 ( _gtsvector PGNSP PGUID -1 f b A f t \054 0 3642 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3645 ( _tsquery PGNSP PGUID -1 f b A f t \054 0 3615 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3735 ( _regconfig PGNSP PGUID -1 f b A f t \054 0 3734 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3770 ( _regdictionary PGNSP PGUID -1 f b A f t \054 0 3769 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3643 ( _tsvector PGNSP PGUID -1 f b A f t \054 0 3614 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3644 ( _gtsvector PGNSP PGUID -1 f b A f t \054 0 3642 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3645 ( _tsquery PGNSP PGUID -1 f b A f t \054 0 3615 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3735 ( _regconfig PGNSP PGUID -1 f b A f t \054 0 3734 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3770 ( _regdictionary PGNSP PGUID -1 f b A f t \054 0 3769 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* jsonb */
-DATA(insert OID = 3802 ( jsonb PGNSP PGUID -1 f b U f t \054 0 0 3807 jsonb_in jsonb_out jsonb_recv jsonb_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3802 ( jsonb PGNSP PGUID -1 f b U f t \054 0 0 3807 jsonb_in jsonb_out jsonb_recv jsonb_send - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("Binary JSON");
#define JSONBOID 3802
-DATA(insert OID = 3807 ( _jsonb PGNSP PGUID -1 f b A f t \054 0 3802 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3807 ( _jsonb PGNSP PGUID -1 f b A f t \054 0 3802 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2970 ( txid_snapshot PGNSP PGUID -1 f b U f t \054 0 0 2949 txid_snapshot_in txid_snapshot_out txid_snapshot_recv txid_snapshot_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2970 ( txid_snapshot PGNSP PGUID -1 f b U f t \054 0 0 2949 txid_snapshot_in txid_snapshot_out txid_snapshot_recv txid_snapshot_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("txid snapshot");
-DATA(insert OID = 2949 ( _txid_snapshot PGNSP PGUID -1 f b A f t \054 0 2970 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2949 ( _txid_snapshot PGNSP PGUID -1 f b A f t \054 0 2970 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* range types */
-DATA(insert OID = 3904 ( int4range PGNSP PGUID -1 f r R f t \054 0 0 3905 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3904 ( int4range PGNSP PGUID -1 f r R f t \054 0 0 3905 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of integers");
#define INT4RANGEOID 3904
-DATA(insert OID = 3905 ( _int4range PGNSP PGUID -1 f b A f t \054 0 3904 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3906 ( numrange PGNSP PGUID -1 f r R f t \054 0 0 3907 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3905 ( _int4range PGNSP PGUID -1 f b A f t \054 0 3904 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3906 ( numrange PGNSP PGUID -1 f r R f t \054 0 0 3907 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of numerics");
-DATA(insert OID = 3907 ( _numrange PGNSP PGUID -1 f b A f t \054 0 3906 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3908 ( tsrange PGNSP PGUID -1 f r R f t \054 0 0 3909 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3907 ( _numrange PGNSP PGUID -1 f b A f t \054 0 3906 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3908 ( tsrange PGNSP PGUID -1 f r R f t \054 0 0 3909 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of timestamps without time zone");
-DATA(insert OID = 3909 ( _tsrange PGNSP PGUID -1 f b A f t \054 0 3908 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3910 ( tstzrange PGNSP PGUID -1 f r R f t \054 0 0 3911 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3909 ( _tsrange PGNSP PGUID -1 f b A f t \054 0 3908 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3910 ( tstzrange PGNSP PGUID -1 f r R f t \054 0 0 3911 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of timestamps with time zone");
-DATA(insert OID = 3911 ( _tstzrange PGNSP PGUID -1 f b A f t \054 0 3910 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3912 ( daterange PGNSP PGUID -1 f r R f t \054 0 0 3913 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3911 ( _tstzrange PGNSP PGUID -1 f b A f t \054 0 3910 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3912 ( daterange PGNSP PGUID -1 f r R f t \054 0 0 3913 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of dates");
-DATA(insert OID = 3913 ( _daterange PGNSP PGUID -1 f b A f t \054 0 3912 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3926 ( int8range PGNSP PGUID -1 f r R f t \054 0 0 3927 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3913 ( _daterange PGNSP PGUID -1 f b A f t \054 0 3912 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3926 ( int8range PGNSP PGUID -1 f r R f t \054 0 0 3927 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of bigints");
-DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/*
* pseudo-types
@@ -676,42 +680,44 @@ DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 arr
* but there is now support for it in records and arrays. Perhaps we should
* just treat it as a regular base type?
*/
-DATA(insert OID = 2249 ( record PGNSP PGUID -1 f p P f t \054 0 0 2287 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2249 ( record PGNSP PGUID -1 f p P f t \054 0 0 2287 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define RECORDOID 2249
-DATA(insert OID = 2287 ( _record PGNSP PGUID -1 f p P f t \054 0 2249 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2287 ( _record PGNSP PGUID -1 f p P f t \054 0 2249 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define RECORDARRAYOID 2287
-DATA(insert OID = 2275 ( cstring PGNSP PGUID -2 f p P f t \054 0 0 1263 cstring_in cstring_out cstring_recv cstring_send - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2275 ( cstring PGNSP PGUID -2 f p P f t \054 0 0 1263 cstring_in cstring_out cstring_recv cstring_send - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define CSTRINGOID 2275
-DATA(insert OID = 2276 ( any PGNSP PGUID 4 t p P f t \054 0 0 0 any_in any_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2276 ( any PGNSP PGUID 4 t p P f t \054 0 0 0 any_in any_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYOID 2276
-DATA(insert OID = 2277 ( anyarray PGNSP PGUID -1 f p P f t \054 0 0 0 anyarray_in anyarray_out anyarray_recv anyarray_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2277 ( anyarray PGNSP PGUID -1 f p P f t \054 0 0 0 anyarray_in anyarray_out anyarray_recv anyarray_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYARRAYOID 2277
-DATA(insert OID = 2278 ( void PGNSP PGUID 4 t p P f t \054 0 0 0 void_in void_out void_recv void_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2278 ( void PGNSP PGUID 4 t p P f t \054 0 0 0 void_in void_out void_recv void_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define VOIDOID 2278
-DATA(insert OID = 2279 ( trigger PGNSP PGUID 4 t p P f t \054 0 0 0 trigger_in trigger_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2279 ( trigger PGNSP PGUID 4 t p P f t \054 0 0 0 trigger_in trigger_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define TRIGGEROID 2279
-DATA(insert OID = 3838 ( event_trigger PGNSP PGUID 4 t p P f t \054 0 0 0 event_trigger_in event_trigger_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3838 ( event_trigger PGNSP PGUID 4 t p P f t \054 0 0 0 event_trigger_in event_trigger_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define EVTTRIGGEROID 3838
-DATA(insert OID = 2280 ( language_handler PGNSP PGUID 4 t p P f t \054 0 0 0 language_handler_in language_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2280 ( language_handler PGNSP PGUID 4 t p P f t \054 0 0 0 language_handler_in language_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define LANGUAGE_HANDLEROID 2280
-DATA(insert OID = 2281 ( internal PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 internal_in internal_out - - - - - ALIGNOF_POINTER p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2281 ( internal PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 internal_in internal_out - - - - - ALIGNOF_POINTER p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INTERNALOID 2281
-DATA(insert OID = 2282 ( opaque PGNSP PGUID 4 t p P f t \054 0 0 0 opaque_in opaque_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2282 ( opaque PGNSP PGUID 4 t p P f t \054 0 0 0 opaque_in opaque_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define OPAQUEOID 2282
-DATA(insert OID = 2283 ( anyelement PGNSP PGUID 4 t p P f t \054 0 0 0 anyelement_in anyelement_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2283 ( anyelement PGNSP PGUID 4 t p P f t \054 0 0 0 anyelement_in anyelement_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYELEMENTOID 2283
-DATA(insert OID = 2776 ( anynonarray PGNSP PGUID 4 t p P f t \054 0 0 0 anynonarray_in anynonarray_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2776 ( anynonarray PGNSP PGUID 4 t p P f t \054 0 0 0 anynonarray_in anynonarray_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYNONARRAYOID 2776
-DATA(insert OID = 3500 ( anyenum PGNSP PGUID 4 t p P f t \054 0 0 0 anyenum_in anyenum_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3500 ( anyenum PGNSP PGUID 4 t p P f t \054 0 0 0 anyenum_in anyenum_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYENUMOID 3500
-DATA(insert OID = 3115 ( fdw_handler PGNSP PGUID 4 t p P f t \054 0 0 0 fdw_handler_in fdw_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3115 ( fdw_handler PGNSP PGUID 4 t p P f t \054 0 0 0 fdw_handler_in fdw_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define FDW_HANDLEROID 3115
-DATA(insert OID = 325 ( index_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 index_am_handler_in index_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 325 ( index_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 index_am_handler_in index_am_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INDEX_AM_HANDLEROID 325
-DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_handler_in tsm_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_handler_in tsm_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define TSM_HANDLEROID 3310
-DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index f7bb4a54f7..01c542e829 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index abd31b68d4..a834cb4480 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -23,7 +23,8 @@
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString);
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt);
extern void RemoveRelations(DropStmt *drop);
diff --git a/src/include/commands/typecmds.h b/src/include/commands/typecmds.h
index 34f6fe328f..d198fd1d05 100644
--- a/src/include/commands/typecmds.h
+++ b/src/include/commands/typecmds.h
@@ -53,5 +53,6 @@ extern Oid AlterTypeNamespaceInternal(Oid typeOid, Oid nspOid,
bool isImplicitArray,
bool errorOnTableType,
ObjectAddresses *objsMoved);
+extern void AlterType(AlterTypeStmt * stmt);
#endif /* TYPECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 27bd4f3363..6c1c6350e3 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -413,6 +413,8 @@ typedef enum NodeTag
T_DropSubscriptionStmt,
T_CreateStatsStmt,
T_AlterCollationStmt,
+ T_AlterTypeStmt,
+ T_AlterTypeCmd,
/*
* TAGS FOR PARSE TREE NODES (parsenodes.h)
@@ -468,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
/*
* TAGS FOR REPLICATION GRAMMAR PARSE NODES (replnodes.h)
@@ -497,7 +500,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3171815320..82a9810e8a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,14 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ Oid methodOid;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1615,6 +1624,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1761,7 +1771,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -1777,8 +1788,8 @@ typedef struct AlterTableCmd /* one subcommand of an ALTER TABLE */
AlterTableType subtype; /* Type of table alteration to apply */
char *name; /* column, constraint, or trigger to act on,
* or tablespace */
- int16 num; /* attribute number for columns referenced
- * by number */
+ int16 num; /* attribute number for columns referenced by
+ * number */
RoleSpec *newowner;
Node *def; /* definition of new column, index,
* constraint, or parent table */
@@ -3431,4 +3442,24 @@ typedef struct DropSubscriptionStmt
DropBehavior behavior; /* RESTRICT or CASCADE behavior */
} DropSubscriptionStmt;
+typedef enum AlterTypeCmdType
+{
+ AT_AlterTypeCompression, /* ALTER TYPE name COMPRESSED cm WITH
+ * (options) */
+} AlterTypeCmdType;
+
+typedef struct AlterTypeCmd
+{
+ NodeTag type;
+ AlterTypeCmdType cmdtype;
+ Node *def;
+} AlterTypeCmd;
+
+typedef struct AlterTypeStmt
+{
+ NodeTag type;
+ List *typeName;
+ List *cmds;
+} AlterTypeStmt;
+
#endif /* PARSENODES_H */
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..f5c879ae60 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -146,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -282,7 +291,12 @@ typedef struct
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +325,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 43273eaab5..fa3bbbc49e 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -202,6 +202,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index 07208b56ce..00db7fead1 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -166,6 +166,7 @@ extern void getTypeBinaryOutputInfo(Oid type, Oid *typSend, bool *typIsVarlena);
extern Oid get_typmodin(Oid typid);
extern Oid get_typcollation(Oid typid);
extern bool type_is_collatable(Oid typid);
+extern Oid get_base_typdefaultcm(HeapTuple typtup);
extern Oid getBaseType(Oid typid);
extern Oid getBaseTypeAndTypmod(Oid typid, int32 *typmod);
extern int32 get_typavgwidth(Oid typid, int32 typmod);
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a92ea27ac..889f9c775a 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/include/utils/tuplesort.h b/src/include/utils/tuplesort.h
index b6b8c8ef8c..7b641e2d84 100644
--- a/src/include/utils/tuplesort.h
+++ b/src/include/utils/tuplesort.h
@@ -43,20 +43,20 @@ typedef enum
SORT_TYPE_QUICKSORT,
SORT_TYPE_EXTERNAL_SORT,
SORT_TYPE_EXTERNAL_MERGE
-} TuplesortMethod;
+} TuplesortMethod;
typedef enum
{
SORT_SPACE_TYPE_DISK,
SORT_SPACE_TYPE_MEMORY
-} TuplesortSpaceType;
+} TuplesortSpaceType;
typedef struct TuplesortInstrumentation
{
TuplesortMethod sortMethod; /* sort algorithm used */
TuplesortSpaceType spaceType; /* type of space spaceUsed represents */
long spaceUsed; /* space consumption, in kB */
-} TuplesortInstrumentation;
+} TuplesortInstrumentation;
/*
@@ -135,7 +135,7 @@ extern bool tuplesort_skiptuples(Tuplesortstate *state, int64 ntuples,
extern void tuplesort_end(Tuplesortstate *state);
extern void tuplesort_get_stats(Tuplesortstate *state,
- TuplesortInstrumentation *stats);
+ TuplesortInstrumentation * stats);
extern const char *tuplesort_method_name(TuplesortMethod m);
extern const char *tuplesort_space_type_name(TuplesortSpaceType t);
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..47c071abb2
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,127 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table cmtest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method for tsvector doesn't take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+NOTICE: merging column "fts" with inherited definition
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+ALTER TYPE tsvector SET COMPRESSED ts1;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+DROP TABLE cmtest;
+ALTER TYPE tsvector SET NOT COMPRESSED;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+DROP TABLE cmtest;
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 5 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index babda8978c..bdcc2bef2a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -543,10 +543,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -649,11 +649,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -676,11 +676,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -705,46 +705,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (MAXVALUE, 0, 0);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (MAXVALUE, 0, 0)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (1, MAXVALUE, 0);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (1, MAXVALUE, 0)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, 0);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, 0)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 3acc696863..fe873fa5e1 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -296,10 +296,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 927d0189a0..d7250ce52c 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1314,12 +1314,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1335,12 +1335,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1359,12 +1359,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1402,12 +1402,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1427,17 +1427,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1459,17 +1459,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1501,17 +1501,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1539,12 +1539,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1576,12 +1576,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1620,12 +1620,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1651,12 +1651,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1678,12 +1678,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1705,12 +1705,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1735,12 +1735,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1766,12 +1766,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 1fa9650ec9..b472007866 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1001,13 +1001,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1020,14 +1020,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1037,14 +1037,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1084,33 +1084,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1121,27 +1121,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1151,37 +1151,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index e159d62b66..7aa0bc8be2 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -568,74 +568,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, 0) TO ('b', MINVALUE),
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, 0)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, 0) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index d582bc9ee4..a110ae1f5b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 6750152e0f..accae702f4 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -107,6 +107,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 2fd3f2b1b1..9a4bacc54b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 76b0de30a7..b49808e814 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..50fafe36d7
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,47 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+
+SELECT length(fts) FROM cmtest2;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+
+ALTER TYPE tsvector SET COMPRESSED ts1;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+DROP TABLE cmtest;
+
+ALTER TYPE tsvector SET NOT COMPRESSED;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+DROP TABLE cmtest;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
Re: Custom compression methods
On Thu, 7 Sep 2017 19:42:36 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
Hello hackers!
I've attached a patch that implements custom compression
methods. This patch is based on Nikita Glukhov's code (which he hasn't
publish in mailing lists) for jsonb compression. This is early but
working version of the patch, and there are still few fixes and
features that should be implemented (like pg_dump support and support
of compression options for types), and it requires more testing. But
I'd like to get some feedback at the current stage first.There's been a proposal [1] of Alexander Korotkov and some discussion
about custom compression methods before. This is an implementation of
per-datum compression. Syntax is similar to the one in proposal but
not the same.Syntax:
CREATE COMPRESSION METHOD <cmname> HANDLER <compression_handler>;
DROP COMPRESSION METHOD <cmname>;Compression handler is a function that returns a structure containing
compression routines:- configure - function called when the compression method applied to
an attribute
- drop - called when the compression method is removed from an
attribute
- compress - compress function
- decompress - decompress functionUser can create compressed columns with the commands below:
CREATE TABLE t(a tsvector COMPRESSED <cmname> WITH <options>);
ALTER TABLE t ALTER COLUMN a SET COMPRESSED <cmname> WITH <options>;
ALTER TABLE t ALTER COLUMN a SET NOT COMPRESSED;Also there is syntax of binding compression methods to types:
ALTER TYPE <type> SET COMPRESSED <cmname>;
ALTER TYPE <type> SET NOT COMPRESSED;There are two new tables in the catalog, pg_compression and
pg_compression_opt. pg_compression is used as storage of compression
methods, and pg_compression_opt is used to store specific compression
options for particular column.When user binds a compression method to some column a new record in
pg_compression_opt is created and all further attribute values will
contain compression options Oid while old values will remain
unchanged. And when we alter a compression method for
the attribute it won't change previous record in pg_compression_opt.
Instead it'll create a new one and new values will be stored
with new Oid. That way there is no need of recompression of the old
tuples. And also tuples containing compressed datums can be copied to
other tables so records in pg_compression_opt shouldn't be removed. In
the current patch they can be removed with DROP COMPRESSION METHOD
CASCADE, but after that decompression won't be possible on compressed
tuples. Maybe CASCADE should keep compression options.I haven't changed the base logic of working with compressed datums. It
means that custom compressed datums behave exactly the same as current
LZ compressed datums, and the logic differs only in
toast_compress_datum and toast_decompress_datum.This patch doesn't break backward compability and should work
seamlessly with older version of database. I used one of two free
bits in `va_rawsize` from `varattrib_4b->va_compressed` as flag of
custom compressed datums. Also I renamed it to `va_info` since it
contains not only rawsize now.The patch also includes custom compression method for tsvector which
is used in tests.[1]
/messages/by-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx+Sig4ykWzWEAfkC6ZKMDy6=Q@mail.gmail.com
Attached rebased version of the patch. Added support of pg_dump, the
code was simplified, and a separate cache for compression options was
added.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v2.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..9efa5a9648 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -92,7 +92,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index ec10762529..21e375ad96 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -935,11 +935,48 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
+void
+freeRelOptions(List *options)
+{
+ ListCell *cell;
+
+ Assert(options != NIL);
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ pfree(def->defname);
+ pfree(defGetString(def));
+ pfree(def->arg);
+ }
+ list_free_deep(options);
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 4436c86361..8ec03457e0 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -226,6 +228,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -380,6 +383,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -442,6 +447,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -547,6 +553,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -659,7 +666,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 5a8f1dab83..83ba21e8c7 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -39,6 +41,8 @@
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
#include "utils/typcache.h"
@@ -53,19 +57,46 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+static HTAB *compression_options_htab = NULL;
+static MemoryContext compression_options_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ ((toast_compress_header *) (ptr))->info &= 0xC0000000; \
+ ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +114,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_compression_options_htab(void);
+static AttributeCompression *get_compression_options_info(Oid cmoptoid);
/* ----------
@@ -741,6 +774,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
@@ -770,10 +805,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +950,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1229,7 +1266,9 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
if (VARATT_IS_EXTERNAL(new_value) ||
VARATT_IS_COMPRESSED(new_value))
{
- new_value = heap_tuple_untoast_attr(new_value);
+ struct varlena *untoasted_value = heap_tuple_untoast_attr(new_value);
+
+ new_value = untoasted_value;
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
}
@@ -1353,7 +1392,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,25 +1406,43 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoptoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize,
+ len = 0;
+ AttributeCompression *ac = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
+ if (OidIsValid(cmoptoid))
+ ac = get_compression_options_info(cmoptoid);
+
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ if (!ac && (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size))
return PointerGetDatum(NULL);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ if (ac)
+ {
+ tmp = ac->routine->compress(ac, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
* We recheck the actual size even if pglz_compress() reports success,
@@ -1398,11 +1454,7 @@ toast_compress_datum(Datum value)
* only one header byte and no padding if the value is short enough. So
* we insist on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!ac && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,10 +1462,20 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (ac && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, ac->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
- pfree(tmp);
+ if (tmp)
+ pfree(tmp);
+
return PointerGetDatum(NULL);
}
}
@@ -2280,15 +2342,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ AttributeCompression *ac;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ ac = get_compression_options_info(hdr->cmoptoid);
+ result = ac->routine->decompress(ac, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2463,44 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+static void
+init_compression_options_htab(void)
+{
+ HASHCTL ctl;
+
+ compression_options_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "compression options cache context",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(AttributeCompression);
+ ctl.hcxt = compression_options_mcxt;
+ compression_options_htab = hash_create("compression options cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+}
+
+static AttributeCompression *
+get_compression_options_info(Oid cmoptoid)
+{
+ bool found;
+ AttributeCompression *result;
+
+ Assert(OidIsValid(cmoptoid));
+ if (!compression_options_htab)
+ init_compression_options_htab();
+
+ result = hash_search(compression_options_htab, &cmoptoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ Assert(compression_options_mcxt);
+ oldcxt = MemoryContextSwitchTo(compression_options_mcxt);
+ result->routine = GetCompressionRoutine(cmoptoid);
+ result->options = GetCompressionOptionsList(cmoptoid);
+ result->cmoptoid = cmoptoid;
+ MemoryContextSwitchTo(oldcxt);
+ }
+ return result;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 0453fd4ac1..53feb17abc 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -718,6 +718,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6fffc290fa..faaf7fb7f1 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2459,6 +2471,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 2eebb061b7..36c7fa0484 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -449,6 +449,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 05e70818e7..b08e857e3a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -707,6 +712,13 @@ AddNewAttributeTuples(Oid new_rel_oid,
referenced.objectSubId = 0;
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+
+ if (OidIsValid(attr->attcompression))
+ {
+ ObjectAddressSet(referenced, CompressionOptRelationId,
+ attr->attcompression);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
}
/*
@@ -1453,6 +1465,22 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+static void
+DropAttributeCompression(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ bool attisdropped = att->attisdropped;
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ att->attisdropped = true;
+ cmr->drop(att, options);
+ att->attisdropped = attisdropped;
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1511,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ DropAttributeCompression(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1611,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ DropAttributeCompression(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1634,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c7b2f031f0..11dc107ab7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6cac2dfd1d..d2b5285a5a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 59ffd2104d..6e806b3334 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
@@ -118,6 +119,7 @@ TypeShellMake(const char *typeName, Oid typeNamespace, Oid ownerId)
values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(-1);
values[Anum_pg_type_typndims - 1] = Int32GetDatum(0);
values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(InvalidOid);
+ values[Anum_pg_type_typdefaultcm - 1] = ObjectIdGetDatum(InvalidOid);
nulls[Anum_pg_type_typdefaultbin - 1] = true;
nulls[Anum_pg_type_typdefault - 1] = true;
nulls[Anum_pg_type_typacl - 1] = true;
@@ -362,6 +364,7 @@ TypeCreate(Oid newTypeOid,
values[Anum_pg_type_typtypmod - 1] = Int32GetDatum(typeMod);
values[Anum_pg_type_typndims - 1] = Int32GetDatum(typNDims);
values[Anum_pg_type_typcollation - 1] = ObjectIdGetDatum(typeCollation);
+ values[Anum_pg_type_typdefaultcm - 1] = ObjectIdGetDatum(InvalidOid);
/*
* initialize the default binary value for this type. Check for nulls of
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..32cbcc3efe
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,485 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/compression.h"
+#include "access/reloptions.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine *get_compression_method_routine(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+LookupCompressionHandlerFunc(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+static ObjectAddress
+CreateCompressionMethod(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = LookupCompressionHandlerFunc(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ return CreateCompressionMethod(cmName, (List *) handlerEl->arg);
+}
+
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, List *options)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Oid result,
+ cmhandler;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+ ObjectAddress myself,
+ ref1,
+ ref2,
+ ref3;
+ CompressionMethodRoutine *routine;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmhandler = ((Form_pg_compression) GETSTRUCT(tuple))->cmhandler;
+ ReleaseSysCache(tuple);
+
+ routine = get_compression_method_routine(cmhandler, attr->atttypid);
+
+ if (routine->configure && options != NIL)
+ routine->configure(attr, options);
+
+ values[Anum_pg_compression_opt_cmid - 1] = ObjectIdGetDatum(cmid);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ if (options)
+ values[Anum_pg_compression_opt_cmoptions - 1] = optionListToArray(options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+ tuple = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ result = CatalogTupleInsert(rel, tuple);
+ heap_freetuple(tuple);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, result);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmhandler);
+ ObjectAddressSubSet(ref2, RelationRelationId, attr->attrelid, attr->attnum);
+ ObjectAddressSet(ref3, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&ref2, &myself, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref3, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return result;
+}
+
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+ ReleaseSysCache(tup);
+ heap_close(relation, RowExclusiveLock);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = NULL;
+ compression->methodOid = cmopt->cmid;
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression * c1, ColumnCompression * c2,
+ const char *attributeName)
+{
+ char *cmname1 = c1->methodName ? c1->methodName :
+ get_compression_method_name(c1->methodOid);
+ char *cmname2 = c2->methodName ? c2->methodName :
+ get_compression_method_name(c2->methodOid);
+
+ if (strcmp(cmname1, cmname2))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", cmname1, cmname2)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ Oid cmid;
+ char *result = NULL;
+ Form_pg_compression cmform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmid = ((Form_pg_compression_opt) GETSTRUCT(tup))->cmid;
+ ReleaseSysCache(tup);
+
+ /* now we can get the name */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ return result;
+}
+
+/* get_compression_options */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+CompressionMethodRoutine *
+GetCompressionRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return get_compression_method_routine(cmhandler, InvalidOid);
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+get_compression_method_routine(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index cfa3f059c2..8cef3be086 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2758,8 +2758,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index e60210cb24..010bdb644b 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -112,7 +112,8 @@ create_ctas_internal(List *attrList, IntoClause *into)
* Create the relation. (This will error out if there's an existing view,
* so we don't need more code to complain if "replace" is false.)
*/
- intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL);
+ intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL,
+ NULL);
/*
* If necessary, create a TOAST table for the target table. Note that
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 938133bbe4..f520c7fe7b 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 62937124ef..857eb14b64 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -212,7 +212,8 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq)
stmt->tablespacename = NULL;
stmt->if_not_exists = seq->if_not_exists;
- address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL);
+ address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL,
+ NULL);
seqoid = address.objectId;
Assert(seqoid != InvalidOid);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2e389e08da..11c5eea243 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -458,6 +463,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression * compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -502,7 +509,8 @@ static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
*/
ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString)
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt)
{
char relname[NAMEDATALEN];
Oid namespaceId;
@@ -522,6 +530,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
static char *validnsps[] = HEAP_RELOPT_NAMESPACES;
Oid ofTypeId;
ObjectAddress address;
+ AlterTableStmt *alterStmt = NULL;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -723,6 +732,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ transformColumnCompression(colDef, stmt->relation, &alterStmt);
}
/*
@@ -920,6 +931,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
relation_close(rel, NoLock);
+ if (pAlterStmt)
+ *pAlterStmt = (Node *) alterStmt;
+ else
+ Assert(!alterStmt);
+
return address;
}
@@ -1609,6 +1625,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1943,6 +1960,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1970,6 +2000,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2179,6 +2212,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3278,6 +3318,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3768,6 +3809,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4107,6 +4154,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5325,6 +5377,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
+ if (!colDef->compression)
+ {
+ /* colDef->compression is handled in subsequent ALTER TABLE statement */
+ Oid cmid = get_base_typdefaultcm(typeTuple);
+
+ if (OidIsValid(cmid))
+ attribute.attcompression = CreateCompressionOptions(&attribute,
+ cmid, NULL);
+ }
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6425,6 +6489,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9070,6 +9139,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
SysScanDesc scan;
HeapTuple depTup;
ObjectAddress address;
+ Oid cmid;
attrelation = heap_open(AttributeRelationId, RowExclusiveLock);
@@ -9345,6 +9415,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9394,7 +9466,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9417,6 +9491,10 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ cmid = get_base_typdefaultcm(typeTuple);
+ if (OidIsValid(cmid))
+ attTup->attcompression = CreateCompressionOptions(attTup, cmid, NULL);
+
ReleaseSysCache(typeTuple);
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
@@ -12413,6 +12491,86 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression * compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName || OidIsValid(compression->methodOid))
+ {
+ /* SET COMPRESSED */
+ Oid cmid,
+ cmoptoid;
+
+ cmid = compression->methodName
+ ? get_compression_method_oid(compression->methodName, false)
+ : compression->methodOid;
+
+ cmoptoid = CreateCompressionOptions(atttableform, cmid, compression->options);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 7ed16aeff4..38a086b584 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -2110,7 +2111,7 @@ DefineCompositeType(RangeVar *typevar, List *coldeflist)
* Finally create the relation. This also creates the type.
*/
DefineRelation(createStmt, RELKIND_COMPOSITE_TYPE, InvalidOid, &address,
- NULL);
+ NULL, NULL);
return address;
}
@@ -3638,3 +3639,100 @@ AlterTypeNamespaceInternal(Oid typeOid, Oid nspOid,
return oldNspOid;
}
+
+
+/*
+ * Execute ALTER TYPE SET COMPRESSED <cm> [WITH (<option>, ...)]
+ */
+static void
+AlterTypeDefaultCompression(Oid typeid, ColumnCompression * compression)
+{
+ Oid cmoid;
+ Type oldtup = typeidType(typeid);
+ Form_pg_type oldtype = (Form_pg_type) GETSTRUCT(oldtup);
+
+ if (oldtype->typtype != TYPTYPE_BASE)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compression can be applied only to base types")));
+
+ cmoid = compression->methodName
+ ? get_compression_method_oid(compression->methodName, false)
+ : InvalidOid;
+
+ if (oldtype->typdefaultcm != cmoid)
+ {
+ Relation typrel;
+ HeapTuple newtup;
+ Datum values[Natts_pg_type];
+ bool nulls[Natts_pg_type];
+ bool replace[Natts_pg_type];
+
+ typrel = heap_open(TypeRelationId, RowExclusiveLock);
+
+ memset(values, 0, sizeof(values));
+ memset(nulls, 0, sizeof(nulls));
+ memset(replace, 0, sizeof(replace));
+
+ values[Anum_pg_type_typdefaultcm - 1] = ObjectIdGetDatum(cmoid);
+ nulls[Anum_pg_type_typdefaultcm - 1] = false;
+ replace[Anum_pg_type_typdefaultcm - 1] = true;
+
+ newtup = heap_modify_tuple(oldtup, RelationGetDescr(typrel),
+ values, nulls, replace);
+
+ CatalogTupleUpdate(typrel, &newtup->t_self, newtup);
+
+ heap_freetuple(newtup);
+ heap_close(typrel, RowExclusiveLock);
+
+ if (OidIsValid(oldtype->typdefaultcm))
+ deleteDependencyRecordsForClass(TypeRelationId, typeid,
+ CompressionMethodRelationId,
+ DEPENDENCY_NORMAL);
+
+ if (OidIsValid(cmoid))
+ {
+ ObjectAddress myself,
+ referenced;
+
+ ObjectAddressSet(myself, TypeRelationId, typeid);
+ ObjectAddressSet(referenced, CompressionMethodRelationId, cmoid);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
+
+ InvokeObjectPostAlterHook(TypeRelationId, typeid, 0);
+ }
+
+ ReleaseSysCache(oldtup);
+}
+
+/*
+ * Execute ALTER TYPE <typeName> <command>, ...
+ */
+void
+AlterType(AlterTypeStmt * stmt)
+{
+ TypeName *typename;
+ Oid typeid;
+ ListCell *lcmd;
+
+ /* Make a TypeName so we can use standard type lookup machinery */
+ typename = makeTypeNameFromNameList(stmt->typeName);
+ typeid = typenameTypeId(NULL, typename);
+
+ foreach(lcmd, stmt->cmds)
+ {
+ AlterTypeCmd *cmd = (AlterTypeCmd *) lfirst(lcmd);
+
+ switch (cmd->cmdtype)
+ {
+ case AT_AlterTypeCompression:
+ AlterTypeDefaultCompression(typeid,
+ (ColumnCompression *) cmd->def);
+ break;
+ default:
+ elog(ERROR, "unknown ALTER TYPE command %d", cmd->cmdtype);
+ }
+ }
+}
diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c
index 076e2a3a40..9af0f11856 100644
--- a/src/backend/commands/view.c
+++ b/src/backend/commands/view.c
@@ -251,7 +251,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace,
* false).
*/
address = DefineRelation(createStmt, RELKIND_VIEW, InvalidOid, NULL,
- NULL);
+ NULL, NULL);
Assert(address.objectId != InvalidOid);
/* Make the new view relation visible */
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 47a34a044a..a58af7ee92 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -300,6 +300,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
var->vartypmod != -1))
return false; /* type mismatch */
+ if (OidIsValid(att_tup->attcompression))
+ return false;
+
tlist_item = lnext(tlist_item);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index f1bed14e2b..e406828ff2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2805,6 +2805,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2823,6 +2824,19 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression * from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_SCALAR_FIELD(methodOid);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -4548,6 +4562,28 @@ _copyDropSubscriptionStmt(const DropSubscriptionStmt *from)
return newnode;
}
+static AlterTypeStmt *
+_copyAlterTypeStmt(const AlterTypeStmt * from)
+{
+ AlterTypeStmt *newnode = makeNode(AlterTypeStmt);
+
+ COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(cmds);
+
+ return newnode;
+}
+
+static AlterTypeCmd *
+_copyAlterTypeCmd(const AlterTypeCmd * from)
+{
+ AlterTypeCmd *newnode = makeNode(AlterTypeCmd);
+
+ COPY_SCALAR_FIELD(cmdtype);
+ COPY_NODE_FIELD(def);
+
+ return newnode;
+}
+
/* ****************************************************************
* pg_list.h copy functions
* ****************************************************************
@@ -5456,6 +5492,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
@@ -5539,6 +5578,14 @@ copyObjectImpl(const void *from)
retval = _copyForeignKeyCacheInfo(from);
break;
+ case T_AlterTypeStmt:
+ retval = _copyAlterTypeStmt(from);
+ break;
+
+ case T_AlterTypeCmd:
+ retval = _copyAlterTypeCmd(from);
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d", (int) nodeTag(from));
retval = 0; /* keep compiler quiet */
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 8b56b9146a..8591050ac3 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2536,6 +2536,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2554,6 +2555,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression * a, const ColumnCompression * b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_SCALAR_FIELD(methodOid);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -2867,6 +2878,24 @@ _equalPartitionCmd(const PartitionCmd *a, const PartitionCmd *b)
return true;
}
+static bool
+_equalAlterTypeStmt(const AlterTypeStmt * a, const AlterTypeStmt * b)
+{
+ COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(cmds);
+
+ return true;
+}
+
+static bool
+_equalAlterTypeCmd(const AlterTypeCmd * a, const AlterTypeCmd * b)
+{
+ COMPARE_SCALAR_FIELD(cmdtype);
+ COMPARE_NODE_FIELD(def);
+
+ return true;
+}
+
/*
* Stuff from pg_list.h
*/
@@ -3602,6 +3631,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
@@ -3677,6 +3709,12 @@ equal(const void *a, const void *b)
case T_PartitionCmd:
retval = _equalPartitionCmd(a, b);
break;
+ case T_AlterTypeStmt:
+ retval = _equalAlterTypeStmt(a, b);
+ break;
+ case T_AlterTypeCmd:
+ retval = _equalAlterTypeCmd(a, b);
+ break;
default:
elog(ERROR, "unrecognized node type: %d",
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index e3eb0c5788..c1bea10eff 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b83d919e40..a051b9a01c 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2792,6 +2792,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2808,6 +2809,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression * node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_OID_FIELD(methodOid);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4097,6 +4108,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c303818c9b..dba93aa3e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -282,6 +282,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
+ AlterTypeStmt
%type <node> select_no_parens select_with_parens select_clause
simple_select values_clause
@@ -290,8 +291,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <ival> add_drop opt_asc_desc opt_nulls_order
%type <node> alter_table_cmd alter_type_cmd opt_collate_clause
- replica_identity partition_cmd
-%type <list> alter_table_cmds alter_type_cmds
+ replica_identity partition_cmd alterTypeCmd
+%type <list> alter_table_cmds alter_type_cmds alterTypeCmds
%type <list> alter_identity_column_option_list
%type <defelt> alter_identity_column_option
@@ -396,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
%type <list> group_by_list
%type <node> group_by_item empty_grouping_set rollup_clause cube_clause
@@ -579,6 +581,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> partbound_datum PartitionRangeDatum
%type <list> partbound_datum_list range_datum_list
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -611,9 +615,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -840,6 +844,7 @@ stmt :
| AlterSubscriptionStmt
| AlterTSConfigurationStmt
| AlterTSDictionaryStmt
+ | AlterTypeStmt
| AlterUserMappingStmt
| AnalyzeStmt
| CheckPointStmt
@@ -2165,6 +2170,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -2733,6 +2747,32 @@ PartitionRangeDatum:
* really variants of the ALTER TABLE subcommands with different spellings
*****************************************************************************/
+AlterTypeStmt:
+ ALTER TYPE_P any_name alterTypeCmds
+ {
+ AlterTypeStmt *n = makeNode(AlterTypeStmt);
+ n->typeName = $3;
+ n->cmds = $4;
+ $$ = (Node *) n;
+ }
+ ;
+
+alterTypeCmds:
+ alterTypeCmd { $$ = list_make1($1); }
+ | alterTypeCmds ',' alterTypeCmd { $$ = lappend($1, $3); }
+ ;
+
+alterTypeCmd:
+ /* ALTER TYPE <name> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ SET columnCompression
+ {
+ AlterTypeCmd *n = makeNode(AlterTypeCmd);
+ n->cmdtype = AT_AlterTypeCompression;
+ n->def = $2;
+ $$ = (Node *)n;
+ }
+ ;
+
AlterCompositeTypeStmt:
ALTER TYPE_P any_name alter_type_cmds
{
@@ -3258,11 +3298,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3272,8 +3313,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3320,6 +3361,39 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->methodOid = InvalidOid;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->methodOid = InvalidOid;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5680,6 +5754,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6188,6 +6271,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6251,7 +6335,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6441,6 +6525,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14649,6 +14734,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 655da02c10..78544a8785 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -493,6 +494,49 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (!column->compression && column->typeName)
+ {
+ Type tup = typenameType(NULL, column->typeName, NULL);
+ Oid cmoid = get_base_typdefaultcm(tup);
+
+ ReleaseSysCache(tup);
+
+ if (OidIsValid(cmoid))
+ {
+ column->compression = makeNode(ColumnCompression);
+ column->compression->methodName = NULL;
+ column->compression->methodOid = cmoid;
+ column->compression->options = NIL;
+ }
+ }
+
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -793,6 +837,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1002,6 +1056,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 775477c6cf..a07327fbf7 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -217,6 +217,7 @@ check_xact_readonly(Node *parsetree)
case T_CreateSubscriptionStmt:
case T_AlterSubscriptionStmt:
case T_DropSubscriptionStmt:
+ case T_AlterTypeStmt:
PreventCommandIfReadOnly(CreateCommandTag(parsetree));
PreventCommandIfParallelMode(CreateCommandTag(parsetree));
break;
@@ -998,6 +999,7 @@ ProcessUtilitySlow(ParseState *pstate,
foreach(l, stmts)
{
Node *stmt = (Node *) lfirst(l);
+ Node *alterStmt = NULL;
if (IsA(stmt, CreateStmt))
{
@@ -1008,7 +1010,9 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_RELATION,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1042,7 +1046,8 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_FOREIGN_TABLE,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
CreateForeignTable((CreateForeignTableStmt *) stmt,
address.objectId);
EventTriggerCollectSimpleCommand(address,
@@ -1074,6 +1079,9 @@ ProcessUtilitySlow(ParseState *pstate,
NULL);
}
+ if (alterStmt)
+ lappend(stmts, alterStmt);
+
/* Need CCI between commands */
if (lnext(l) != NULL)
CommandCounterIncrement();
@@ -1283,6 +1291,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -1643,6 +1656,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = AlterCollation((AlterCollationStmt *) parsetree);
break;
+ case T_AlterTypeStmt:
+ AlterType((AlterTypeStmt *) parsetree);
+ break;
+
default:
elog(ERROR, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -1696,6 +1713,11 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_FOREIGN_TABLE:
RemoveRelations(stmt);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ if (stmt->behavior == DROP_CASCADE)
+ {
+ /* TODO decompress columns instead of their deletion */
+ }
default:
RemoveObjects(stmt);
break;
@@ -2309,6 +2331,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2437,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2846,6 +2874,10 @@ CreateCommandTag(Node *parsetree)
}
break;
+ case T_AlterTypeStmt:
+ tag = "ALTER TYPE";
+ break;
+
default:
elog(WARNING, "unrecognized node type: %d",
(int) nodeTag(parsetree));
@@ -3291,6 +3323,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_AlterTypeStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
/* already-planned queries */
case T_PlannedStmt:
{
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 6f66c1f58c..9a2b5c5b1f 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,92 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(AttributeCompression *ac, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static void
+tsvector_configure(Form_pg_attribute attr, List *options)
+{
+ if (options != NIL)
+ elog(ERROR, "the compression method for tsvector doesn't take any options");
+}
+
+static struct varlena *
+tsvector_decompress(AttributeCompression *ac, const struct varlena *data)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = tsvector_configure;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index b7a14dc87e..ae7320707c 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2276,16 +2276,12 @@ getBaseType(Oid typid)
}
/*
- * getBaseTypeAndTypmod
- * If the given type is a domain, return its base type and typmod;
- * otherwise return the type's own OID, and leave *typmod unchanged.
- *
* Note that the "applied typmod" should be -1 for every domain level
* above the bottommost; therefore, if the passed-in typid is indeed
* a domain, *typmod should be -1.
*/
-Oid
-getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+static inline HeapTuple
+getBaseTypeTuple(Oid *typid, int32 *typmod)
{
/*
* We loop to find the bottom base type in a stack of domains.
@@ -2295,24 +2291,33 @@ getBaseTypeAndTypmod(Oid typid, int32 *typmod)
HeapTuple tup;
Form_pg_type typTup;
- tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*typid));
if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for type %u", typid);
+ elog(ERROR, "cache lookup failed for type %u", *typid);
typTup = (Form_pg_type) GETSTRUCT(tup);
if (typTup->typtype != TYPTYPE_DOMAIN)
- {
/* Not a domain, so done */
- ReleaseSysCache(tup);
- break;
- }
+ return tup;
Assert(*typmod == -1);
- typid = typTup->typbasetype;
+ *typid = typTup->typbasetype;
*typmod = typTup->typtypmod;
ReleaseSysCache(tup);
}
+}
+/*
+ * getBaseTypeAndTypmod
+ * If the given type is a domain, return its base type and typmod;
+ * otherwise return the type's own OID, and leave *typmod unchanged.
+ */
+Oid
+getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+{
+ HeapTuple tup = getBaseTypeTuple(&typid, typmod);
+
+ ReleaseSysCache(tup);
return typid;
}
@@ -2808,6 +2813,39 @@ type_is_collatable(Oid typid)
return OidIsValid(get_typcollation(typid));
}
+/*
+ * get_base_typdefaultcm
+ *
+ * Given the type tuple, return the base type's typdefaultcm attribute.
+ */
+Oid
+get_base_typdefaultcm(HeapTuple typtup)
+{
+ Oid typid;
+ Oid base;
+ Oid cm = InvalidOid;
+
+ for (typid = (Oid) -1; !OidIsValid(cm) && OidIsValid(typid); typid = base)
+ {
+ Form_pg_type type;
+
+ if (typid != (Oid) -1)
+ {
+ typtup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ if (!HeapTupleIsValid(typtup))
+ elog(ERROR, "cache lookup failed for type %u", typid);
+ }
+
+ type = (Form_pg_type) GETSTRUCT(typtup);
+ base = type->typtype == TYPTYPE_DOMAIN ? type->typbasetype : InvalidOid;
+ cm = type->typdefaultcm;
+
+ if (typid != (Oid) -1)
+ ReleaseSysCache(typtup);
+ }
+
+ return cm;
+}
/* ---------- STATISTICS CACHE ---------- */
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index b8e37809b0..f8849c89e7 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/nbtree.h"
@@ -76,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -565,6 +567,7 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index fcbb683a99..634482e326 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 4b47951de1..2d6827f1f3 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
static DumpableObject **nspinfoindex;
static DumpableObject **extinfoindex;
+static DumpableObject **cminfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
@@ -61,6 +62,7 @@ static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
+static int numCompressionMethods;
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
@@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+ CompressionMethodInfo *cminfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr)
write_msg(NULL, "reading subscriptions\n");
getSubscriptions(fout);
+ if (g_verbose)
+ write_msg(NULL, "reading compression methods\n");
+ cminfo = getCompressionMethods(fout, &numCompressionMethods);
+ cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo));
+
*numTablesPtr = numTables;
return tblinfo;
}
@@ -827,6 +836,17 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
+/*
+ * findCompressionMethodByOid
+ * finds the entry (in cminfo) of the compression method with the given oid
+ * returns NULL if not found
+ */
+CompressionMethodInfo *
+findCompressionMethodByOid(Oid oid)
+{
+ return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex,
+ numCompressionMethods);
+}
/*
* setExtensionMembership
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..6b962a0ba0 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -149,6 +150,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 75f08cd792..5cd9f74780 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -358,6 +358,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -1500,6 +1501,11 @@ selectDumpableType(TypeInfo *tyinfo, Archive *fout)
/* Dump based on if the contents of the namespace are being dumped */
tyinfo->dobj.dump = tyinfo->dobj.namespace->dobj.dump_contains;
+
+ /* Add flag of compression is type was altered */
+ if (tyinfo->typdefaultcm)
+ tyinfo->dobj.dump |= DUMP_COMPONENT_MODIFICATION;
+
}
/*
@@ -3946,6 +3952,99 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getCompressionMethods
+ * get information about compression methods
+ */
+CompressionMethodInfo *
+getCompressionMethods(Archive *fout, int *numMethods)
+{
+ DumpOptions *dopt = fout->dopt;
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionMethodInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i,
+ ntups;
+
+ if (dopt->no_compression_methods || fout->remoteVersion < 110000)
+ return NULL;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ /* Get the compression methods in current database. */
+ appendPQExpBuffer(query,
+ "SELECT c.tableoid, c.oid, c.cmname, c.cmhandler "
+ "FROM pg_catalog.pg_compression c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "oid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_METHOD;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+
+ /* Decide whether we want to dump it */
+ selectDumpableObject(&(cminfo[i].dobj), fout);
+ }
+ if (numMethods)
+ *numMethods = ntups;
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+
+ return cminfo;
+}
+
+/*
+ * dumpCompressionMethod
+ * dump the definition of the given compression method
+ */
+static void
+dumpCompressionMethod(Archive *fout, CompressionMethodInfo *cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER",
+ fmtId(cminfo->dobj.name));
+ appendPQExpBuffer(query, " %s;\n", fmtId(cminfo->cmhandler));
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION METHOD", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -4422,6 +4521,7 @@ getTypes(Archive *fout, int *numTypes)
int i_typtype;
int i_typisdefined;
int i_isarray;
+ int i_typdefaultcm;
/*
* we include even the built-in types because those may be used as array
@@ -4442,7 +4542,48 @@ getTypes(Archive *fout, int *numTypes)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- if (fout->remoteVersion >= 90600)
+ if (fout->remoteVersion >= 110000)
+ {
+ PQExpBuffer acl_subquery = createPQExpBuffer();
+ PQExpBuffer racl_subquery = createPQExpBuffer();
+ PQExpBuffer initacl_subquery = createPQExpBuffer();
+ PQExpBuffer initracl_subquery = createPQExpBuffer();
+
+ buildACLQueries(acl_subquery, racl_subquery, initacl_subquery,
+ initracl_subquery, "t.typacl", "t.typowner", "'T'",
+ dopt->binary_upgrade);
+
+ appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
+ "t.typnamespace, "
+ "t.typdefaultcm, "
+ "%s AS typacl, "
+ "%s AS rtypacl, "
+ "%s AS inittypacl, "
+ "%s AS initrtypacl, "
+ "(%s t.typowner) AS rolname, "
+ "t.typelem, t.typrelid, "
+ "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" "
+ "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, "
+ "t.typtype, t.typisdefined, "
+ "t.typname[0] = '_' AND t.typelem != 0 AND "
+ "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray "
+ "FROM pg_type t "
+ "LEFT JOIN pg_init_privs pip ON "
+ "(t.oid = pip.objoid "
+ "AND pip.classoid = 'pg_type'::regclass "
+ "AND pip.objsubid = 0) ",
+ acl_subquery->data,
+ racl_subquery->data,
+ initacl_subquery->data,
+ initracl_subquery->data,
+ username_subquery);
+
+ destroyPQExpBuffer(acl_subquery);
+ destroyPQExpBuffer(racl_subquery);
+ destroyPQExpBuffer(initacl_subquery);
+ destroyPQExpBuffer(initracl_subquery);
+ }
+ else if (fout->remoteVersion >= 90600)
{
PQExpBuffer acl_subquery = createPQExpBuffer();
PQExpBuffer racl_subquery = createPQExpBuffer();
@@ -4455,6 +4596,7 @@ getTypes(Archive *fout, int *numTypes)
appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
"t.typnamespace, "
+ "NULL AS typdefaultcm, "
"%s AS typacl, "
"%s AS rtypacl, "
"%s AS inittypacl, "
@@ -4487,6 +4629,7 @@ getTypes(Archive *fout, int *numTypes)
appendPQExpBuffer(query, "SELECT tableoid, oid, typname, "
"typnamespace, typacl, NULL as rtypacl, "
"NULL AS inittypacl, NULL AS initrtypacl, "
+ "NULL AS typdefaultcm, "
"(%s typowner) AS rolname, "
"typelem, typrelid, "
"CASE WHEN typrelid = 0 THEN ' '::\"char\" "
@@ -4502,6 +4645,7 @@ getTypes(Archive *fout, int *numTypes)
appendPQExpBuffer(query, "SELECT tableoid, oid, typname, "
"typnamespace, NULL AS typacl, NULL as rtypacl, "
"NULL AS inittypacl, NULL AS initrtypacl, "
+ "NULL AS typdefaultcm, "
"(%s typowner) AS rolname, "
"typelem, typrelid, "
"CASE WHEN typrelid = 0 THEN ' '::\"char\" "
@@ -4517,6 +4661,7 @@ getTypes(Archive *fout, int *numTypes)
appendPQExpBuffer(query, "SELECT tableoid, oid, typname, "
"typnamespace, NULL AS typacl, NULL as rtypacl, "
"NULL AS inittypacl, NULL AS initrtypacl, "
+ "NULL AS typdefaultcm, "
"(%s typowner) AS rolname, "
"typelem, typrelid, "
"CASE WHEN typrelid = 0 THEN ' '::\"char\" "
@@ -4548,6 +4693,7 @@ getTypes(Archive *fout, int *numTypes)
i_typtype = PQfnumber(res, "typtype");
i_typisdefined = PQfnumber(res, "typisdefined");
i_isarray = PQfnumber(res, "isarray");
+ i_typdefaultcm = PQfnumber(res, "typdefaultcm");
for (i = 0; i < ntups; i++)
{
@@ -4569,6 +4715,7 @@ getTypes(Archive *fout, int *numTypes)
tyinfo[i].typrelkind = *PQgetvalue(res, i, i_typrelkind);
tyinfo[i].typtype = *PQgetvalue(res, i, i_typtype);
tyinfo[i].shellType = NULL;
+ tyinfo[i].typdefaultcm = atooid(PQgetvalue(res, i, i_typdefaultcm));
if (strcmp(PQgetvalue(res, i, i_typisdefined), "t") == 0)
tyinfo[i].isDefined = true;
@@ -7853,6 +8000,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7888,7 +8037,48 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.cmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "cm.cmname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
+ "LEFT JOIN pg_catalog.pg_compression cm "
+ "ON c.cmid = cm.oid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7907,9 +8097,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2 "
"ORDER BY a.attnum",
@@ -7933,7 +8127,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7957,7 +8153,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7975,7 +8173,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7992,7 +8192,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8022,6 +8224,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8038,6 +8242,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8065,6 +8271,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9554,6 +9762,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_COMPRESSION_METHOD:
+ dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -10398,6 +10609,26 @@ dumpBaseType(Archive *fout, TypeInfo *tyinfo)
NULL, 0,
NULL, NULL);
+ if ((tyinfo->dobj.dump & DUMP_COMPONENT_MODIFICATION) && tyinfo->typdefaultcm)
+ {
+ PQExpBuffer alterq = createPQExpBuffer();
+ CompressionMethodInfo *cminfo = findCompressionMethodByOid(tyinfo->typdefaultcm);
+
+ appendPQExpBuffer(alterq, "ALTER TYPE %s SET COMPRESSED %s,", qtypname,
+ cminfo->dobj.name);
+ ArchiveEntry(fout, tyinfo->dobj.catId, tyinfo->dobj.dumpId,
+ tyinfo->dobj.name,
+ tyinfo->dobj.namespace->dobj.name,
+ NULL,
+ tyinfo->rolname, false,
+ "TYPE CM", SECTION_PRE_DATA,
+ alterq->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(alterq);
+ }
+
/* Dump Type Comments and Security Labels */
if (tyinfo->dobj.dump & DUMP_COMPONENT_COMMENT)
dumpComment(fout, labelq->data,
@@ -15379,6 +15610,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSED %s",
+ fmtId(tbinfo->attcmnames[j]));
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH %s",
+ fmtId(tbinfo->attcmoptions[j]));
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17641,6 +17881,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_COMPRESSION_METHOD:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e7593e6da7..5fc612ee5d 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_COMPRESSION_METHOD
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -96,6 +97,7 @@ typedef uint32 DumpComponents; /* a bitmask of dump object components */
#define DUMP_COMPONENT_ACL (1 << 4)
#define DUMP_COMPONENT_POLICY (1 << 5)
#define DUMP_COMPONENT_USERMAP (1 << 6)
+#define DUMP_COMPONENT_MODIFICATION (1 << 7)
#define DUMP_COMPONENT_ALL (0xFFFF)
/*
@@ -179,6 +181,7 @@ typedef struct _typeInfo
char typtype; /* 'b', 'c', etc */
bool isArray; /* true if auto-generated array type */
bool isDefined; /* true if typisdefined */
+ Oid typdefaultcm;
/* If needed, we'll create a "shell type" entry for it; link that here: */
struct _shellTypeInfo *shellType; /* shell-type entry, or NULL */
/* If it's a domain, we store links to its constraints here: */
@@ -315,6 +318,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +616,13 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+/* The CompressionMethodInfo struct is used to represent compression method */
+typedef struct _CompressionMethodInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+} CompressionMethodInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -654,6 +666,7 @@ extern OprInfo *findOprByOid(Oid oid);
extern CollInfo *findCollationByOid(Oid oid);
extern NamespaceInfo *findNamespaceByOid(Oid oid);
extern ExtensionInfo *findExtensionByOid(Oid oid);
+extern CompressionMethodInfo *findCompressionMethodByOid(Oid oid);
extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems);
extern ExtensionInfo *findOwningExtension(CatalogId catalogId);
@@ -711,5 +724,7 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern CompressionMethodInfo *getCompressionMethods(Archive *fout,
+ int *numMethods);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5044a76787..7195f54cdc 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -40,47 +40,48 @@ static const int dbObjectTypePriority[] =
{
1, /* DO_NAMESPACE */
4, /* DO_EXTENSION */
- 5, /* DO_TYPE */
- 5, /* DO_SHELL_TYPE */
- 6, /* DO_FUNC */
- 7, /* DO_AGG */
- 8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
- 9, /* DO_OPCLASS */
- 9, /* DO_OPFAMILY */
- 3, /* DO_COLLATION */
- 11, /* DO_CONVERSION */
- 18, /* DO_TABLE */
- 20, /* DO_ATTRDEF */
- 28, /* DO_INDEX */
- 29, /* DO_STATSEXT */
- 30, /* DO_RULE */
- 31, /* DO_TRIGGER */
- 27, /* DO_CONSTRAINT */
- 32, /* DO_FK_CONSTRAINT */
- 2, /* DO_PROCLANG */
- 10, /* DO_CAST */
- 23, /* DO_TABLE_DATA */
- 24, /* DO_SEQUENCE_SET */
- 19, /* DO_DUMMY_TYPE */
- 12, /* DO_TSPARSER */
- 14, /* DO_TSDICT */
- 13, /* DO_TSTEMPLATE */
- 15, /* DO_TSCONFIG */
- 16, /* DO_FDW */
- 17, /* DO_FOREIGN_SERVER */
- 32, /* DO_DEFAULT_ACL */
- 3, /* DO_TRANSFORM */
- 21, /* DO_BLOB */
- 25, /* DO_BLOB_DATA */
- 22, /* DO_PRE_DATA_BOUNDARY */
- 26, /* DO_POST_DATA_BOUNDARY */
- 33, /* DO_EVENT_TRIGGER */
- 38, /* DO_REFRESH_MATVIEW */
- 34, /* DO_POLICY */
- 35, /* DO_PUBLICATION */
- 36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 6, /* DO_TYPE */
+ 6, /* DO_SHELL_TYPE */
+ 7, /* DO_FUNC */
+ 8, /* DO_AGG */
+ 9, /* DO_OPERATOR */
+ 9, /* DO_ACCESS_METHOD */
+ 10, /* DO_OPCLASS */
+ 10, /* DO_OPFAMILY */
+ 4, /* DO_COLLATION */
+ 12, /* DO_CONVERSION */
+ 19, /* DO_TABLE */
+ 21, /* DO_ATTRDEF */
+ 29, /* DO_INDEX */
+ 30, /* DO_STATSEXT */
+ 31, /* DO_RULE */
+ 32, /* DO_TRIGGER */
+ 28, /* DO_CONSTRAINT */
+ 33, /* DO_FK_CONSTRAINT */
+ 3, /* DO_PROCLANG */
+ 11, /* DO_CAST */
+ 24, /* DO_TABLE_DATA */
+ 25, /* DO_SEQUENCE_SET */
+ 20, /* DO_DUMMY_TYPE */
+ 13, /* DO_TSPARSER */
+ 15, /* DO_TSDICT */
+ 14, /* DO_TSTEMPLATE */
+ 16, /* DO_TSCONFIG */
+ 17, /* DO_FDW */
+ 18, /* DO_FOREIGN_SERVER */
+ 33, /* DO_DEFAULT_ACL */
+ 4, /* DO_TRANSFORM */
+ 22, /* DO_BLOB */
+ 26, /* DO_BLOB_DATA */
+ 23, /* DO_PRE_DATA_BOUNDARY */
+ 27, /* DO_POST_DATA_BOUNDARY */
+ 34, /* DO_EVENT_TRIGGER */
+ 39, /* DO_REFRESH_MATVIEW */
+ 35, /* DO_POLICY */
+ 36, /* DO_PUBLICATION */
+ 37, /* DO_PUBLICATION_REL */
+ 38, /* DO_SUBSCRIPTION */
+ 5 /* DO_COMPRESSION_METHOD */
};
static DumpId preDataBoundId;
@@ -1436,6 +1437,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_COMPRESSION_METHOD:
+ snprintf(buf, bufsize,
+ "COMPRESSION METHOD %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..6b72ccf9ea 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 041b5e0c87..0ded023858 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index d22ec68431..57862497d0 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1620,6 +1683,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ if (pset.sversion >= 100000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT cm.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " JOIN pg_catalog.pg_compression cm ON (cm.oid = c.cmid) "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1741,6 +1820,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1840,6 +1923,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1847,7 +1935,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1858,7 +1946,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index 4d1c0ec3c6..307f8bfbe5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a09c49d6cf..1327abdfef 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..1320d8f882
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ CompressionMethodRoutine *routine;
+ List *options;
+ Oid cmoptoid;
+} AttributeCompression;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (AttributeCompression * ac, const struct varlena *data);
+
+/*
+ * API struct for an compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionConfigureRoutine drop;
+ CompressionRoutine compress;
+ CompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine * GetCompressionRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr, Oid cmid,
+ List *options);
+extern ColumnCompression * GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression * c1,
+ ColumnCompression * c2, const char *attributeName);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 5cdaa3bff1..573512367a 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, char *name, char *desc,
extern Datum transformRelOptions(Datum oldOptions, List *defList,
char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,9 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern void freeRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 989fe738bb..9aad2f01b1 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -82,6 +84,9 @@ typedef struct tupleDesc
/* Accessor for the i'th attribute of tupdesc. */
#define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)])
+#define TupleDescAttrCompression(tupdesc, i) \
+ ((tupdesc)->tdcompression? &((tupdesc)->tdcompression[i]) : NULL)
+
extern TupleDesc CreateTemplateTupleDesc(int natts, bool hasoid);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..11092e5e9d 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -210,7 +210,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..b580f1971a 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(oid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..04c7c18d70 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -147,9 +147,9 @@ typedef FormData_pg_class *Form_pg_class;
* Note: "3" in the relfrozenxid column stands for FirstNormalTransactionId;
* similarly, "1" in relminmxid stands for FirstMultiXactId
*/
-DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 31 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..343d3355d9
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420)
+{
+ Oid cmid; /* compression method oid */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 3
+#define Anum_pg_compression_opt_cmid 1
+#define Anum_pg_compression_opt_cmhandler 2
+#define Anum_pg_compression_opt_cmoptions 3
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index d820b56aa1..5a2be99f0b 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3874,6 +3874,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4676,6 +4680,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index ffdb452b02..23921db534 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -199,6 +199,9 @@ CATALOG(pg_type,1247) BKI_BOOTSTRAP BKI_ROWTYPE_OID(71) BKI_SCHEMA_MACRO
*/
Oid typcollation;
+ /* Default compression method for the datatype or InvalidOid */
+ Oid typdefaultcm;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/*
@@ -236,7 +239,7 @@ typedef FormData_pg_type *Form_pg_type;
* compiler constants for pg_type
* ----------------
*/
-#define Natts_pg_type 30
+#define Natts_pg_type 31
#define Anum_pg_type_typname 1
#define Anum_pg_type_typnamespace 2
#define Anum_pg_type_typowner 3
@@ -264,9 +267,10 @@ typedef FormData_pg_type *Form_pg_type;
#define Anum_pg_type_typtypmod 25
#define Anum_pg_type_typndims 26
#define Anum_pg_type_typcollation 27
-#define Anum_pg_type_typdefaultbin 28
-#define Anum_pg_type_typdefault 29
-#define Anum_pg_type_typacl 30
+#define Anum_pg_type_typdefaultcm 28
+#define Anum_pg_type_typdefaultbin 29
+#define Anum_pg_type_typdefault 30
+#define Anum_pg_type_typacl 31
/* ----------------
@@ -283,102 +287,102 @@ typedef FormData_pg_type *Form_pg_type;
*/
/* OIDS 1 - 99 */
-DATA(insert OID = 16 ( bool PGNSP PGUID 1 t b B t t \054 0 0 1000 boolin boolout boolrecv boolsend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 16 ( bool PGNSP PGUID 1 t b B t t \054 0 0 1000 boolin boolout boolrecv boolsend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("boolean, 'true'/'false'");
#define BOOLOID 16
-DATA(insert OID = 17 ( bytea PGNSP PGUID -1 f b U f t \054 0 0 1001 byteain byteaout bytearecv byteasend - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 17 ( bytea PGNSP PGUID -1 f b U f t \054 0 0 1001 byteain byteaout bytearecv byteasend - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("variable-length string, binary values escaped");
#define BYTEAOID 17
-DATA(insert OID = 18 ( char PGNSP PGUID 1 t b S f t \054 0 0 1002 charin charout charrecv charsend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 18 ( char PGNSP PGUID 1 t b S f t \054 0 0 1002 charin charout charrecv charsend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("single character");
#define CHAROID 18
-DATA(insert OID = 19 ( name PGNSP PGUID NAMEDATALEN f b S f t \054 0 18 1003 namein nameout namerecv namesend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 19 ( name PGNSP PGUID NAMEDATALEN f b S f t \054 0 18 1003 namein nameout namerecv namesend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("63-byte type for storing system identifiers");
#define NAMEOID 19
-DATA(insert OID = 20 ( int8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 1016 int8in int8out int8recv int8send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 20 ( int8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 1016 int8in int8out int8recv int8send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("~18 digit integer, 8-byte storage");
#define INT8OID 20
-DATA(insert OID = 21 ( int2 PGNSP PGUID 2 t b N f t \054 0 0 1005 int2in int2out int2recv int2send - - - s p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 21 ( int2 PGNSP PGUID 2 t b N f t \054 0 0 1005 int2in int2out int2recv int2send - - - s p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("-32 thousand to 32 thousand, 2-byte storage");
#define INT2OID 21
-DATA(insert OID = 22 ( int2vector PGNSP PGUID -1 f b A f t \054 0 21 1006 int2vectorin int2vectorout int2vectorrecv int2vectorsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 22 ( int2vector PGNSP PGUID -1 f b A f t \054 0 21 1006 int2vectorin int2vectorout int2vectorrecv int2vectorsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("array of int2, used in system tables");
#define INT2VECTOROID 22
-DATA(insert OID = 23 ( int4 PGNSP PGUID 4 t b N f t \054 0 0 1007 int4in int4out int4recv int4send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 23 ( int4 PGNSP PGUID 4 t b N f t \054 0 0 1007 int4in int4out int4recv int4send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("-2 billion to 2 billion integer, 4-byte storage");
#define INT4OID 23
-DATA(insert OID = 24 ( regproc PGNSP PGUID 4 t b N f t \054 0 0 1008 regprocin regprocout regprocrecv regprocsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 24 ( regproc PGNSP PGUID 4 t b N f t \054 0 0 1008 regprocin regprocout regprocrecv regprocsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered procedure");
#define REGPROCOID 24
-DATA(insert OID = 25 ( text PGNSP PGUID -1 f b S t t \054 0 0 1009 textin textout textrecv textsend - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 25 ( text PGNSP PGUID -1 f b S t t \054 0 0 1009 textin textout textrecv textsend - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("variable-length string, no limit specified");
#define TEXTOID 25
-DATA(insert OID = 26 ( oid PGNSP PGUID 4 t b N t t \054 0 0 1028 oidin oidout oidrecv oidsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 26 ( oid PGNSP PGUID 4 t b N t t \054 0 0 1028 oidin oidout oidrecv oidsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("object identifier(oid), maximum 4 billion");
#define OIDOID 26
-DATA(insert OID = 27 ( tid PGNSP PGUID 6 f b U f t \054 0 0 1010 tidin tidout tidrecv tidsend - - - s p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 27 ( tid PGNSP PGUID 6 f b U f t \054 0 0 1010 tidin tidout tidrecv tidsend - - - s p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("(block, offset), physical location of tuple");
#define TIDOID 27
-DATA(insert OID = 28 ( xid PGNSP PGUID 4 t b U f t \054 0 0 1011 xidin xidout xidrecv xidsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 28 ( xid PGNSP PGUID 4 t b U f t \054 0 0 1011 xidin xidout xidrecv xidsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("transaction id");
#define XIDOID 28
-DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b U f t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 29 ( cid PGNSP PGUID 4 t b U f t \054 0 0 1012 cidin cidout cidrecv cidsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("command identifier type, sequence in transaction id");
#define CIDOID 29
-DATA(insert OID = 30 ( oidvector PGNSP PGUID -1 f b A f t \054 0 26 1013 oidvectorin oidvectorout oidvectorrecv oidvectorsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 30 ( oidvector PGNSP PGUID -1 f b A f t \054 0 26 1013 oidvectorin oidvectorout oidvectorrecv oidvectorsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("array of oids, used in system tables");
#define OIDVECTOROID 30
/* hand-built rowtype entries for bootstrapped catalogs */
/* NB: OIDs assigned here must match the BKI_ROWTYPE_OID declarations */
-DATA(insert OID = 71 ( pg_type PGNSP PGUID -1 f c C f t \054 1247 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 75 ( pg_attribute PGNSP PGUID -1 f c C f t \054 1249 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 81 ( pg_proc PGNSP PGUID -1 f c C f t \054 1255 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 83 ( pg_class PGNSP PGUID -1 f c C f t \054 1259 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 71 ( pg_type PGNSP PGUID -1 f c C f t \054 1247 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 75 ( pg_attribute PGNSP PGUID -1 f c C f t \054 1249 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 81 ( pg_proc PGNSP PGUID -1 f c C f t \054 1255 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 83 ( pg_class PGNSP PGUID -1 f c C f t \054 1259 0 0 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 100 - 199 */
-DATA(insert OID = 114 ( json PGNSP PGUID -1 f b U f t \054 0 0 199 json_in json_out json_recv json_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 114 ( json PGNSP PGUID -1 f b U f t \054 0 0 199 json_in json_out json_recv json_send - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define JSONOID 114
-DATA(insert OID = 142 ( xml PGNSP PGUID -1 f b U f t \054 0 0 143 xml_in xml_out xml_recv xml_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 142 ( xml PGNSP PGUID -1 f b U f t \054 0 0 143 xml_in xml_out xml_recv xml_send - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("XML content");
#define XMLOID 142
-DATA(insert OID = 143 ( _xml PGNSP PGUID -1 f b A f t \054 0 142 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 199 ( _json PGNSP PGUID -1 f b A f t \054 0 114 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 143 ( _xml PGNSP PGUID -1 f b A f t \054 0 142 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 199 ( _json PGNSP PGUID -1 f b A f t \054 0 114 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 194 ( pg_node_tree PGNSP PGUID -1 f b S f t \054 0 0 0 pg_node_tree_in pg_node_tree_out pg_node_tree_recv pg_node_tree_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 194 ( pg_node_tree PGNSP PGUID -1 f b S f t \054 0 0 0 pg_node_tree_in pg_node_tree_out pg_node_tree_recv pg_node_tree_send - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("string representing an internal node tree");
#define PGNODETREEOID 194
-DATA(insert OID = 3361 ( pg_ndistinct PGNSP PGUID -1 f b S f t \054 0 0 0 pg_ndistinct_in pg_ndistinct_out pg_ndistinct_recv pg_ndistinct_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 3361 ( pg_ndistinct PGNSP PGUID -1 f b S f t \054 0 0 0 pg_ndistinct_in pg_ndistinct_out pg_ndistinct_recv pg_ndistinct_send - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("multivariate ndistinct coefficients");
#define PGNDISTINCTOID 3361
-DATA(insert OID = 3402 ( pg_dependencies PGNSP PGUID -1 f b S f t \054 0 0 0 pg_dependencies_in pg_dependencies_out pg_dependencies_recv pg_dependencies_send - - - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 3402 ( pg_dependencies PGNSP PGUID -1 f b S f t \054 0 0 0 pg_dependencies_in pg_dependencies_out pg_dependencies_recv pg_dependencies_send - - - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("multivariate dependencies");
#define PGDEPENDENCIESOID 3402
-DATA(insert OID = 32 ( pg_ddl_command PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 pg_ddl_command_in pg_ddl_command_out pg_ddl_command_recv pg_ddl_command_send - - - ALIGNOF_POINTER p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 32 ( pg_ddl_command PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 pg_ddl_command_in pg_ddl_command_out pg_ddl_command_recv pg_ddl_command_send - - - ALIGNOF_POINTER p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("internal type for passing CollectedCommand");
#define PGDDLCOMMANDOID 32
/* OIDS 200 - 299 */
-DATA(insert OID = 210 ( smgr PGNSP PGUID 2 t b U f t \054 0 0 0 smgrin smgrout - - - - - s p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 210 ( smgr PGNSP PGUID 2 t b U f t \054 0 0 0 smgrin smgrout - - - - - s p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("storage manager");
/* OIDS 300 - 399 */
@@ -388,280 +392,280 @@ DESCR("storage manager");
/* OIDS 500 - 599 */
/* OIDS 600 - 699 */
-DATA(insert OID = 600 ( point PGNSP PGUID 16 f b G f t \054 0 701 1017 point_in point_out point_recv point_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 600 ( point PGNSP PGUID 16 f b G f t \054 0 701 1017 point_in point_out point_recv point_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric point '(x, y)'");
#define POINTOID 600
-DATA(insert OID = 601 ( lseg PGNSP PGUID 32 f b G f t \054 0 600 1018 lseg_in lseg_out lseg_recv lseg_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 601 ( lseg PGNSP PGUID 32 f b G f t \054 0 600 1018 lseg_in lseg_out lseg_recv lseg_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric line segment '(pt1,pt2)'");
#define LSEGOID 601
-DATA(insert OID = 602 ( path PGNSP PGUID -1 f b G f t \054 0 0 1019 path_in path_out path_recv path_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 602 ( path PGNSP PGUID -1 f b G f t \054 0 0 1019 path_in path_out path_recv path_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric path '(pt1,...)'");
#define PATHOID 602
-DATA(insert OID = 603 ( box PGNSP PGUID 32 f b G f t \073 0 600 1020 box_in box_out box_recv box_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 603 ( box PGNSP PGUID 32 f b G f t \073 0 600 1020 box_in box_out box_recv box_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric box '(lower left,upper right)'");
#define BOXOID 603
-DATA(insert OID = 604 ( polygon PGNSP PGUID -1 f b G f t \054 0 0 1027 poly_in poly_out poly_recv poly_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 604 ( polygon PGNSP PGUID -1 f b G f t \054 0 0 1027 poly_in poly_out poly_recv poly_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric polygon '(pt1,...)'");
#define POLYGONOID 604
-DATA(insert OID = 628 ( line PGNSP PGUID 24 f b G f t \054 0 701 629 line_in line_out line_recv line_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 628 ( line PGNSP PGUID 24 f b G f t \054 0 701 629 line_in line_out line_recv line_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric line");
#define LINEOID 628
-DATA(insert OID = 629 ( _line PGNSP PGUID -1 f b A f t \054 0 628 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 629 ( _line PGNSP PGUID -1 f b A f t \054 0 628 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 700 - 799 */
-DATA(insert OID = 700 ( float4 PGNSP PGUID 4 FLOAT4PASSBYVAL b N f t \054 0 0 1021 float4in float4out float4recv float4send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 700 ( float4 PGNSP PGUID 4 FLOAT4PASSBYVAL b N f t \054 0 0 1021 float4in float4out float4recv float4send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("single-precision floating point number, 4-byte storage");
#define FLOAT4OID 700
-DATA(insert OID = 701 ( float8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N t t \054 0 0 1022 float8in float8out float8recv float8send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 701 ( float8 PGNSP PGUID 8 FLOAT8PASSBYVAL b N t t \054 0 0 1022 float8in float8out float8recv float8send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("double-precision floating point number, 8-byte storage");
#define FLOAT8OID 701
-DATA(insert OID = 702 ( abstime PGNSP PGUID 4 t b D f t \054 0 0 1023 abstimein abstimeout abstimerecv abstimesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 702 ( abstime PGNSP PGUID 4 t b D f t \054 0 0 1023 abstimein abstimeout abstimerecv abstimesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("absolute, limited-range date and time (Unix system time)");
#define ABSTIMEOID 702
-DATA(insert OID = 703 ( reltime PGNSP PGUID 4 t b T f t \054 0 0 1024 reltimein reltimeout reltimerecv reltimesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 703 ( reltime PGNSP PGUID 4 t b T f t \054 0 0 1024 reltimein reltimeout reltimerecv reltimesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("relative, limited-range time interval (Unix delta time)");
#define RELTIMEOID 703
-DATA(insert OID = 704 ( tinterval PGNSP PGUID 12 f b T f t \054 0 0 1025 tintervalin tintervalout tintervalrecv tintervalsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 704 ( tinterval PGNSP PGUID 12 f b T f t \054 0 0 1025 tintervalin tintervalout tintervalrecv tintervalsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("(abstime,abstime), time interval");
#define TINTERVALOID 704
-DATA(insert OID = 705 ( unknown PGNSP PGUID -2 f p X f t \054 0 0 0 unknownin unknownout unknownrecv unknownsend - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 705 ( unknown PGNSP PGUID -2 f p X f t \054 0 0 0 unknownin unknownout unknownrecv unknownsend - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("");
#define UNKNOWNOID 705
-DATA(insert OID = 718 ( circle PGNSP PGUID 24 f b G f t \054 0 0 719 circle_in circle_out circle_recv circle_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 718 ( circle PGNSP PGUID 24 f b G f t \054 0 0 719 circle_in circle_out circle_recv circle_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("geometric circle '(center,radius)'");
#define CIRCLEOID 718
-DATA(insert OID = 719 ( _circle PGNSP PGUID -1 f b A f t \054 0 718 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 790 ( money PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 791 cash_in cash_out cash_recv cash_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 719 ( _circle PGNSP PGUID -1 f b A f t \054 0 718 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 790 ( money PGNSP PGUID 8 FLOAT8PASSBYVAL b N f t \054 0 0 791 cash_in cash_out cash_recv cash_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("monetary amounts, $d,ddd.cc");
#define CASHOID 790
-DATA(insert OID = 791 ( _money PGNSP PGUID -1 f b A f t \054 0 790 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 791 ( _money PGNSP PGUID -1 f b A f t \054 0 790 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 800 - 899 */
-DATA(insert OID = 829 ( macaddr PGNSP PGUID 6 f b U f t \054 0 0 1040 macaddr_in macaddr_out macaddr_recv macaddr_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 829 ( macaddr PGNSP PGUID 6 f b U f t \054 0 0 1040 macaddr_in macaddr_out macaddr_recv macaddr_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("XX:XX:XX:XX:XX:XX, MAC address");
#define MACADDROID 829
-DATA(insert OID = 869 ( inet PGNSP PGUID -1 f b I t t \054 0 0 1041 inet_in inet_out inet_recv inet_send - - - i m f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 869 ( inet PGNSP PGUID -1 f b I t t \054 0 0 1041 inet_in inet_out inet_recv inet_send - - - i m f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("IP address/netmask, host address, netmask optional");
#define INETOID 869
-DATA(insert OID = 650 ( cidr PGNSP PGUID -1 f b I f t \054 0 0 651 cidr_in cidr_out cidr_recv cidr_send - - - i m f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 650 ( cidr PGNSP PGUID -1 f b I f t \054 0 0 651 cidr_in cidr_out cidr_recv cidr_send - - - i m f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("network IP address/netmask, network address");
#define CIDROID 650
-DATA(insert OID = 774 ( macaddr8 PGNSP PGUID 8 f b U f t \054 0 0 775 macaddr8_in macaddr8_out macaddr8_recv macaddr8_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 774 ( macaddr8 PGNSP PGUID 8 f b U f t \054 0 0 775 macaddr8_in macaddr8_out macaddr8_recv macaddr8_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("XX:XX:XX:XX:XX:XX:XX:XX, MAC address");
#define MACADDR8OID 774
/* OIDS 900 - 999 */
/* OIDS 1000 - 1099 */
-DATA(insert OID = 1000 ( _bool PGNSP PGUID -1 f b A f t \054 0 16 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1001 ( _bytea PGNSP PGUID -1 f b A f t \054 0 17 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1002 ( _char PGNSP PGUID -1 f b A f t \054 0 18 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1003 ( _name PGNSP PGUID -1 f b A f t \054 0 19 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1005 ( _int2 PGNSP PGUID -1 f b A f t \054 0 21 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1000 ( _bool PGNSP PGUID -1 f b A f t \054 0 16 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1001 ( _bytea PGNSP PGUID -1 f b A f t \054 0 17 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1002 ( _char PGNSP PGUID -1 f b A f t \054 0 18 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1003 ( _name PGNSP PGUID -1 f b A f t \054 0 19 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1005 ( _int2 PGNSP PGUID -1 f b A f t \054 0 21 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INT2ARRAYOID 1005
-DATA(insert OID = 1006 ( _int2vector PGNSP PGUID -1 f b A f t \054 0 22 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1007 ( _int4 PGNSP PGUID -1 f b A f t \054 0 23 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1006 ( _int2vector PGNSP PGUID -1 f b A f t \054 0 22 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1007 ( _int4 PGNSP PGUID -1 f b A f t \054 0 23 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INT4ARRAYOID 1007
-DATA(insert OID = 1008 ( _regproc PGNSP PGUID -1 f b A f t \054 0 24 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1009 ( _text PGNSP PGUID -1 f b A f t \054 0 25 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1008 ( _regproc PGNSP PGUID -1 f b A f t \054 0 24 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1009 ( _text PGNSP PGUID -1 f b A f t \054 0 25 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
#define TEXTARRAYOID 1009
-DATA(insert OID = 1028 ( _oid PGNSP PGUID -1 f b A f t \054 0 26 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1028 ( _oid PGNSP PGUID -1 f b A f t \054 0 26 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define OIDARRAYOID 1028
-DATA(insert OID = 1010 ( _tid PGNSP PGUID -1 f b A f t \054 0 27 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1011 ( _xid PGNSP PGUID -1 f b A f t \054 0 28 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1012 ( _cid PGNSP PGUID -1 f b A f t \054 0 29 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1013 ( _oidvector PGNSP PGUID -1 f b A f t \054 0 30 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1014 ( _bpchar PGNSP PGUID -1 f b A f t \054 0 1042 0 array_in array_out array_recv array_send bpchartypmodin bpchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
-DATA(insert OID = 1015 ( _varchar PGNSP PGUID -1 f b A f t \054 0 1043 0 array_in array_out array_recv array_send varchartypmodin varchartypmodout array_typanalyze i x f 0 -1 0 100 _null_ _null_ _null_ ));
-DATA(insert OID = 1016 ( _int8 PGNSP PGUID -1 f b A f t \054 0 20 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1017 ( _point PGNSP PGUID -1 f b A f t \054 0 600 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1018 ( _lseg PGNSP PGUID -1 f b A f t \054 0 601 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1019 ( _path PGNSP PGUID -1 f b A f t \054 0 602 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1020 ( _box PGNSP PGUID -1 f b A f t \073 0 603 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1021 ( _float4 PGNSP PGUID -1 f b A f t \054 0 700 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1010 ( _tid PGNSP PGUID -1 f b A f t \054 0 27 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1011 ( _xid PGNSP PGUID -1 f b A f t \054 0 28 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1012 ( _cid PGNSP PGUID -1 f b A f t \054 0 29 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1013 ( _oidvector PGNSP PGUID -1 f b A f t \054 0 30 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1014 ( _bpchar PGNSP PGUID -1 f b A f t \054 0 1042 0 array_in array_out array_recv array_send bpchartypmodin bpchartypmodout array_typanalyze i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1015 ( _varchar PGNSP PGUID -1 f b A f t \054 0 1043 0 array_in array_out array_recv array_send varchartypmodin varchartypmodout array_typanalyze i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1016 ( _int8 PGNSP PGUID -1 f b A f t \054 0 20 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1017 ( _point PGNSP PGUID -1 f b A f t \054 0 600 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1018 ( _lseg PGNSP PGUID -1 f b A f t \054 0 601 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1019 ( _path PGNSP PGUID -1 f b A f t \054 0 602 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1020 ( _box PGNSP PGUID -1 f b A f t \073 0 603 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1021 ( _float4 PGNSP PGUID -1 f b A f t \054 0 700 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define FLOAT4ARRAYOID 1021
-DATA(insert OID = 1022 ( _float8 PGNSP PGUID -1 f b A f t \054 0 701 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1023 ( _abstime PGNSP PGUID -1 f b A f t \054 0 702 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1024 ( _reltime PGNSP PGUID -1 f b A f t \054 0 703 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1025 ( _tinterval PGNSP PGUID -1 f b A f t \054 0 704 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1027 ( _polygon PGNSP PGUID -1 f b A f t \054 0 604 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1033 ( aclitem PGNSP PGUID 12 f b U f t \054 0 0 1034 aclitemin aclitemout - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1022 ( _float8 PGNSP PGUID -1 f b A f t \054 0 701 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1023 ( _abstime PGNSP PGUID -1 f b A f t \054 0 702 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1024 ( _reltime PGNSP PGUID -1 f b A f t \054 0 703 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1025 ( _tinterval PGNSP PGUID -1 f b A f t \054 0 704 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1027 ( _polygon PGNSP PGUID -1 f b A f t \054 0 604 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1033 ( aclitem PGNSP PGUID 12 f b U f t \054 0 0 1034 aclitemin aclitemout - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("access control list");
#define ACLITEMOID 1033
-DATA(insert OID = 1034 ( _aclitem PGNSP PGUID -1 f b A f t \054 0 1033 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1040 ( _macaddr PGNSP PGUID -1 f b A f t \054 0 829 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 775 ( _macaddr8 PGNSP PGUID -1 f b A f t \054 0 774 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1041 ( _inet PGNSP PGUID -1 f b A f t \054 0 869 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 651 ( _cidr PGNSP PGUID -1 f b A f t \054 0 650 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1263 ( _cstring PGNSP PGUID -1 f b A f t \054 0 2275 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1034 ( _aclitem PGNSP PGUID -1 f b A f t \054 0 1033 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1040 ( _macaddr PGNSP PGUID -1 f b A f t \054 0 829 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 775 ( _macaddr8 PGNSP PGUID -1 f b A f t \054 0 774 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1041 ( _inet PGNSP PGUID -1 f b A f t \054 0 869 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 651 ( _cidr PGNSP PGUID -1 f b A f t \054 0 650 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1263 ( _cstring PGNSP PGUID -1 f b A f t \054 0 2275 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define CSTRINGARRAYOID 1263
-DATA(insert OID = 1042 ( bpchar PGNSP PGUID -1 f b S f t \054 0 0 1014 bpcharin bpcharout bpcharrecv bpcharsend bpchartypmodin bpchartypmodout - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1042 ( bpchar PGNSP PGUID -1 f b S f t \054 0 0 1014 bpcharin bpcharout bpcharrecv bpcharsend bpchartypmodin bpchartypmodout - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("char(length), blank-padded string, fixed storage length");
#define BPCHAROID 1042
-DATA(insert OID = 1043 ( varchar PGNSP PGUID -1 f b S f t \054 0 0 1015 varcharin varcharout varcharrecv varcharsend varchartypmodin varchartypmodout - i x f 0 -1 0 100 _null_ _null_ _null_ ));
+DATA(insert OID = 1043 ( varchar PGNSP PGUID -1 f b S f t \054 0 0 1015 varcharin varcharout varcharrecv varcharsend varchartypmodin varchartypmodout - i x f 0 -1 0 100 0 _null_ _null_ _null_ ));
DESCR("varchar(length), non-blank-padded string, variable storage length");
#define VARCHAROID 1043
-DATA(insert OID = 1082 ( date PGNSP PGUID 4 t b D f t \054 0 0 1182 date_in date_out date_recv date_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1082 ( date PGNSP PGUID 4 t b D f t \054 0 0 1182 date_in date_out date_recv date_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("date");
#define DATEOID 1082
-DATA(insert OID = 1083 ( time PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1183 time_in time_out time_recv time_send timetypmodin timetypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1083 ( time PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1183 time_in time_out time_recv time_send timetypmodin timetypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("time of day");
#define TIMEOID 1083
/* OIDS 1100 - 1199 */
-DATA(insert OID = 1114 ( timestamp PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1115 timestamp_in timestamp_out timestamp_recv timestamp_send timestamptypmodin timestamptypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1114 ( timestamp PGNSP PGUID 8 FLOAT8PASSBYVAL b D f t \054 0 0 1115 timestamp_in timestamp_out timestamp_recv timestamp_send timestamptypmodin timestamptypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("date and time");
#define TIMESTAMPOID 1114
-DATA(insert OID = 1115 ( _timestamp PGNSP PGUID -1 f b A f t \054 0 1114 0 array_in array_out array_recv array_send timestamptypmodin timestamptypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1182 ( _date PGNSP PGUID -1 f b A f t \054 0 1082 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1183 ( _time PGNSP PGUID -1 f b A f t \054 0 1083 0 array_in array_out array_recv array_send timetypmodin timetypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1184 ( timestamptz PGNSP PGUID 8 FLOAT8PASSBYVAL b D t t \054 0 0 1185 timestamptz_in timestamptz_out timestamptz_recv timestamptz_send timestamptztypmodin timestamptztypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1115 ( _timestamp PGNSP PGUID -1 f b A f t \054 0 1114 0 array_in array_out array_recv array_send timestamptypmodin timestamptypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1182 ( _date PGNSP PGUID -1 f b A f t \054 0 1082 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1183 ( _time PGNSP PGUID -1 f b A f t \054 0 1083 0 array_in array_out array_recv array_send timetypmodin timetypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1184 ( timestamptz PGNSP PGUID 8 FLOAT8PASSBYVAL b D t t \054 0 0 1185 timestamptz_in timestamptz_out timestamptz_recv timestamptz_send timestamptztypmodin timestamptztypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("date and time with time zone");
#define TIMESTAMPTZOID 1184
-DATA(insert OID = 1185 ( _timestamptz PGNSP PGUID -1 f b A f t \054 0 1184 0 array_in array_out array_recv array_send timestamptztypmodin timestamptztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1186 ( interval PGNSP PGUID 16 f b T t t \054 0 0 1187 interval_in interval_out interval_recv interval_send intervaltypmodin intervaltypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1185 ( _timestamptz PGNSP PGUID -1 f b A f t \054 0 1184 0 array_in array_out array_recv array_send timestamptztypmodin timestamptztypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1186 ( interval PGNSP PGUID 16 f b T t t \054 0 0 1187 interval_in interval_out interval_recv interval_send intervaltypmodin intervaltypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("@ <number> <units>, time interval");
#define INTERVALOID 1186
-DATA(insert OID = 1187 ( _interval PGNSP PGUID -1 f b A f t \054 0 1186 0 array_in array_out array_recv array_send intervaltypmodin intervaltypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1187 ( _interval PGNSP PGUID -1 f b A f t \054 0 1186 0 array_in array_out array_recv array_send intervaltypmodin intervaltypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 1200 - 1299 */
-DATA(insert OID = 1231 ( _numeric PGNSP PGUID -1 f b A f t \054 0 1700 0 array_in array_out array_recv array_send numerictypmodin numerictypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1266 ( timetz PGNSP PGUID 12 f b D f t \054 0 0 1270 timetz_in timetz_out timetz_recv timetz_send timetztypmodin timetztypmodout - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1231 ( _numeric PGNSP PGUID -1 f b A f t \054 0 1700 0 array_in array_out array_recv array_send numerictypmodin numerictypmodout array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1266 ( timetz PGNSP PGUID 12 f b D f t \054 0 0 1270 timetz_in timetz_out timetz_recv timetz_send timetztypmodin timetztypmodout - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("time of day with time zone");
#define TIMETZOID 1266
-DATA(insert OID = 1270 ( _timetz PGNSP PGUID -1 f b A f t \054 0 1266 0 array_in array_out array_recv array_send timetztypmodin timetztypmodout array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1270 ( _timetz PGNSP PGUID -1 f b A f t \054 0 1266 0 array_in array_out array_recv array_send timetztypmodin timetztypmodout array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 1500 - 1599 */
-DATA(insert OID = 1560 ( bit PGNSP PGUID -1 f b V f t \054 0 0 1561 bit_in bit_out bit_recv bit_send bittypmodin bittypmodout - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1560 ( bit PGNSP PGUID -1 f b V f t \054 0 0 1561 bit_in bit_out bit_recv bit_send bittypmodin bittypmodout - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("fixed-length bit string");
#define BITOID 1560
-DATA(insert OID = 1561 ( _bit PGNSP PGUID -1 f b A f t \054 0 1560 0 array_in array_out array_recv array_send bittypmodin bittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 1562 ( varbit PGNSP PGUID -1 f b V t t \054 0 0 1563 varbit_in varbit_out varbit_recv varbit_send varbittypmodin varbittypmodout - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1561 ( _bit PGNSP PGUID -1 f b A f t \054 0 1560 0 array_in array_out array_recv array_send bittypmodin bittypmodout array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1562 ( varbit PGNSP PGUID -1 f b V t t \054 0 0 1563 varbit_in varbit_out varbit_recv varbit_send varbittypmodin varbittypmodout - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("variable-length bit string");
#define VARBITOID 1562
-DATA(insert OID = 1563 ( _varbit PGNSP PGUID -1 f b A f t \054 0 1562 0 array_in array_out array_recv array_send varbittypmodin varbittypmodout array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1563 ( _varbit PGNSP PGUID -1 f b A f t \054 0 1562 0 array_in array_out array_recv array_send varbittypmodin varbittypmodout array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* OIDS 1600 - 1699 */
/* OIDS 1700 - 1799 */
-DATA(insert OID = 1700 ( numeric PGNSP PGUID -1 f b N f t \054 0 0 1231 numeric_in numeric_out numeric_recv numeric_send numerictypmodin numerictypmodout - i m f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1700 ( numeric PGNSP PGUID -1 f b N f t \054 0 0 1231 numeric_in numeric_out numeric_recv numeric_send numerictypmodin numerictypmodout - i m f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("numeric(precision, decimal), arbitrary precision number");
#define NUMERICOID 1700
-DATA(insert OID = 1790 ( refcursor PGNSP PGUID -1 f b U f t \054 0 0 2201 textin textout textrecv textsend - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 1790 ( refcursor PGNSP PGUID -1 f b U f t \054 0 0 2201 textin textout textrecv textsend - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("reference to cursor (portal name)");
#define REFCURSOROID 1790
/* OIDS 2200 - 2299 */
-DATA(insert OID = 2201 ( _refcursor PGNSP PGUID -1 f b A f t \054 0 1790 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2201 ( _refcursor PGNSP PGUID -1 f b A f t \054 0 1790 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2202 ( regprocedure PGNSP PGUID 4 t b N f t \054 0 0 2207 regprocedurein regprocedureout regprocedurerecv regproceduresend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2202 ( regprocedure PGNSP PGUID 4 t b N f t \054 0 0 2207 regprocedurein regprocedureout regprocedurerecv regproceduresend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered procedure (with args)");
#define REGPROCEDUREOID 2202
-DATA(insert OID = 2203 ( regoper PGNSP PGUID 4 t b N f t \054 0 0 2208 regoperin regoperout regoperrecv regopersend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2203 ( regoper PGNSP PGUID 4 t b N f t \054 0 0 2208 regoperin regoperout regoperrecv regopersend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered operator");
#define REGOPEROID 2203
-DATA(insert OID = 2204 ( regoperator PGNSP PGUID 4 t b N f t \054 0 0 2209 regoperatorin regoperatorout regoperatorrecv regoperatorsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2204 ( regoperator PGNSP PGUID 4 t b N f t \054 0 0 2209 regoperatorin regoperatorout regoperatorrecv regoperatorsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered operator (with args)");
#define REGOPERATOROID 2204
-DATA(insert OID = 2205 ( regclass PGNSP PGUID 4 t b N f t \054 0 0 2210 regclassin regclassout regclassrecv regclasssend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2205 ( regclass PGNSP PGUID 4 t b N f t \054 0 0 2210 regclassin regclassout regclassrecv regclasssend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered class");
#define REGCLASSOID 2205
-DATA(insert OID = 2206 ( regtype PGNSP PGUID 4 t b N f t \054 0 0 2211 regtypein regtypeout regtyperecv regtypesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2206 ( regtype PGNSP PGUID 4 t b N f t \054 0 0 2211 regtypein regtypeout regtyperecv regtypesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered type");
#define REGTYPEOID 2206
-DATA(insert OID = 4096 ( regrole PGNSP PGUID 4 t b N f t \054 0 0 4097 regrolein regroleout regrolerecv regrolesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4096 ( regrole PGNSP PGUID 4 t b N f t \054 0 0 4097 regrolein regroleout regrolerecv regrolesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered role");
#define REGROLEOID 4096
-DATA(insert OID = 4089 ( regnamespace PGNSP PGUID 4 t b N f t \054 0 0 4090 regnamespacein regnamespaceout regnamespacerecv regnamespacesend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4089 ( regnamespace PGNSP PGUID 4 t b N f t \054 0 0 4090 regnamespacein regnamespaceout regnamespacerecv regnamespacesend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered namespace");
#define REGNAMESPACEOID 4089
-DATA(insert OID = 2207 ( _regprocedure PGNSP PGUID -1 f b A f t \054 0 2202 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2208 ( _regoper PGNSP PGUID -1 f b A f t \054 0 2203 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2209 ( _regoperator PGNSP PGUID -1 f b A f t \054 0 2204 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2210 ( _regclass PGNSP PGUID -1 f b A f t \054 0 2205 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2211 ( _regtype PGNSP PGUID -1 f b A f t \054 0 2206 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2207 ( _regprocedure PGNSP PGUID -1 f b A f t \054 0 2202 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2208 ( _regoper PGNSP PGUID -1 f b A f t \054 0 2203 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2209 ( _regoperator PGNSP PGUID -1 f b A f t \054 0 2204 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2210 ( _regclass PGNSP PGUID -1 f b A f t \054 0 2205 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2211 ( _regtype PGNSP PGUID -1 f b A f t \054 0 2206 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define REGTYPEARRAYOID 2211
-DATA(insert OID = 4097 ( _regrole PGNSP PGUID -1 f b A f t \054 0 4096 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 4090 ( _regnamespace PGNSP PGUID -1 f b A f t \054 0 4089 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4097 ( _regrole PGNSP PGUID -1 f b A f t \054 0 4096 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 4090 ( _regnamespace PGNSP PGUID -1 f b A f t \054 0 4089 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* uuid */
-DATA(insert OID = 2950 ( uuid PGNSP PGUID 16 f b U f t \054 0 0 2951 uuid_in uuid_out uuid_recv uuid_send - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2950 ( uuid PGNSP PGUID 16 f b U f t \054 0 0 2951 uuid_in uuid_out uuid_recv uuid_send - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("UUID datatype");
#define UUIDOID 2950
-DATA(insert OID = 2951 ( _uuid PGNSP PGUID -1 f b A f t \054 0 2950 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2951 ( _uuid PGNSP PGUID -1 f b A f t \054 0 2950 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* pg_lsn */
-DATA(insert OID = 3220 ( pg_lsn PGNSP PGUID 8 FLOAT8PASSBYVAL b U f t \054 0 0 3221 pg_lsn_in pg_lsn_out pg_lsn_recv pg_lsn_send - - - d p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3220 ( pg_lsn PGNSP PGUID 8 FLOAT8PASSBYVAL b U f t \054 0 0 3221 pg_lsn_in pg_lsn_out pg_lsn_recv pg_lsn_send - - - d p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("PostgreSQL LSN datatype");
#define LSNOID 3220
-DATA(insert OID = 3221 ( _pg_lsn PGNSP PGUID -1 f b A f t \054 0 3220 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3221 ( _pg_lsn PGNSP PGUID -1 f b A f t \054 0 3220 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* text search */
-DATA(insert OID = 3614 ( tsvector PGNSP PGUID -1 f b U f t \054 0 0 3643 tsvectorin tsvectorout tsvectorrecv tsvectorsend - - ts_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3614 ( tsvector PGNSP PGUID -1 f b U f t \054 0 0 3643 tsvectorin tsvectorout tsvectorrecv tsvectorsend - - ts_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("text representation for text search");
#define TSVECTOROID 3614
-DATA(insert OID = 3642 ( gtsvector PGNSP PGUID -1 f b U f t \054 0 0 3644 gtsvectorin gtsvectorout - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3642 ( gtsvector PGNSP PGUID -1 f b U f t \054 0 0 3644 gtsvectorin gtsvectorout - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("GiST index internal text representation for text search");
#define GTSVECTOROID 3642
-DATA(insert OID = 3615 ( tsquery PGNSP PGUID -1 f b U f t \054 0 0 3645 tsqueryin tsqueryout tsqueryrecv tsquerysend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3615 ( tsquery PGNSP PGUID -1 f b U f t \054 0 0 3645 tsqueryin tsqueryout tsqueryrecv tsquerysend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("query representation for text search");
#define TSQUERYOID 3615
-DATA(insert OID = 3734 ( regconfig PGNSP PGUID 4 t b N f t \054 0 0 3735 regconfigin regconfigout regconfigrecv regconfigsend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3734 ( regconfig PGNSP PGUID 4 t b N f t \054 0 0 3735 regconfigin regconfigout regconfigrecv regconfigsend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered text search configuration");
#define REGCONFIGOID 3734
-DATA(insert OID = 3769 ( regdictionary PGNSP PGUID 4 t b N f t \054 0 0 3770 regdictionaryin regdictionaryout regdictionaryrecv regdictionarysend - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3769 ( regdictionary PGNSP PGUID 4 t b N f t \054 0 0 3770 regdictionaryin regdictionaryout regdictionaryrecv regdictionarysend - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("registered text search dictionary");
#define REGDICTIONARYOID 3769
-DATA(insert OID = 3643 ( _tsvector PGNSP PGUID -1 f b A f t \054 0 3614 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3644 ( _gtsvector PGNSP PGUID -1 f b A f t \054 0 3642 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3645 ( _tsquery PGNSP PGUID -1 f b A f t \054 0 3615 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3735 ( _regconfig PGNSP PGUID -1 f b A f t \054 0 3734 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3770 ( _regdictionary PGNSP PGUID -1 f b A f t \054 0 3769 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3643 ( _tsvector PGNSP PGUID -1 f b A f t \054 0 3614 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3644 ( _gtsvector PGNSP PGUID -1 f b A f t \054 0 3642 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3645 ( _tsquery PGNSP PGUID -1 f b A f t \054 0 3615 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3735 ( _regconfig PGNSP PGUID -1 f b A f t \054 0 3734 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3770 ( _regdictionary PGNSP PGUID -1 f b A f t \054 0 3769 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* jsonb */
-DATA(insert OID = 3802 ( jsonb PGNSP PGUID -1 f b U f t \054 0 0 3807 jsonb_in jsonb_out jsonb_recv jsonb_send - - - i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3802 ( jsonb PGNSP PGUID -1 f b U f t \054 0 0 3807 jsonb_in jsonb_out jsonb_recv jsonb_send - - - i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("Binary JSON");
#define JSONBOID 3802
-DATA(insert OID = 3807 ( _jsonb PGNSP PGUID -1 f b A f t \054 0 3802 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3807 ( _jsonb PGNSP PGUID -1 f b A f t \054 0 3802 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 2970 ( txid_snapshot PGNSP PGUID -1 f b U f t \054 0 0 2949 txid_snapshot_in txid_snapshot_out txid_snapshot_recv txid_snapshot_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2970 ( txid_snapshot PGNSP PGUID -1 f b U f t \054 0 0 2949 txid_snapshot_in txid_snapshot_out txid_snapshot_recv txid_snapshot_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("txid snapshot");
-DATA(insert OID = 2949 ( _txid_snapshot PGNSP PGUID -1 f b A f t \054 0 2970 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2949 ( _txid_snapshot PGNSP PGUID -1 f b A f t \054 0 2970 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/* range types */
-DATA(insert OID = 3904 ( int4range PGNSP PGUID -1 f r R f t \054 0 0 3905 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3904 ( int4range PGNSP PGUID -1 f r R f t \054 0 0 3905 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of integers");
#define INT4RANGEOID 3904
-DATA(insert OID = 3905 ( _int4range PGNSP PGUID -1 f b A f t \054 0 3904 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3906 ( numrange PGNSP PGUID -1 f r R f t \054 0 0 3907 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3905 ( _int4range PGNSP PGUID -1 f b A f t \054 0 3904 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3906 ( numrange PGNSP PGUID -1 f r R f t \054 0 0 3907 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of numerics");
-DATA(insert OID = 3907 ( _numrange PGNSP PGUID -1 f b A f t \054 0 3906 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3908 ( tsrange PGNSP PGUID -1 f r R f t \054 0 0 3909 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3907 ( _numrange PGNSP PGUID -1 f b A f t \054 0 3906 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3908 ( tsrange PGNSP PGUID -1 f r R f t \054 0 0 3909 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of timestamps without time zone");
-DATA(insert OID = 3909 ( _tsrange PGNSP PGUID -1 f b A f t \054 0 3908 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3910 ( tstzrange PGNSP PGUID -1 f r R f t \054 0 0 3911 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3909 ( _tsrange PGNSP PGUID -1 f b A f t \054 0 3908 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3910 ( tstzrange PGNSP PGUID -1 f r R f t \054 0 0 3911 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of timestamps with time zone");
-DATA(insert OID = 3911 ( _tstzrange PGNSP PGUID -1 f b A f t \054 0 3910 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3912 ( daterange PGNSP PGUID -1 f r R f t \054 0 0 3913 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3911 ( _tstzrange PGNSP PGUID -1 f b A f t \054 0 3910 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3912 ( daterange PGNSP PGUID -1 f r R f t \054 0 0 3913 range_in range_out range_recv range_send - - range_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of dates");
-DATA(insert OID = 3913 ( _daterange PGNSP PGUID -1 f b A f t \054 0 3912 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 _null_ _null_ _null_ ));
-DATA(insert OID = 3926 ( int8range PGNSP PGUID -1 f r R f t \054 0 0 3927 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3913 ( _daterange PGNSP PGUID -1 f b A f t \054 0 3912 0 array_in array_out array_recv array_send - - array_typanalyze i x f 0 -1 0 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3926 ( int8range PGNSP PGUID -1 f r R f t \054 0 0 3927 range_in range_out range_recv range_send - - range_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
DESCR("range of bigints");
-DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
/*
* pseudo-types
@@ -676,42 +680,44 @@ DATA(insert OID = 3927 ( _int8range PGNSP PGUID -1 f b A f t \054 0 3926 0 arr
* but there is now support for it in records and arrays. Perhaps we should
* just treat it as a regular base type?
*/
-DATA(insert OID = 2249 ( record PGNSP PGUID -1 f p P f t \054 0 0 2287 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2249 ( record PGNSP PGUID -1 f p P f t \054 0 0 2287 record_in record_out record_recv record_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define RECORDOID 2249
-DATA(insert OID = 2287 ( _record PGNSP PGUID -1 f p P f t \054 0 2249 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2287 ( _record PGNSP PGUID -1 f p P f t \054 0 2249 0 array_in array_out array_recv array_send - - array_typanalyze d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define RECORDARRAYOID 2287
-DATA(insert OID = 2275 ( cstring PGNSP PGUID -2 f p P f t \054 0 0 1263 cstring_in cstring_out cstring_recv cstring_send - - - c p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2275 ( cstring PGNSP PGUID -2 f p P f t \054 0 0 1263 cstring_in cstring_out cstring_recv cstring_send - - - c p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define CSTRINGOID 2275
-DATA(insert OID = 2276 ( any PGNSP PGUID 4 t p P f t \054 0 0 0 any_in any_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2276 ( any PGNSP PGUID 4 t p P f t \054 0 0 0 any_in any_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYOID 2276
-DATA(insert OID = 2277 ( anyarray PGNSP PGUID -1 f p P f t \054 0 0 0 anyarray_in anyarray_out anyarray_recv anyarray_send - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2277 ( anyarray PGNSP PGUID -1 f p P f t \054 0 0 0 anyarray_in anyarray_out anyarray_recv anyarray_send - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYARRAYOID 2277
-DATA(insert OID = 2278 ( void PGNSP PGUID 4 t p P f t \054 0 0 0 void_in void_out void_recv void_send - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2278 ( void PGNSP PGUID 4 t p P f t \054 0 0 0 void_in void_out void_recv void_send - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define VOIDOID 2278
-DATA(insert OID = 2279 ( trigger PGNSP PGUID 4 t p P f t \054 0 0 0 trigger_in trigger_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2279 ( trigger PGNSP PGUID 4 t p P f t \054 0 0 0 trigger_in trigger_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define TRIGGEROID 2279
-DATA(insert OID = 3838 ( event_trigger PGNSP PGUID 4 t p P f t \054 0 0 0 event_trigger_in event_trigger_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3838 ( event_trigger PGNSP PGUID 4 t p P f t \054 0 0 0 event_trigger_in event_trigger_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define EVTTRIGGEROID 3838
-DATA(insert OID = 2280 ( language_handler PGNSP PGUID 4 t p P f t \054 0 0 0 language_handler_in language_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2280 ( language_handler PGNSP PGUID 4 t p P f t \054 0 0 0 language_handler_in language_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define LANGUAGE_HANDLEROID 2280
-DATA(insert OID = 2281 ( internal PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 internal_in internal_out - - - - - ALIGNOF_POINTER p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2281 ( internal PGNSP PGUID SIZEOF_POINTER t p P f t \054 0 0 0 internal_in internal_out - - - - - ALIGNOF_POINTER p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INTERNALOID 2281
-DATA(insert OID = 2282 ( opaque PGNSP PGUID 4 t p P f t \054 0 0 0 opaque_in opaque_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2282 ( opaque PGNSP PGUID 4 t p P f t \054 0 0 0 opaque_in opaque_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define OPAQUEOID 2282
-DATA(insert OID = 2283 ( anyelement PGNSP PGUID 4 t p P f t \054 0 0 0 anyelement_in anyelement_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2283 ( anyelement PGNSP PGUID 4 t p P f t \054 0 0 0 anyelement_in anyelement_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYELEMENTOID 2283
-DATA(insert OID = 2776 ( anynonarray PGNSP PGUID 4 t p P f t \054 0 0 0 anynonarray_in anynonarray_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 2776 ( anynonarray PGNSP PGUID 4 t p P f t \054 0 0 0 anynonarray_in anynonarray_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYNONARRAYOID 2776
-DATA(insert OID = 3500 ( anyenum PGNSP PGUID 4 t p P f t \054 0 0 0 anyenum_in anyenum_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3500 ( anyenum PGNSP PGUID 4 t p P f t \054 0 0 0 anyenum_in anyenum_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYENUMOID 3500
-DATA(insert OID = 3115 ( fdw_handler PGNSP PGUID 4 t p P f t \054 0 0 0 fdw_handler_in fdw_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3115 ( fdw_handler PGNSP PGUID 4 t p P f t \054 0 0 0 fdw_handler_in fdw_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define FDW_HANDLEROID 3115
-DATA(insert OID = 325 ( index_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 index_am_handler_in index_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 325 ( index_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 index_am_handler_in index_am_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define INDEX_AM_HANDLEROID 325
-DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_handler_in tsm_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_handler_in tsm_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define TSM_HANDLEROID 3310
-DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
+DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index f7bb4a54f7..01c542e829 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index da3ff5dbee..c50d9525d0 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -24,7 +24,8 @@
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString);
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt);
extern void RemoveRelations(DropStmt *drop);
diff --git a/src/include/commands/typecmds.h b/src/include/commands/typecmds.h
index 34f6fe328f..d198fd1d05 100644
--- a/src/include/commands/typecmds.h
+++ b/src/include/commands/typecmds.h
@@ -53,5 +53,6 @@ extern Oid AlterTypeNamespaceInternal(Oid typeOid, Oid nspOid,
bool isImplicitArray,
bool errorOnTableType,
ObjectAddresses *objsMoved);
+extern void AlterType(AlterTypeStmt * stmt);
#endif /* TYPECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 27bd4f3363..6c1c6350e3 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -413,6 +413,8 @@ typedef enum NodeTag
T_DropSubscriptionStmt,
T_CreateStatsStmt,
T_AlterCollationStmt,
+ T_AlterTypeStmt,
+ T_AlterTypeCmd,
/*
* TAGS FOR PARSE TREE NODES (parsenodes.h)
@@ -468,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
/*
* TAGS FOR REPLICATION GRAMMAR PARSE NODES (replnodes.h)
@@ -497,7 +500,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f2a5ab12f6..16aa08f8e8 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,14 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ Oid methodOid;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1616,6 +1625,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1762,7 +1772,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
@@ -3432,4 +3443,24 @@ typedef struct DropSubscriptionStmt
DropBehavior behavior; /* RESTRICT or CASCADE behavior */
} DropSubscriptionStmt;
+typedef enum AlterTypeCmdType
+{
+ AT_AlterTypeCompression, /* ALTER TYPE name COMPRESSED cm WITH
+ * (options) */
+} AlterTypeCmdType;
+
+typedef struct AlterTypeCmd
+{
+ NodeTag type;
+ AlterTypeCmdType cmdtype;
+ Node *def;
+} AlterTypeCmd;
+
+typedef struct AlterTypeStmt
+{
+ NodeTag type;
+ List *typeName;
+ List *cmds;
+} AlterTypeStmt;
+
#endif /* PARSENODES_H */
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..f5c879ae60 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -146,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -282,7 +291,12 @@ typedef struct
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +325,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 43273eaab5..fa3bbbc49e 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -202,6 +202,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/lsyscache.h b/src/include/utils/lsyscache.h
index 07208b56ce..00db7fead1 100644
--- a/src/include/utils/lsyscache.h
+++ b/src/include/utils/lsyscache.h
@@ -166,6 +166,7 @@ extern void getTypeBinaryOutputInfo(Oid type, Oid *typSend, bool *typIsVarlena);
extern Oid get_typmodin(Oid typid);
extern Oid get_typcollation(Oid typid);
extern bool type_is_collatable(Oid typid);
+extern Oid get_base_typdefaultcm(HeapTuple typtup);
extern Oid getBaseType(Oid typid);
extern Oid getBaseTypeAndTypmod(Oid typid, int32 *typmod);
extern int32 get_typavgwidth(Oid typid, int32 typmod);
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a92ea27ac..889f9c775a 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..47c071abb2
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,127 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table cmtest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method for tsvector doesn't take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+NOTICE: merging column "fts" with inherited definition
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+ALTER TYPE tsvector SET COMPRESSED ts1;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+DROP TABLE cmtest;
+ALTER TYPE tsvector SET NOT COMPRESSED;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+DROP TABLE cmtest;
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 5 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 58c755be50..5f04ca3e2a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -547,10 +547,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -669,11 +669,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -696,11 +696,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -725,46 +725,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (MAXVALUE, 0, 0);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (MAXVALUE, 0, 0)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (1, MAXVALUE, 0);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, 0, 0) TO (1, MAXVALUE, 0)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, 0);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, 0)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 3acc696863..fe873fa5e1 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -296,10 +296,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index c6e558b07f..f147740a87 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1314,12 +1314,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1335,12 +1335,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1359,12 +1359,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1402,12 +1402,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1427,17 +1427,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1459,17 +1459,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1501,17 +1501,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1539,12 +1539,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1576,12 +1576,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1620,12 +1620,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1651,12 +1651,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1678,12 +1678,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1705,12 +1705,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1735,12 +1735,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1766,12 +1766,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 1fa9650ec9..b472007866 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1001,13 +1001,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1020,14 +1020,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1037,14 +1037,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1084,33 +1084,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1121,27 +1121,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1151,37 +1151,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 73a5600f19..18699c5069 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -389,10 +389,10 @@ drop table range_parted, list_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -691,74 +691,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, 0) TO ('b', MINVALUE),
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, 0)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, 0) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index e996640593..62b06da011 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index cef70b1a1e..a652883172 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 2fd3f2b1b1..9a4bacc54b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 76b0de30a7..b49808e814 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..50fafe36d7
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,47 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+
+SELECT length(fts) FROM cmtest2;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+
+ALTER TYPE tsvector SET COMPRESSED ts1;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+DROP TABLE cmtest;
+
+ALTER TYPE tsvector SET NOT COMPRESSED;
+CREATE TABLE cmtest(fts tsvector);
+\d+ cmtest
+DROP TABLE cmtest;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
Re: Custom compression methods
On 9/12/17 10:55, Ildus Kurbangaliev wrote:
The patch also includes custom compression method for tsvector which
is used in tests.[1]
/messages/by-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx+Sig4ykWzWEAfkC6ZKMDy6=Q@mail.gmail.comAttached rebased version of the patch. Added support of pg_dump, the
code was simplified, and a separate cache for compression options was
added.
I would like to see some more examples of how this would be used, so we
can see how it should all fit together.
So far, it's not clear to me that we need a compression method as a
standalone top-level object. It would make sense, perhaps, to have a
compression function attached to a type, so a type can provide a
compression function that is suitable for its specific storage.
The proposal here is very general: You can use any of the eligible
compression methods for any attribute. That seems very complicated to
manage. Any attribute could be compressed using either a choice of
general compression methods or a type-specific compression method, or
perhaps another type-specific compression method. That's a lot. Is
this about packing certain types better, or trying out different
compression algorithms, or about changing the TOAST thresholds, and so on?
Ideally, we would like something that just works, with minimal
configuration and nudging. Let's see a list of problems to be solved
and then we can discuss what the right set of primitives might be to
address them.
--
Peter Eisentraut http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
On Wed, 1 Nov 2017 17:05:58 -0400
Peter Eisentraut <peter.eisentraut@2ndquadrant.com> wrote:
On 9/12/17 10:55, Ildus Kurbangaliev wrote:
The patch also includes custom compression method for tsvector
which is used in tests.[1]
/messages/by-id/CAPpHfdsdTA5uZeq6MNXL5ZRuNx+Sig4ykWzWEAfkC6ZKMDy6=Q@mail.gmail.comAttached rebased version of the patch. Added support of pg_dump, the
code was simplified, and a separate cache for compression options
was added.I would like to see some more examples of how this would be used, so
we can see how it should all fit together.So far, it's not clear to me that we need a compression method as a
standalone top-level object. It would make sense, perhaps, to have a
compression function attached to a type, so a type can provide a
compression function that is suitable for its specific storage.
In this patch compression methods is suitable for MAIN and EXTENDED
storages like in current implementation in postgres. Just instead only
of LZ4 you can specify any other compression method.
Idea is not to change compression for some types, but give the user and
extension developers opportunity to change how data in some attribute
will be compressed because they know about it more than database itself.
The proposal here is very general: You can use any of the eligible
compression methods for any attribute. That seems very complicated to
manage. Any attribute could be compressed using either a choice of
general compression methods or a type-specific compression method, or
perhaps another type-specific compression method. That's a lot. Is
this about packing certain types better, or trying out different
compression algorithms, or about changing the TOAST thresholds, and
so on?
It is about extensibility of postgres, for example if you
need to store a lot of time series data you can create an extension that
stores array of timestamps in more optimized way, using delta encoding
or something else. I'm not sure that such specialized things should be
in core.
In case of array of timestamps in could look like this:
CREATE EXTENSION timeseries; -- some extension that provides compression
method
Extension installs a compression method:
CREATE OR REPLACE FUNCTION timestamps_compression_handler(INTERNAL)
RETURNS COMPRESSION_HANDLER AS 'MODULE_PATHNAME',
'timestamps_compression_handler' LANGUAGE C STRICT;
CREATE COMPRESSION METHOD cm1 HANDLER timestamps_compression_handler;
And user can specify it in his table:
CREATE TABLE t1 (
time_series_data timestamp[] COMPRESSED cm1;
)
I think generalization of some method to a type is not a good idea. For
some attribute you could be happy with builtin LZ4, for other you can
need more compressibility and so on.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
On Tue, 12 Sep 2017 17:55:05 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
Attached rebased version of the patch. Added support of pg_dump, the
code was simplified, and a separate cache for compression options was
added.
Attached version 3 of the patch. Rebased to the current master, removed
ALTER TYPE .. SET COMPRESSED syntax, fixed bug in compression options
cache.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v3.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 01acc2ef9d..f43f09cc19 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -59,6 +59,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 41acda003f..edc46a2966 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,8 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET NOT COMPRESSED
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +322,34 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. Setting a compression method doesn't change anything in the
+ table and affects only future table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>SET NOT COMPRESSED</literal>
+ </term>
+ <listitem>
+ <para>
+ This form removes compression from a column. Removing compresssion from
+ a column doesn't change already compressed tuples and affects only future
+ table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..663010ecd9
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,50 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="SQL-CREATECOMPRESSIONMETHOD">
+ <indexterm zone="sql-createcompressionmethod">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">compression_method_name</replaceable>
+ HANDLER <replaceable class="parameter">compression_method_handler</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method
+ with <replaceable class="parameter">compression_method_name</replaceable>.
+ </para>
+
+ <para>
+ A compression method links a name with a compression handler. And the
+ handler is a special function that returns collection of methods that
+ can be used for compression.
+ </para>
+
+ <para>
+ After a compression method is created, you can specify it in
+ <xref linkend="SQL-CREATETABLE"> or <xref linkend="SQL-ALTERTABLE">
+ statements.
+ </para>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 4f7b741526..8ae763db8b 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -783,6 +784,18 @@ FROM ( { <replaceable class="parameter">numeric_literal</replaceable> | <replace
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index 9000b3aaaa..cc0bd70be3 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -87,6 +87,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..9efa5a9648 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -92,7 +92,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index ec10762529..21e375ad96 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -935,11 +935,48 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
+void
+freeRelOptions(List *options)
+{
+ ListCell *cell;
+
+ Assert(options != NIL);
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ pfree(def->defname);
+ pfree(defGetString(def));
+ pfree(def->arg);
+ }
+ list_free_deep(options);
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca73a8..d206cce18e 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -242,6 +244,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -396,6 +399,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -458,6 +463,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -563,6 +569,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -675,7 +682,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 5a8f1dab83..bd50ef9af9 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -39,6 +41,8 @@
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
#include "utils/typcache.h"
@@ -53,19 +57,46 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+static HTAB *compression_options_htab = NULL;
+static MemoryContext compression_options_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ ((toast_compress_header *) (ptr))->info &= 0xC0000000; \
+ ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +114,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_compression_options_htab(void);
+static AttributeCompression *get_compression_options_info(Oid cmoptoid);
/* ----------
@@ -741,6 +774,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
@@ -770,10 +805,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +950,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1229,7 +1266,9 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
if (VARATT_IS_EXTERNAL(new_value) ||
VARATT_IS_COMPRESSED(new_value))
{
- new_value = heap_tuple_untoast_attr(new_value);
+ struct varlena *untoasted_value = heap_tuple_untoast_attr(new_value);
+
+ new_value = untoasted_value;
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
}
@@ -1353,7 +1392,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,25 +1406,43 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoptoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize,
+ len = 0;
+ AttributeCompression *ac = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
+ if (OidIsValid(cmoptoid))
+ ac = get_compression_options_info(cmoptoid);
+
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ if (!ac && (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size))
return PointerGetDatum(NULL);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ if (ac)
+ {
+ tmp = ac->routine->compress(ac, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
* We recheck the actual size even if pglz_compress() reports success,
@@ -1398,11 +1454,7 @@ toast_compress_datum(Datum value)
* only one header byte and no padding if the value is short enough. So
* we insist on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!ac && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,10 +1462,20 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (ac && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, ac->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
- pfree(tmp);
+ if (tmp)
+ pfree(tmp);
+
return PointerGetDatum(NULL);
}
}
@@ -2280,15 +2342,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ AttributeCompression *ac;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ ac = get_compression_options_info(hdr->cmoptoid);
+ result = ac->routine->decompress(ac, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2463,44 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+static void
+init_compression_options_htab(void)
+{
+ HASHCTL ctl;
+
+ compression_options_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "compression options cache context",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(AttributeCompression);
+ ctl.hcxt = compression_options_mcxt;
+ compression_options_htab = hash_create("compression options cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+}
+
+static AttributeCompression *
+get_compression_options_info(Oid cmoptoid)
+{
+ bool found;
+ AttributeCompression *result;
+
+ Assert(OidIsValid(cmoptoid));
+ if (!compression_options_htab)
+ init_compression_options_htab();
+
+ result = hash_search(compression_options_htab, &cmoptoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ Assert(compression_options_mcxt);
+ oldcxt = MemoryContextSwitchTo(compression_options_mcxt);
+ result->cmoptoid = cmoptoid;
+ result->routine = GetCompressionRoutine(cmoptoid);
+ result->options = GetCompressionOptionsList(cmoptoid);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ return result;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 8287de97a2..be6b460aee 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 033c4358ea..e1bfc7c6bf 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2512,6 +2524,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 256a9c9c93..c5838fa779 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -451,6 +451,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 05e70818e7..b08e857e3a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -707,6 +712,13 @@ AddNewAttributeTuples(Oid new_rel_oid,
referenced.objectSubId = 0;
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+
+ if (OidIsValid(attr->attcompression))
+ {
+ ObjectAddressSet(referenced, CompressionOptRelationId,
+ attr->attcompression);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
}
/*
@@ -1453,6 +1465,22 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+static void
+DropAttributeCompression(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ bool attisdropped = att->attisdropped;
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ att->attisdropped = true;
+ cmr->drop(att, options);
+ att->attisdropped = attisdropped;
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1511,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ DropAttributeCompression(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1611,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ DropAttributeCompression(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1634,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c7b2f031f0..11dc107ab7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index c2ad7c675e..0e1497cf7c 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index 59ffd2104d..8d65c5fbe8 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..32cbcc3efe
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,485 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/compression.h"
+#include "access/reloptions.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine *get_compression_method_routine(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+LookupCompressionHandlerFunc(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+static ObjectAddress
+CreateCompressionMethod(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = LookupCompressionHandlerFunc(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ return CreateCompressionMethod(cmName, (List *) handlerEl->arg);
+}
+
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, List *options)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Oid result,
+ cmhandler;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+ ObjectAddress myself,
+ ref1,
+ ref2,
+ ref3;
+ CompressionMethodRoutine *routine;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmhandler = ((Form_pg_compression) GETSTRUCT(tuple))->cmhandler;
+ ReleaseSysCache(tuple);
+
+ routine = get_compression_method_routine(cmhandler, attr->atttypid);
+
+ if (routine->configure && options != NIL)
+ routine->configure(attr, options);
+
+ values[Anum_pg_compression_opt_cmid - 1] = ObjectIdGetDatum(cmid);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ if (options)
+ values[Anum_pg_compression_opt_cmoptions - 1] = optionListToArray(options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+ tuple = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ result = CatalogTupleInsert(rel, tuple);
+ heap_freetuple(tuple);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, result);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmhandler);
+ ObjectAddressSubSet(ref2, RelationRelationId, attr->attrelid, attr->attnum);
+ ObjectAddressSet(ref3, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&ref2, &myself, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref3, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return result;
+}
+
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+ ReleaseSysCache(tup);
+ heap_close(relation, RowExclusiveLock);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = NULL;
+ compression->methodOid = cmopt->cmid;
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression * c1, ColumnCompression * c2,
+ const char *attributeName)
+{
+ char *cmname1 = c1->methodName ? c1->methodName :
+ get_compression_method_name(c1->methodOid);
+ char *cmname2 = c2->methodName ? c2->methodName :
+ get_compression_method_name(c2->methodOid);
+
+ if (strcmp(cmname1, cmname2))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", cmname1, cmname2)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ Oid cmid;
+ char *result = NULL;
+ Form_pg_compression cmform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmid = ((Form_pg_compression_opt) GETSTRUCT(tup))->cmid;
+ ReleaseSysCache(tup);
+
+ /* now we can get the name */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ return result;
+}
+
+/* get_compression_options */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+CompressionMethodRoutine *
+GetCompressionRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return get_compression_method_routine(cmhandler, InvalidOid);
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+get_compression_method_routine(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 05232ea505..4f71a240c6 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2768,8 +2768,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 4d77411a68..aba2087839 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -112,7 +112,8 @@ create_ctas_internal(List *attrList, IntoClause *into)
* Create the relation. (This will error out if there's an existing view,
* so we don't need more code to complain if "replace" is false.)
*/
- intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL);
+ intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL,
+ NULL);
/*
* If necessary, create a TOAST table for the target table. Note that
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 938133bbe4..f520c7fe7b 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 5e1b0fe289..eb9f72dcba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -212,7 +212,8 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq)
stmt->tablespacename = NULL;
stmt->if_not_exists = seq->if_not_exists;
- address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL);
+ address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL,
+ NULL);
seqoid = address.objectId;
Assert(seqoid != InvalidOid);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index c902293741..cbc80d282e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -459,6 +464,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression * compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -503,7 +510,8 @@ static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
*/
ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString)
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt)
{
char relname[NAMEDATALEN];
Oid namespaceId;
@@ -523,6 +531,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
static char *validnsps[] = HEAP_RELOPT_NAMESPACES;
Oid ofTypeId;
ObjectAddress address;
+ AlterTableStmt *alterStmt = NULL;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -724,6 +733,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ transformColumnCompression(colDef, stmt->relation, &alterStmt);
}
/*
@@ -921,6 +932,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
relation_close(rel, NoLock);
+ if (pAlterStmt)
+ *pAlterStmt = (Node *) alterStmt;
+ else
+ Assert(!alterStmt);
+
return address;
}
@@ -1610,6 +1626,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1944,6 +1961,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1971,6 +2001,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2180,6 +2213,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3279,6 +3319,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3770,6 +3811,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4118,6 +4165,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5338,6 +5390,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6438,6 +6492,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9358,6 +9417,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9407,7 +9468,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9429,6 +9492,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attbyval = tform->typbyval;
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ attTup->attcompression = InvalidOid;
ReleaseSysCache(typeTuple);
@@ -12481,6 +12545,86 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression * compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName || OidIsValid(compression->methodOid))
+ {
+ /* SET COMPRESSED */
+ Oid cmid,
+ cmoptoid;
+
+ cmid = compression->methodName
+ ? get_compression_method_oid(compression->methodName, false)
+ : compression->methodOid;
+
+ cmoptoid = CreateCompressionOptions(atttableform, cmid, compression->options);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index 7df942b18b..462ed577ae 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -2182,7 +2183,7 @@ DefineCompositeType(RangeVar *typevar, List *coldeflist)
* Finally create the relation. This also creates the type.
*/
DefineRelation(createStmt, RELKIND_COMPOSITE_TYPE, InvalidOid, &address,
- NULL);
+ NULL, NULL);
return address;
}
diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c
index 076e2a3a40..9af0f11856 100644
--- a/src/backend/commands/view.c
+++ b/src/backend/commands/view.c
@@ -251,7 +251,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace,
* false).
*/
address = DefineRelation(createStmt, RELKIND_VIEW, InvalidOid, NULL,
- NULL);
+ NULL, NULL);
Assert(address.objectId != InvalidOid);
/* Make the new view relation visible */
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 5dfc49deb9..3a80e997a9 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -302,6 +302,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
var->vartypmod != -1))
return false; /* type mismatch */
+ if (OidIsValid(att_tup->attcompression))
+ return false;
+
tlist_item = lnext(tlist_item);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e819188acc..417aaabe66 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2804,6 +2804,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2822,6 +2823,19 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression * from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_SCALAR_FIELD(methodOid);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5469,6 +5483,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ccdcfa2d60..c9ee23d843 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2544,6 +2544,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2562,6 +2563,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression * a, const ColumnCompression * b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_SCALAR_FIELD(methodOid);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3613,6 +3624,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 8e6f27e153..d22c2d8fbd 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 43d62062bc..a182978df8 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2796,6 +2796,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2812,6 +2813,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression * node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_OID_FIELD(methodOid);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4101,6 +4112,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4c83a63f7d..02011e3cf4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -397,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -581,6 +582,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <node> partbound_datum PartitionRangeDatum
%type <list> partbound_datum_list range_datum_list
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -613,9 +616,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2167,6 +2170,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3260,11 +3272,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3274,8 +3287,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3322,6 +3335,39 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->methodOid = InvalidOid;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->methodOid = InvalidOid;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5682,6 +5728,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6190,6 +6245,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6253,7 +6309,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6443,6 +6499,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14632,6 +14689,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index decf4e3830..1a6621b438 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -494,6 +495,33 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -794,6 +822,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1003,6 +1041,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 82a707af7b..331d133660 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -998,6 +998,7 @@ ProcessUtilitySlow(ParseState *pstate,
foreach(l, stmts)
{
Node *stmt = (Node *) lfirst(l);
+ Node *alterStmt = NULL;
if (IsA(stmt, CreateStmt))
{
@@ -1008,7 +1009,9 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_RELATION,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1042,7 +1045,8 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_FOREIGN_TABLE,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
CreateForeignTable((CreateForeignTableStmt *) stmt,
address.objectId);
EventTriggerCollectSimpleCommand(address,
@@ -1074,6 +1078,9 @@ ProcessUtilitySlow(ParseState *pstate,
NULL);
}
+ if (alterStmt)
+ lappend(stmts, alterStmt);
+
/* Need CCI between commands */
if (lnext(l) != NULL)
CommandCounterIncrement();
@@ -1283,6 +1290,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -1696,6 +1708,11 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_FOREIGN_TABLE:
RemoveRelations(stmt);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ if (stmt->behavior == DROP_CASCADE)
+ {
+ /* TODO decompress columns instead of their deletion */
+ }
default:
RemoveObjects(stmt);
break;
@@ -2309,6 +2326,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2432,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index b0a9217d1e..c8f9004a38 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,92 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(AttributeCompression *ac, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static void
+tsvector_configure(Form_pg_attribute attr, List *options)
+{
+ if (options != NIL)
+ elog(ERROR, "the compression method for tsvector doesn't take any options");
+}
+
+static struct varlena *
+tsvector_decompress(AttributeCompression *ac, const struct varlena *data)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = tsvector_configure;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index 48961e31aa..92004b9512 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2276,16 +2276,12 @@ getBaseType(Oid typid)
}
/*
- * getBaseTypeAndTypmod
- * If the given type is a domain, return its base type and typmod;
- * otherwise return the type's own OID, and leave *typmod unchanged.
- *
* Note that the "applied typmod" should be -1 for every domain level
* above the bottommost; therefore, if the passed-in typid is indeed
* a domain, *typmod should be -1.
*/
-Oid
-getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+static inline HeapTuple
+getBaseTypeTuple(Oid *typid, int32 *typmod)
{
/*
* We loop to find the bottom base type in a stack of domains.
@@ -2295,24 +2291,33 @@ getBaseTypeAndTypmod(Oid typid, int32 *typmod)
HeapTuple tup;
Form_pg_type typTup;
- tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*typid));
if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for type %u", typid);
+ elog(ERROR, "cache lookup failed for type %u", *typid);
typTup = (Form_pg_type) GETSTRUCT(tup);
if (typTup->typtype != TYPTYPE_DOMAIN)
- {
/* Not a domain, so done */
- ReleaseSysCache(tup);
- break;
- }
+ return tup;
Assert(*typmod == -1);
- typid = typTup->typbasetype;
+ *typid = typTup->typbasetype;
*typmod = typTup->typtypmod;
ReleaseSysCache(tup);
}
+}
+
+/*
+ * getBaseTypeAndTypmod
+ * If the given type is a domain, return its base type and typmod;
+ * otherwise return the type's own OID, and leave *typmod unchanged.
+ */
+Oid
+getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+{
+ HeapTuple tup = getBaseTypeTuple(&typid, typmod);
+ ReleaseSysCache(tup);
return typid;
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index b8e37809b0..f8849c89e7 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
#include "access/nbtree.h"
@@ -76,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -565,6 +567,7 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 888edbb325..c80e31fd9b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 4b47951de1..2d6827f1f3 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
static DumpableObject **nspinfoindex;
static DumpableObject **extinfoindex;
+static DumpableObject **cminfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
@@ -61,6 +62,7 @@ static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
+static int numCompressionMethods;
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
@@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+ CompressionMethodInfo *cminfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr)
write_msg(NULL, "reading subscriptions\n");
getSubscriptions(fout);
+ if (g_verbose)
+ write_msg(NULL, "reading compression methods\n");
+ cminfo = getCompressionMethods(fout, &numCompressionMethods);
+ cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo));
+
*numTablesPtr = numTables;
return tblinfo;
}
@@ -827,6 +836,17 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
+/*
+ * findCompressionMethodByOid
+ * finds the entry (in cminfo) of the compression method with the given oid
+ * returns NULL if not found
+ */
+CompressionMethodInfo *
+findCompressionMethodByOid(Oid oid)
+{
+ return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex,
+ numCompressionMethods);
+}
/*
* setExtensionMembership
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..6b962a0ba0 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -149,6 +150,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 6d4c28852c..03a80c5fe5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -361,6 +361,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -3957,6 +3958,99 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getCompressionMethods
+ * get information about compression methods
+ */
+CompressionMethodInfo *
+getCompressionMethods(Archive *fout, int *numMethods)
+{
+ DumpOptions *dopt = fout->dopt;
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionMethodInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i,
+ ntups;
+
+ if (dopt->no_compression_methods || fout->remoteVersion < 110000)
+ return NULL;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ /* Get the compression methods in current database. */
+ appendPQExpBuffer(query,
+ "SELECT c.tableoid, c.oid, c.cmname, c.cmhandler "
+ "FROM pg_catalog.pg_compression c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "oid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_METHOD;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+
+ /* Decide whether we want to dump it */
+ selectDumpableObject(&(cminfo[i].dobj), fout);
+ }
+ if (numMethods)
+ *numMethods = ntups;
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+
+ return cminfo;
+}
+
+/*
+ * dumpCompressionMethod
+ * dump the definition of the given compression method
+ */
+static void
+dumpCompressionMethod(Archive *fout, CompressionMethodInfo *cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER",
+ fmtId(cminfo->dobj.name));
+ appendPQExpBuffer(query, " %s;\n", fmtId(cminfo->cmhandler));
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION METHOD", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -4484,7 +4578,47 @@ getTypes(Archive *fout, int *numTypes)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- if (fout->remoteVersion >= 90600)
+ if (fout->remoteVersion >= 110000)
+ {
+ PQExpBuffer acl_subquery = createPQExpBuffer();
+ PQExpBuffer racl_subquery = createPQExpBuffer();
+ PQExpBuffer initacl_subquery = createPQExpBuffer();
+ PQExpBuffer initracl_subquery = createPQExpBuffer();
+
+ buildACLQueries(acl_subquery, racl_subquery, initacl_subquery,
+ initracl_subquery, "t.typacl", "t.typowner", "'T'",
+ dopt->binary_upgrade);
+
+ appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
+ "t.typnamespace, "
+ "%s AS typacl, "
+ "%s AS rtypacl, "
+ "%s AS inittypacl, "
+ "%s AS initrtypacl, "
+ "(%s t.typowner) AS rolname, "
+ "t.typelem, t.typrelid, "
+ "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" "
+ "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, "
+ "t.typtype, t.typisdefined, "
+ "t.typname[0] = '_' AND t.typelem != 0 AND "
+ "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray "
+ "FROM pg_type t "
+ "LEFT JOIN pg_init_privs pip ON "
+ "(t.oid = pip.objoid "
+ "AND pip.classoid = 'pg_type'::regclass "
+ "AND pip.objsubid = 0) ",
+ acl_subquery->data,
+ racl_subquery->data,
+ initacl_subquery->data,
+ initracl_subquery->data,
+ username_subquery);
+
+ destroyPQExpBuffer(acl_subquery);
+ destroyPQExpBuffer(racl_subquery);
+ destroyPQExpBuffer(initacl_subquery);
+ destroyPQExpBuffer(initracl_subquery);
+ }
+ else if (fout->remoteVersion >= 90600)
{
PQExpBuffer acl_subquery = createPQExpBuffer();
PQExpBuffer racl_subquery = createPQExpBuffer();
@@ -7895,6 +8029,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7930,7 +8066,48 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.cmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "cm.cmname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
+ "LEFT JOIN pg_catalog.pg_compression cm "
+ "ON c.cmid = cm.oid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7949,9 +8126,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2 "
"ORDER BY a.attnum",
@@ -7975,7 +8156,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7999,7 +8182,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8017,7 +8202,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8034,7 +8221,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8064,6 +8253,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8080,6 +8271,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8107,6 +8300,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9596,6 +9791,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_COMPRESSION_METHOD:
+ dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -15513,6 +15711,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSED %s",
+ fmtId(tbinfo->attcmnames[j]));
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH %s",
+ fmtId(tbinfo->attcmoptions[j]));
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17778,6 +17985,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_COMPRESSION_METHOD:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e7593e6da7..cd12f8055a 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_COMPRESSION_METHOD
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -315,6 +316,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +614,13 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+/* The CompressionMethodInfo struct is used to represent compression method */
+typedef struct _CompressionMethodInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+} CompressionMethodInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -654,6 +664,7 @@ extern OprInfo *findOprByOid(Oid oid);
extern CollInfo *findCollationByOid(Oid oid);
extern NamespaceInfo *findNamespaceByOid(Oid oid);
extern ExtensionInfo *findExtensionByOid(Oid oid);
+extern CompressionMethodInfo *findCompressionMethodByOid(Oid oid);
extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems);
extern ExtensionInfo *findOwningExtension(CatalogId catalogId);
@@ -711,5 +722,7 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern CompressionMethodInfo *getCompressionMethods(Archive *fout,
+ int *numMethods);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5044a76787..7195f54cdc 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -40,47 +40,48 @@ static const int dbObjectTypePriority[] =
{
1, /* DO_NAMESPACE */
4, /* DO_EXTENSION */
- 5, /* DO_TYPE */
- 5, /* DO_SHELL_TYPE */
- 6, /* DO_FUNC */
- 7, /* DO_AGG */
- 8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
- 9, /* DO_OPCLASS */
- 9, /* DO_OPFAMILY */
- 3, /* DO_COLLATION */
- 11, /* DO_CONVERSION */
- 18, /* DO_TABLE */
- 20, /* DO_ATTRDEF */
- 28, /* DO_INDEX */
- 29, /* DO_STATSEXT */
- 30, /* DO_RULE */
- 31, /* DO_TRIGGER */
- 27, /* DO_CONSTRAINT */
- 32, /* DO_FK_CONSTRAINT */
- 2, /* DO_PROCLANG */
- 10, /* DO_CAST */
- 23, /* DO_TABLE_DATA */
- 24, /* DO_SEQUENCE_SET */
- 19, /* DO_DUMMY_TYPE */
- 12, /* DO_TSPARSER */
- 14, /* DO_TSDICT */
- 13, /* DO_TSTEMPLATE */
- 15, /* DO_TSCONFIG */
- 16, /* DO_FDW */
- 17, /* DO_FOREIGN_SERVER */
- 32, /* DO_DEFAULT_ACL */
- 3, /* DO_TRANSFORM */
- 21, /* DO_BLOB */
- 25, /* DO_BLOB_DATA */
- 22, /* DO_PRE_DATA_BOUNDARY */
- 26, /* DO_POST_DATA_BOUNDARY */
- 33, /* DO_EVENT_TRIGGER */
- 38, /* DO_REFRESH_MATVIEW */
- 34, /* DO_POLICY */
- 35, /* DO_PUBLICATION */
- 36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 6, /* DO_TYPE */
+ 6, /* DO_SHELL_TYPE */
+ 7, /* DO_FUNC */
+ 8, /* DO_AGG */
+ 9, /* DO_OPERATOR */
+ 9, /* DO_ACCESS_METHOD */
+ 10, /* DO_OPCLASS */
+ 10, /* DO_OPFAMILY */
+ 4, /* DO_COLLATION */
+ 12, /* DO_CONVERSION */
+ 19, /* DO_TABLE */
+ 21, /* DO_ATTRDEF */
+ 29, /* DO_INDEX */
+ 30, /* DO_STATSEXT */
+ 31, /* DO_RULE */
+ 32, /* DO_TRIGGER */
+ 28, /* DO_CONSTRAINT */
+ 33, /* DO_FK_CONSTRAINT */
+ 3, /* DO_PROCLANG */
+ 11, /* DO_CAST */
+ 24, /* DO_TABLE_DATA */
+ 25, /* DO_SEQUENCE_SET */
+ 20, /* DO_DUMMY_TYPE */
+ 13, /* DO_TSPARSER */
+ 15, /* DO_TSDICT */
+ 14, /* DO_TSTEMPLATE */
+ 16, /* DO_TSCONFIG */
+ 17, /* DO_FDW */
+ 18, /* DO_FOREIGN_SERVER */
+ 33, /* DO_DEFAULT_ACL */
+ 4, /* DO_TRANSFORM */
+ 22, /* DO_BLOB */
+ 26, /* DO_BLOB_DATA */
+ 23, /* DO_PRE_DATA_BOUNDARY */
+ 27, /* DO_POST_DATA_BOUNDARY */
+ 34, /* DO_EVENT_TRIGGER */
+ 39, /* DO_REFRESH_MATVIEW */
+ 35, /* DO_POLICY */
+ 36, /* DO_PUBLICATION */
+ 37, /* DO_PUBLICATION_REL */
+ 38, /* DO_SUBSCRIPTION */
+ 5 /* DO_COMPRESSION_METHOD */
};
static DumpId preDataBoundId;
@@ -1436,6 +1437,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_COMPRESSION_METHOD:
+ snprintf(buf, bufsize,
+ "COMPRESSION METHOD %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..6b72ccf9ea 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 041b5e0c87..0ded023858 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index b7b978a361..eb6d29e4e9 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1716,6 +1779,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ if (pset.sversion >= 100000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT cm.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " JOIN pg_catalog.pg_compression cm ON (cm.oid = c.cmid) "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1830,6 +1909,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1925,6 +2008,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1932,7 +2020,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1943,7 +2031,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a926c40b9b..25d2fbc125 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a09c49d6cf..1327abdfef 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..1320d8f882
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ CompressionMethodRoutine *routine;
+ List *options;
+ Oid cmoptoid;
+} AttributeCompression;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (AttributeCompression * ac, const struct varlena *data);
+
+/*
+ * API struct for an compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionConfigureRoutine drop;
+ CompressionRoutine compress;
+ CompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine * GetCompressionRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr, Oid cmid,
+ List *options);
+extern ColumnCompression * GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression * c1,
+ ColumnCompression * c2, const char *attributeName);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 5cdaa3bff1..573512367a 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, char *name, char *desc,
extern Datum transformRelOptions(Datum oldOptions, List *defList,
char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,9 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern void freeRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1d3e..867adc1bd8 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -88,6 +90,9 @@ typedef struct tupleDesc
/* Accessor for the i'th attribute of tupdesc. */
#define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)])
+#define TupleDescAttrCompression(tupdesc, i) \
+ ((tupdesc)->tdcompression? &((tupdesc)->tdcompression[i]) : NULL)
+
extern TupleDesc CreateTemplateTupleDesc(int natts, bool hasoid);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..11092e5e9d 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -210,7 +210,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..b580f1971a 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(oid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..40f5cc4f18 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..343d3355d9
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,54 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420)
+{
+ Oid cmid; /* compression method oid */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 3
+#define Anum_pg_compression_opt_cmid 1
+#define Anum_pg_compression_opt_cmhandler 2
+#define Anum_pg_compression_opt_cmoptions 3
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 93c031aad7..f30dd7fbcb 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3881,6 +3881,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4677,6 +4681,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index ffdb452b02..be68bdfbc3 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index f7bb4a54f7..01c542e829 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index da3ff5dbee..c50d9525d0 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -24,7 +24,8 @@
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString);
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt);
extern void RemoveRelations(DropStmt *drop);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ffeeb4919b..6dc49a73b6 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -468,6 +468,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -498,7 +499,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60a6cc4b26..083416d155 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,14 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ Oid methodOid;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1616,6 +1625,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1763,7 +1773,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..f5c879ae60 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -146,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -282,7 +291,12 @@ typedef struct
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +325,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 254a811aff..6ad889af7a 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -210,6 +210,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a0be41929..ff7cb530fd 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..693e5a5e8c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,108 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table cmtest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method for tsvector doesn't take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+NOTICE: merging column "fts" with inherited definition
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 4 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 60ab28a96a..093d6074a7 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -547,10 +547,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -669,11 +669,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -696,11 +696,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -725,46 +725,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..2ac75c44b5 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 331f7a911f..530ce1b1d0 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index c698faff2f..a147b8217f 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index b715619313..bbcc5e8680 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -389,10 +389,10 @@ drop table range_parted, list_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -704,74 +704,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index e996640593..62b06da011 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index cef70b1a1e..a652883172 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index aa5e6af621..a44cf1c910 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 3866314a92..5c72cb16f5 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..83307597e7
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,37 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+
+SELECT length(fts) FROM cmtest2;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
diff --git a/src/tools/pgindent/exclude_file_patterns b/src/tools/pgindent/exclude_file_patterns
index cb2f902a90..95b9f0160f 100644
--- a/src/tools/pgindent/exclude_file_patterns
+++ b/src/tools/pgindent/exclude_file_patterns
@@ -5,3 +5,4 @@
/ecpg/test/expected/
/snowball/libstemmer/
/pl/plperl/ppport\.h$
+/tmp_install.*$
Re: Custom compression methods
On 2 November 2017 at 17:41, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:
In this patch compression methods is suitable for MAIN and EXTENDED
storages like in current implementation in postgres. Just instead only
of LZ4 you can specify any other compression method.
We've had this discussion before.
Please read the "pluggable compression support" thread. See you in a
few days ;) sorry, it's kinda long.
/messages/by-id/20130621000900.GA12425@alap2.anarazel.de
IIRC there were some concerns about what happened with pg_upgrade,
with consuming precious toast bits, and a few other things.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
On Thu, Nov 2, 2017 at 6:02 PM, Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 November 2017 at 17:41, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:In this patch compression methods is suitable for MAIN and EXTENDED
storages like in current implementation in postgres. Just instead only
of LZ4 you can specify any other compression method.We've had this discussion before.
Please read the "pluggable compression support" thread. See you in a
few days ;) sorry, it's kinda long.
the proposed patch provides "pluggable" compression and let's user
decide by their own which algorithm to use.
The postgres core doesn't responsible for any patent problem.
IIRC there were some concerns about what happened with pg_upgrade,
with consuming precious toast bits, and a few other things.
yes, pg_upgrade may be a problem.
--
Craig Ringer http://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Training & Services--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
On Sun, Nov 5, 2017 at 2:22 PM, Oleg Bartunov <obartunov@gmail.com> wrote:
IIRC there were some concerns about what happened with pg_upgrade,
with consuming precious toast bits, and a few other things.yes, pg_upgrade may be a problem.
A basic problem here is that, as proposed, DROP COMPRESSION METHOD may
break your database irretrievably. If there's no data compressed
using the compression method you dropped, everything is cool -
otherwise everything is broken and there's no way to recover. The
only obvious alternative is to disallow DROP altogether (or make it
not really DROP).
Both of those alternatives sound fairly unpleasant to me, but I'm not
exactly sure what to recommend in terms of how to make it better.
Ideally anything we expose as an SQL command should have a DROP
command that undoes whatever CREATE did and leaves the database in an
intact state, but that seems hard to achieve in this case.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
If there's no data compressed
using the compression method you dropped, everything is cool -
otherwise everything is broken and there's no way to recover.
The only obvious alternative is to disallow DROP altogether (or make it
not really DROP).
Wouldn't whatever was using the compression method have something
marking which method was used? If so, couldn't we just scan if there is
any data using it, and if so disallow the drop, or possibly an option to allow
the drop and rewrite the table either uncompressed, or with the default
compression method?
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
Robert Haas <robertmhaas@gmail.com> writes:
A basic problem here is that, as proposed, DROP COMPRESSION METHOD may
break your database irretrievably. If there's no data compressed
using the compression method you dropped, everything is cool -
otherwise everything is broken and there's no way to recover. The
only obvious alternative is to disallow DROP altogether (or make it
not really DROP).
Both of those alternatives sound fairly unpleasant to me, but I'm not
exactly sure what to recommend in terms of how to make it better.
Ideally anything we expose as an SQL command should have a DROP
command that undoes whatever CREATE did and leaves the database in an
intact state, but that seems hard to achieve in this case.
If the use of a compression method is tied to specific data types and/or
columns, then each of those could have a dependency on the compression
method, forcing a type or column drop if you did DROP COMPRESSION METHOD.
That would leave no reachable data using the removed compression method.
So that part doesn't seem unworkable on its face.
IIRC, the bigger concerns in the last discussion had to do with
replication, ie, can downstream servers make sense of the data.
Maybe that's not any worse than the issues you get with non-core
index AMs, but I'm not sure.
regards, tom lane
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: Custom compression methods
On Thu, 2 Nov 2017 23:02:34 +0800
Craig Ringer <craig@2ndquadrant.com> wrote:
On 2 November 2017 at 17:41, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:In this patch compression methods is suitable for MAIN and EXTENDED
storages like in current implementation in postgres. Just instead
only of LZ4 you can specify any other compression method.We've had this discussion before.
Please read the "pluggable compression support" thread. See you in a
few days ;) sorry, it's kinda long./messages/by-id/20130621000900.GA12425@alap2.anarazel.de
IIRC there were some concerns about what happened with pg_upgrade,
with consuming precious toast bits, and a few other things.
Thank you for the link, I didn't see that thread when I looked over
mailing lists. I read it briefly, and I can address few things
relating to my patch.
Most concerns have been related with legal issues.
Actually that was the reason I did not include any new compression
algorithms to my patch. Unlike that patch mine only provides syntax
and is just a way to give the users use their own compression algorithms
and deal with any legal issues themselves.
I use only one unused bit in header (there's still one free ;), that's
enough to determine that data is compressed or not.
I did found out that pg_upgrade doesn't work properly with my patch,
soon I will send fix for it.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
--
Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-hackers
Re: [HACKERS] Custom compression methods
On Thu, 2 Nov 2017 15:28:36 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
On Tue, 12 Sep 2017 17:55:05 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:Attached rebased version of the patch. Added support of pg_dump, the
code was simplified, and a separate cache for compression options
was added.Attached version 3 of the patch. Rebased to the current master,
removed ALTER TYPE .. SET COMPRESSED syntax, fixed bug in compression
options cache.
Attached version 4 of the patch. Fixed pg_upgrade and few other bugs.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v4.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 01acc2ef9d..f43f09cc19 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -59,6 +59,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 3b19ea7131..3dc8eb2952 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,8 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET NOT COMPRESSED
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +322,34 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. Setting a compression method doesn't change anything in the
+ table and affects only future table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>SET NOT COMPRESSED</literal>
+ </term>
+ <listitem>
+ <para>
+ This form removes compression from a column. Removing compresssion from
+ a column doesn't change already compressed tuples and affects only future
+ table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..663010ecd9
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,50 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="SQL-CREATECOMPRESSIONMETHOD">
+ <indexterm zone="sql-createcompressionmethod">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">compression_method_name</replaceable>
+ HANDLER <replaceable class="parameter">compression_method_handler</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method
+ with <replaceable class="parameter">compression_method_name</replaceable>.
+ </para>
+
+ <para>
+ A compression method links a name with a compression handler. And the
+ handler is a special function that returns collection of methods that
+ can be used for compression.
+ </para>
+
+ <para>
+ After a compression method is created, you can specify it in
+ <xref linkend="SQL-CREATETABLE"> or <xref linkend="SQL-ALTERTABLE">
+ statements.
+ </para>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index bbb3a51def..70de6352a5 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index 9000b3aaaa..cc0bd70be3 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -87,6 +87,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..629f3575a9 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -92,7 +92,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 3d0ce9af6f..e96b77a629 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -935,11 +935,48 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
+void
+freeRelOptions(List *options)
+{
+ ListCell *cell;
+
+ Assert(options != NIL);
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ pfree(def->defname);
+ pfree(defGetString(def));
+ pfree(def->arg);
+ }
+ list_free_deep(options);
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca73a8..d206cce18e 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -242,6 +244,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -396,6 +399,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -458,6 +463,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -563,6 +569,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -675,7 +682,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 5a8f1dab83..a255fee3fb 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -39,6 +41,8 @@
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
#include "utils/typcache.h"
@@ -53,19 +57,46 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+static HTAB *compression_options_htab = NULL;
+static MemoryContext compression_options_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ ((toast_compress_header *) (ptr))->info &= 0xC0000000; \
+ ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +114,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_compression_options_htab(void);
+static AttributeCompression * get_compression_options_info(Oid cmoptoid);
/* ----------
@@ -741,6 +774,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
@@ -770,10 +805,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +950,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1229,7 +1266,9 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
if (VARATT_IS_EXTERNAL(new_value) ||
VARATT_IS_COMPRESSED(new_value))
{
- new_value = heap_tuple_untoast_attr(new_value);
+ struct varlena *untoasted_value = heap_tuple_untoast_attr(new_value);
+
+ new_value = untoasted_value;
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
}
@@ -1353,7 +1392,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,25 +1406,43 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoptoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize,
+ len = 0;
+ AttributeCompression *ac = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
+ if (OidIsValid(cmoptoid))
+ ac = get_compression_options_info(cmoptoid);
+
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ if (!ac && (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size))
return PointerGetDatum(NULL);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ if (ac)
+ {
+ tmp = ac->routine->compress(ac, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
* We recheck the actual size even if pglz_compress() reports success,
@@ -1398,11 +1454,7 @@ toast_compress_datum(Datum value)
* only one header byte and no padding if the value is short enough. So
* we insist on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!ac && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,10 +1462,20 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (ac && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, ac->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
- pfree(tmp);
+ if (tmp)
+ pfree(tmp);
+
return PointerGetDatum(NULL);
}
}
@@ -2280,15 +2342,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ AttributeCompression *ac;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ ac = get_compression_options_info(hdr->cmoptoid);
+ result = ac->routine->decompress(ac, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2463,44 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+static void
+init_compression_options_htab(void)
+{
+ HASHCTL ctl;
+
+ compression_options_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "compression options cache context",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(AttributeCompression);
+ ctl.hcxt = compression_options_mcxt;
+ compression_options_htab = hash_create("compression options cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+}
+
+static AttributeCompression *
+get_compression_options_info(Oid cmoptoid)
+{
+ bool found;
+ AttributeCompression *result;
+
+ Assert(OidIsValid(cmoptoid));
+ if (!compression_options_htab)
+ init_compression_options_htab();
+
+ result = hash_search(compression_options_htab, &cmoptoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ Assert(compression_options_mcxt);
+ oldcxt = MemoryContextSwitchTo(compression_options_mcxt);
+ result->cmoptoid = cmoptoid;
+ result->routine = GetCompressionRoutine(cmoptoid);
+ result->options = GetCompressionOptionsList(cmoptoid);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ return result;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 8287de97a2..be6b460aee 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 033c4358ea..e1bfc7c6bf 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2512,6 +2524,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 256a9c9c93..c5838fa779 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -451,6 +451,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9e14880b99..0a7ee5961e 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -707,6 +712,13 @@ AddNewAttributeTuples(Oid new_rel_oid,
referenced.objectSubId = 0;
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+
+ if (OidIsValid(attr->attcompression))
+ {
+ ObjectAddressSet(referenced, CompressionOptRelationId,
+ attr->attcompression);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
}
/*
@@ -1453,6 +1465,22 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+static void
+DropAttributeCompression(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ bool attisdropped = att->attisdropped;
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ att->attisdropped = true;
+ cmr->drop(att, options);
+ att->attisdropped = attisdropped;
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1511,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ DropAttributeCompression(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1611,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ DropAttributeCompression(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1634,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c7b2f031f0..11dc107ab7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8d55c76fc4..9fd1cb763d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index e02d312008..531a820464 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..ad51ee759f
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,479 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/compression.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine * get_compression_method_routine(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+LookupCompressionHandlerFunc(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+static ObjectAddress
+CreateCompressionMethod(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = LookupCompressionHandlerFunc(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ return CreateCompressionMethod(cmName, (List *) handlerEl->arg);
+}
+
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, List *options)
+{
+ Relation rel;
+ HeapTuple tup,
+ newtup;
+ Oid cmoptoid;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+
+ ObjectAddress myself,
+ ref1,
+ ref2,
+ ref3;
+
+ CompressionMethodRoutine *routine;
+ Form_pg_compression cmform;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ /* Get handler function OID for the compression method */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ routine = get_compression_method_routine(cmform->cmhandler, attr->atttypid);
+
+ if (routine->configure && options != NIL)
+ routine->configure(attr, options);
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ cmoptoid = GetNewOidWithIndex(rel, CompressionOptionsOidIndexId,
+ Anum_pg_compression_opt_cmoptoid);
+ values[Anum_pg_compression_opt_cmoptoid - 1] = ObjectIdGetDatum(cmoptoid);
+ values[Anum_pg_compression_opt_cmname - 1] = NameGetDatum(&cmform->cmname);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmform->cmhandler);
+
+ if (options)
+ values[Anum_pg_compression_opt_cmoptions - 1] = optionListToArray(options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+
+ ReleaseSysCache(tup);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, cmoptoid);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmform->cmhandler);
+ ObjectAddressSubSet(ref2, RelationRelationId, attr->attrelid, attr->attnum);
+ ObjectAddressSet(ref3, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&ref2, &myself, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref3, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return cmoptoid;
+}
+
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+ ReleaseSysCache(tup);
+ heap_close(relation, RowExclusiveLock);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmoptform;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = pstrdup(NameStr(cmoptform->cmname));
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression * c1, ColumnCompression * c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->methodName, c2->methodName))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->methodName, c2->methodName)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+ Form_pg_compression_opt cmoptform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmoptform->cmname));
+ ReleaseSysCache(tup);
+
+ return result;
+}
+
+/* get_compression_options */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+CompressionMethodRoutine *
+GetCompressionRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return get_compression_method_routine(cmhandler, InvalidOid);
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+get_compression_method_routine(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 99ad7a9ccf..8ed9c7f454 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2780,8 +2780,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 4d77411a68..aba2087839 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -112,7 +112,8 @@ create_ctas_internal(List *attrList, IntoClause *into)
* Create the relation. (This will error out if there's an existing view,
* so we don't need more code to complain if "replace" is false.)
*/
- intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL);
+ intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL,
+ NULL);
/*
* If necessary, create a TOAST table for the target table. Note that
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index fa7d0d015a..a2eee78a64 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 5e1b0fe289..eb9f72dcba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -212,7 +212,8 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq)
stmt->tablespacename = NULL;
stmt->if_not_exists = seq->if_not_exists;
- address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL);
+ address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL,
+ NULL);
seqoid = address.objectId;
Assert(seqoid != InvalidOid);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9c66aa75ed..67cb187180 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -459,6 +464,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression * compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -503,7 +510,8 @@ static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
*/
ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString)
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt)
{
char relname[NAMEDATALEN];
Oid namespaceId;
@@ -523,6 +531,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
static char *validnsps[] = HEAP_RELOPT_NAMESPACES;
Oid ofTypeId;
ObjectAddress address;
+ AlterTableStmt *alterStmt = NULL;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -724,6 +733,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ transformColumnCompression(colDef, stmt->relation, &alterStmt);
}
/*
@@ -921,6 +932,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
relation_close(rel, NoLock);
+ if (pAlterStmt)
+ *pAlterStmt = (Node *) alterStmt;
+ else
+ Assert(!alterStmt);
+
return address;
}
@@ -1610,6 +1626,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1944,6 +1961,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1971,6 +2001,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2180,6 +2213,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3279,6 +3319,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3770,6 +3811,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4118,6 +4165,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5338,6 +5390,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6438,6 +6492,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9358,6 +9417,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9407,7 +9468,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9429,6 +9492,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attbyval = tform->typbyval;
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ attTup->attcompression = InvalidOid;
ReleaseSysCache(typeTuple);
@@ -12481,6 +12545,83 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression * compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName)
+ {
+ /* SET COMPRESSED */
+ Oid cmid,
+ cmoptoid;
+
+ cmid = get_compression_method_oid(compression->methodName, false);
+ cmoptoid = CreateCompressionOptions(atttableform, cmid, compression->options);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c054..be4a6c5f38 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -2182,7 +2183,7 @@ DefineCompositeType(RangeVar *typevar, List *coldeflist)
* Finally create the relation. This also creates the type.
*/
DefineRelation(createStmt, RELKIND_COMPOSITE_TYPE, InvalidOid, &address,
- NULL);
+ NULL, NULL);
return address;
}
diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c
index c1e80e61d4..cc0182075e 100644
--- a/src/backend/commands/view.c
+++ b/src/backend/commands/view.c
@@ -251,7 +251,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace,
* false).
*/
address = DefineRelation(createStmt, RELKIND_VIEW, InvalidOid, NULL,
- NULL);
+ NULL, NULL);
Assert(address.objectId != InvalidOid);
/* Make the new view relation visible */
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 5dfc49deb9..3a80e997a9 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -302,6 +302,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
var->vartypmod != -1))
return false; /* type mismatch */
+ if (OidIsValid(att_tup->attcompression))
+ return false;
+
tlist_item = lnext(tlist_item);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index b8bd4004e1..95cdb8029b 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2804,6 +2804,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2822,6 +2823,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression * from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5471,6 +5484,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2a83da9aa9..d618cc637e 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2544,6 +2544,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2562,6 +2563,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression * a, const ColumnCompression * b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3615,6 +3625,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index c2a93b2d4c..81e334a1d3 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index dc35df9e4f..6bfd7ed0ee 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2796,6 +2796,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2812,6 +2813,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression * node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4103,6 +4113,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c301ca465d..122d3de4d6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -397,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +583,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +617,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2168,6 +2171,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3332,11 +3344,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3346,8 +3359,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3394,6 +3407,37 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5754,6 +5798,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6262,6 +6315,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6325,7 +6379,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6515,6 +6569,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14704,6 +14759,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 8461da490a..167f70965a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -494,6 +495,33 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -794,6 +822,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1003,6 +1041,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 82a707af7b..331d133660 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -998,6 +998,7 @@ ProcessUtilitySlow(ParseState *pstate,
foreach(l, stmts)
{
Node *stmt = (Node *) lfirst(l);
+ Node *alterStmt = NULL;
if (IsA(stmt, CreateStmt))
{
@@ -1008,7 +1009,9 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_RELATION,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1042,7 +1045,8 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_FOREIGN_TABLE,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
CreateForeignTable((CreateForeignTableStmt *) stmt,
address.objectId);
EventTriggerCollectSimpleCommand(address,
@@ -1074,6 +1078,9 @@ ProcessUtilitySlow(ParseState *pstate,
NULL);
}
+ if (alterStmt)
+ lappend(stmts, alterStmt);
+
/* Need CCI between commands */
if (lnext(l) != NULL)
CommandCounterIncrement();
@@ -1283,6 +1290,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -1696,6 +1708,11 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_FOREIGN_TABLE:
RemoveRelations(stmt);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ if (stmt->behavior == DROP_CASCADE)
+ {
+ /* TODO decompress columns instead of their deletion */
+ }
default:
RemoveObjects(stmt);
break;
@@ -2309,6 +2326,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2432,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index b0a9217d1e..b6d42399c6 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,92 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(AttributeCompression * ac, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static void
+tsvector_configure(Form_pg_attribute attr, List *options)
+{
+ if (options != NIL)
+ elog(ERROR, "the compression method for tsvector doesn't take any options");
+}
+
+static struct varlena *
+tsvector_decompress(AttributeCompression * ac, const struct varlena *data)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = tsvector_configure;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/lsyscache.c b/src/backend/utils/cache/lsyscache.c
index 0ea2f2bc54..894e3148b9 100644
--- a/src/backend/utils/cache/lsyscache.c
+++ b/src/backend/utils/cache/lsyscache.c
@@ -2276,16 +2276,12 @@ getBaseType(Oid typid)
}
/*
- * getBaseTypeAndTypmod
- * If the given type is a domain, return its base type and typmod;
- * otherwise return the type's own OID, and leave *typmod unchanged.
- *
* Note that the "applied typmod" should be -1 for every domain level
* above the bottommost; therefore, if the passed-in typid is indeed
* a domain, *typmod should be -1.
*/
-Oid
-getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+static inline HeapTuple
+getBaseTypeTuple(Oid *typid, int32 *typmod)
{
/*
* We loop to find the bottom base type in a stack of domains.
@@ -2295,24 +2291,33 @@ getBaseTypeAndTypmod(Oid typid, int32 *typmod)
HeapTuple tup;
Form_pg_type typTup;
- tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(typid));
+ tup = SearchSysCache1(TYPEOID, ObjectIdGetDatum(*typid));
if (!HeapTupleIsValid(tup))
- elog(ERROR, "cache lookup failed for type %u", typid);
+ elog(ERROR, "cache lookup failed for type %u", *typid);
typTup = (Form_pg_type) GETSTRUCT(tup);
if (typTup->typtype != TYPTYPE_DOMAIN)
- {
/* Not a domain, so done */
- ReleaseSysCache(tup);
- break;
- }
+ return tup;
Assert(*typmod == -1);
- typid = typTup->typbasetype;
+ *typid = typTup->typbasetype;
*typmod = typTup->typtypmod;
ReleaseSysCache(tup);
}
+}
+
+/*
+ * getBaseTypeAndTypmod
+ * If the given type is a domain, return its base type and typmod;
+ * otherwise return the type's own OID, and leave *typmod unchanged.
+ */
+Oid
+getBaseTypeAndTypmod(Oid typid, int32 *typmod)
+{
+ HeapTuple tup = getBaseTypeTuple(&typid, typmod);
+ ReleaseSysCache(tup);
return typid;
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1908420d82..61f7ccef61 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -561,6 +563,7 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 888edbb325..c689f60e47 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ Anum_pg_compression_opt_cmoptoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 4b47951de1..7ba6d31251 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
static DumpableObject **nspinfoindex;
static DumpableObject **extinfoindex;
+static DumpableObject **cminfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
@@ -61,6 +62,7 @@ static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
+static int numCompressionMethods;
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
@@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+ CompressionMethodInfo *cminfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr)
write_msg(NULL, "reading subscriptions\n");
getSubscriptions(fout);
+ if (g_verbose)
+ write_msg(NULL, "reading compression methods\n");
+ cminfo = getCompressionMethods(fout, &numCompressionMethods);
+ cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo));
+
*numTablesPtr = numTables;
return tblinfo;
}
@@ -827,6 +836,17 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
+/*
+ * findCompressionMethodByOid
+ * finds the entry (in cminfo) of the compression method with the given oid
+ * returns NULL if not found
+ */
+CompressionMethodInfo *
+findCompressionMethodByOid(Oid oid)
+{
+ return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex,
+ numCompressionMethods);
+}
/*
* setExtensionMembership
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..5c1aaad48e 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,8 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +152,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +174,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index ec2fa8b9b9..77eb55eb69 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_options_data = ropt->compression_options_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d8fb356130..6df688adb8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -361,6 +361,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -581,6 +582,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_options_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -785,6 +789,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_options_data)
+ getCompressionOptions(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -3957,6 +3964,205 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getCompressionMethods
+ * get information about compression methods
+ */
+CompressionMethodInfo *
+getCompressionMethods(Archive *fout, int *numMethods)
+{
+ DumpOptions *dopt = fout->dopt;
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionMethodInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i,
+ ntups;
+
+ if (dopt->no_compression_methods || fout->remoteVersion < 110000)
+ return NULL;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ /* Get the compression methods in current database. */
+ appendPQExpBuffer(query, "SELECT c.tableoid, c.oid, c.cmname, "
+ "c.cmhandler::pg_catalog.regproc as cmhandler "
+ "FROM pg_catalog.pg_compression c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "oid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_METHOD;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+
+ /* Decide whether we want to dump it */
+ selectDumpableObject(&(cminfo[i].dobj), fout);
+ }
+ if (numMethods)
+ *numMethods = ntups;
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+
+ return cminfo;
+}
+
+/*
+ * dumpCompressionMethod
+ * dump the definition of the given compression method
+ */
+static void
+dumpCompressionMethod(Archive *fout, CompressionMethodInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER",
+ fmtId(cminfo->dobj.name));
+ appendPQExpBuffer(query, " %s;\n", cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION METHOD", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * getCompressionOptions
+ * get information about compression options.
+ * this information should be migrated at binary upgrade
+ */
+void
+getCompressionOptions(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionOptionsInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i_options;
+ int i,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ appendPQExpBuffer(query,
+ "SELECT c.tableoid, c.cmoptoid, c.cmname,"
+ " c.cmhandler::pg_catalog.regproc as cmhandler, c.cmoptions "
+ "FROM pg_catalog.pg_compression_opt c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "cmoptoid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+ i_options = PQfnumber(res, "cmoptions");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionOptionsInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_OPTIONS;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].cmoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpCompressionOptions
+ * dump the given compression options
+ */
+static void
+dumpCompressionOptions(Archive *fout, CompressionOptionsInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ if (strlen(cminfo->cmoptions) > 0)
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), '%s');\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler,
+ cminfo->cmoptions);
+ else
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), NULL);\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION OPTIONS", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -4484,7 +4690,47 @@ getTypes(Archive *fout, int *numTypes)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- if (fout->remoteVersion >= 90600)
+ if (fout->remoteVersion >= 110000)
+ {
+ PQExpBuffer acl_subquery = createPQExpBuffer();
+ PQExpBuffer racl_subquery = createPQExpBuffer();
+ PQExpBuffer initacl_subquery = createPQExpBuffer();
+ PQExpBuffer initracl_subquery = createPQExpBuffer();
+
+ buildACLQueries(acl_subquery, racl_subquery, initacl_subquery,
+ initracl_subquery, "t.typacl", "t.typowner", "'T'",
+ dopt->binary_upgrade);
+
+ appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
+ "t.typnamespace, "
+ "%s AS typacl, "
+ "%s AS rtypacl, "
+ "%s AS inittypacl, "
+ "%s AS initrtypacl, "
+ "(%s t.typowner) AS rolname, "
+ "t.typelem, t.typrelid, "
+ "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" "
+ "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, "
+ "t.typtype, t.typisdefined, "
+ "t.typname[0] = '_' AND t.typelem != 0 AND "
+ "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray "
+ "FROM pg_type t "
+ "LEFT JOIN pg_init_privs pip ON "
+ "(t.oid = pip.objoid "
+ "AND pip.classoid = 'pg_type'::regclass "
+ "AND pip.objsubid = 0) ",
+ acl_subquery->data,
+ racl_subquery->data,
+ initacl_subquery->data,
+ initracl_subquery->data,
+ username_subquery);
+
+ destroyPQExpBuffer(acl_subquery);
+ destroyPQExpBuffer(racl_subquery);
+ destroyPQExpBuffer(initacl_subquery);
+ destroyPQExpBuffer(initracl_subquery);
+ }
+ else if (fout->remoteVersion >= 90600)
{
PQExpBuffer acl_subquery = createPQExpBuffer();
PQExpBuffer racl_subquery = createPQExpBuffer();
@@ -7895,6 +8141,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7930,7 +8178,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.cmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.cmname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.cmoptoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7949,9 +8236,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2 "
"ORDER BY a.attnum",
@@ -7975,7 +8266,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7999,7 +8292,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8017,7 +8312,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8034,7 +8331,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8064,6 +8363,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8080,6 +8381,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8107,6 +8410,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9596,6 +9901,12 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_COMPRESSION_METHOD:
+ dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj);
+ break;
+ case DO_COMPRESSION_OPTIONS:
+ dumpCompressionOptions(fout, (CompressionOptionsInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -15513,6 +15824,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSED %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17778,6 +18098,8 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_COMPRESSION_METHOD:
+ case DO_COMPRESSION_OPTIONS:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da884ffd09..88b18ba15c 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,9 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_COMPRESSION_METHOD,
+ DO_COMPRESSION_OPTIONS
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -315,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +615,21 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+/* The CompressionMethodInfo struct is used to represent compression method */
+typedef struct _CompressionMethodInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+} CompressionMethodInfo;
+
+/* The CompressionOptionsInfo struct is used to represent compression options */
+typedef struct _CompressionOptionsInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+ char *cmoptions;
+} CompressionOptionsInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -654,6 +673,7 @@ extern OprInfo *findOprByOid(Oid oid);
extern CollInfo *findCollationByOid(Oid oid);
extern NamespaceInfo *findNamespaceByOid(Oid oid);
extern ExtensionInfo *findExtensionByOid(Oid oid);
+extern CompressionMethodInfo * findCompressionMethodByOid(Oid oid);
extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems);
extern ExtensionInfo *findOwningExtension(CatalogId catalogId);
@@ -711,5 +731,8 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern CompressionMethodInfo * getCompressionMethods(Archive *fout,
+ int *numMethods);
+void getCompressionOptions(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 48b6dd594c..f5db0280e2 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -80,7 +80,9 @@ static const int dbObjectTypePriority[] =
34, /* DO_POLICY */
35, /* DO_PUBLICATION */
36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 37, /* DO_SUBSCRIPTION */
+ 17, /* DO_COMPRESSION_METHOD */
+ 17 /* DO_COMPRESSION_OPTIONS */
};
static DumpId preDataBoundId;
@@ -1436,6 +1438,16 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_COMPRESSION_METHOD:
+ snprintf(buf, bufsize,
+ "COMPRESSION METHOD %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
+ case DO_COMPRESSION_OPTIONS:
+ snprintf(buf, bufsize,
+ "COMPRESSION OPTIONS %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..6b72ccf9ea 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_upgrade/info.c b/src/bin/pg_upgrade/info.c
index 0b14998f4b..4689b72e88 100644
--- a/src/bin/pg_upgrade/info.c
+++ b/src/bin/pg_upgrade/info.c
@@ -66,6 +66,7 @@ gen_db_file_maps(DbInfo *old_db, DbInfo *new_db,
RelInfo *new_rel = (new_relnum < new_db->rel_arr.nrels) ?
&new_db->rel_arr.rels[new_relnum] : NULL;
+ fprintf(stderr, "mapping: %s\n", old_rel->relname);
/* handle running off one array before the other */
if (!new_rel)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 8cc4de3878..8b0dbfa45c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index b7b978a361..49160ded0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1716,6 +1779,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " WHERE c.cmoptoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1830,6 +1909,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1925,6 +2008,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1932,7 +2020,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1943,7 +2031,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a926c40b9b..25d2fbc125 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b3e3799c13..a323ca80c6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..1320d8f882
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ CompressionMethodRoutine *routine;
+ List *options;
+ Oid cmoptoid;
+} AttributeCompression;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (AttributeCompression * ac, const struct varlena *data);
+
+/*
+ * API struct for an compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionConfigureRoutine drop;
+ CompressionRoutine compress;
+ CompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine * GetCompressionRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr, Oid cmid,
+ List *options);
+extern ColumnCompression * GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression * c1,
+ ColumnCompression * c2, const char *attributeName);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index cd43e3a52e..ad133c9b90 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,9 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern void freeRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1d3e..867adc1bd8 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -88,6 +90,9 @@ typedef struct tupleDesc
/* Accessor for the i'th attribute of tupdesc. */
#define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)])
+#define TupleDescAttrCompression(tupdesc, i) \
+ ((tupdesc)->tdcompression? &((tupdesc)->tdcompression[i]) : NULL)
+
extern TupleDesc CreateTemplateTupleDesc(int natts, bool hasoid);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..11092e5e9d 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -210,7 +210,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..7fe9f1f059 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(cmoptoid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..40f5cc4f18 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..ddcef814a7
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420) BKI_WITHOUT_OIDS
+{
+ Oid cmoptoid; /* compression options oid */
+ NameData cmname; /* name of compression method */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 4
+#define Anum_pg_compression_opt_cmoptoid 1
+#define Anum_pg_compression_opt_cmname 2
+#define Anum_pg_compression_opt_cmhandler 3
+#define Anum_pg_compression_opt_cmoptions 4
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0330c04f16..25c2d5c8f2 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3881,6 +3881,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4677,6 +4681,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e3551440a0..ec8c3df953 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bfead9af3d..a98ecc12ce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index da3ff5dbee..c50d9525d0 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -24,7 +24,8 @@
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString);
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt);
extern void RemoveRelations(DropStmt *drop);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ffeeb4919b..6dc49a73b6 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -468,6 +468,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -498,7 +499,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 80c19b2a55..6b25ba3eb0 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,13 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +645,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1622,6 +1630,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1769,7 +1778,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..f5c879ae60 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -146,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -282,7 +291,12 @@ typedef struct
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +325,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 254a811aff..6ad889af7a 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -210,6 +210,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a0be41929..ff7cb530fd 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..693e5a5e8c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,108 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table cmtest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method for tsvector doesn't take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+NOTICE: merging column "fts" with inherited definition
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 4 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 335cd37e18..b511daf9fa 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -590,10 +590,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -729,11 +729,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -756,11 +756,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -793,46 +793,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..2ac75c44b5 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 331f7a911f..530ce1b1d0 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index c698faff2f..a147b8217f 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 9d84ba4658..bbab453247 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -435,10 +435,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -750,74 +750,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index e996640593..62b06da011 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index a4fe96112e..adbe764196 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index aa5e6af621..a44cf1c910 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 3866314a92..5c72cb16f5 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..83307597e7
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,37 @@
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+DROP COMPRESSION METHOD ts1;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) inherits (cmtest);
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+
+SELECT length(fts) FROM cmtest2;
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest3;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
diff --git a/src/tools/pgindent/exclude_file_patterns b/src/tools/pgindent/exclude_file_patterns
index cb2f902a90..95b9f0160f 100644
--- a/src/tools/pgindent/exclude_file_patterns
+++ b/src/tools/pgindent/exclude_file_patterns
@@ -5,3 +5,4 @@
/ecpg/test/expected/
/snowball/libstemmer/
/pl/plperl/ppport\.h$
+/tmp_install.*$
Re: [HACKERS] Custom compression methods
On Sun, 5 Nov 2017 17:34:23 -0500
Robert Haas <robertmhaas@gmail.com> wrote:
On Sun, Nov 5, 2017 at 2:22 PM, Oleg Bartunov <obartunov@gmail.com>
wrote:IIRC there were some concerns about what happened with pg_upgrade,
with consuming precious toast bits, and a few other things.yes, pg_upgrade may be a problem.
A basic problem here is that, as proposed, DROP COMPRESSION METHOD may
break your database irretrievably. If there's no data compressed
using the compression method you dropped, everything is cool -
otherwise everything is broken and there's no way to recover. The
only obvious alternative is to disallow DROP altogether (or make it
not really DROP).
In the patch I use separate table for compresssion options (because
each attribute can have additional options for compression). So basicly
compressed attribute linked to compression options, not the compression
method and this method can be safely dropped.
So in the next version of the patch I can just unlink the options from
compression methods and dropping compression method will not affect
already compressed tuples. They still could be decompressed.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Wed, Nov 15, 2017 at 4:09 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:
So in the next version of the patch I can just unlink the options from
compression methods and dropping compression method will not affect
already compressed tuples. They still could be decompressed.
I guess I don't understand how that can work. I mean, if somebody
removes a compression method - i.e. uninstalls the library - and you
don't have a way to make sure there are no tuples that can only be
uncompressed by that library - then you've broken the database.
Ideally, there should be a way to add a new compression method via an
extension ... and then get rid of it and all dependencies thereupon.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
Hi Ildus,
On 14.11.2017 16:23, Ildus Kurbangaliev wrote:
On Thu, 2 Nov 2017 15:28:36 +0300 Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:On Tue, 12 Sep 2017 17:55:05 +0300 Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:Attached rebased version of the patch. Added support of pg_dump,
the code was simplified, and a separate cache for compression
options was added.Attached version 3 of the patch. Rebased to the current master,
removed ALTER TYPE .. SET COMPRESSED syntax, fixed bug in
compression options cache.Attached version 4 of the patch. Fixed pg_upgrade and few other
bugs.
I've started to review your code. And even though it's fine overall I
have few questions and comments (aside from DROP COMPRESSION METHOD
discussion).
1. I'm not sure about proposed syntax for ALTER TABLE command:
ALTER TABLE t ALTER COLUMN a SET COMPRESSED <cmname> WITH
(<options>); ALTER TABLE t ALTER COLUMN a SET NOT COMPRESSED;
ISTM it is more common for Postgres to use syntax like SET/DROP for
column options (SET/DROP NOT NULL, DEFAULT etc). My suggestion would be:
ALTER TABLE t ALTER COLUMN a SET COMPRESSED USING <compression_method>
WITH (<options>);
ALTER TABLE t ALTER COLUMN a DROP COMPRESSED;
(keyword USING here is similar to "CREATE INDEX ... USING <method>" syntax)
2. The way you changed DefineRelation() implies that caller is
responsible for creation of compression options. Probably it would be
better to create them within DefineRelation().
3. Few minor issues which seem like obsolete code:
Function freeRelOptions() is defined but never used.
Function getBaseTypeTuple() has been extracted from
getBaseTypeAndTypmod() but never used separately.
In toast_flatten_tuple_to_datum() there is untoasted_value variable
which is only used for meaningless assignment.
(Should I send a patch for that kind of issues?)
--
Ildar Musin
i.musin@postgrespro.ru
Re: [HACKERS] Custom compression methods
Hi,
On 11/14/2017 02:23 PM, Ildus Kurbangaliev wrote:
...
Attached version 4 of the patch. Fixed pg_upgrade and few other bugs.
I did a review of this today, and I think there are some things that
need improvement / fixing.
Firstly, some basic comments from just eye-balling the diff, then some
bugs I discovered after writing an extension adding lz4.
1) formatRelOptions/freeRelOptions are no longer needed (I see Ildar
already pointer that out)
2) There's unnecessary whitespace (extra newlines) on a couple of
places, which is needlessly increasing the size of the patch. Small
difference, but annoying.
3) tuptoaster.c
Why do you change 'info' from int32 to uint32? Seems unnecessary.
Adding new 'att' variable in toast_insert_or_update is confusing, as
there already is 'att' in the very next loop. Technically it's correct,
but I'd bet it'll lead to some WTF?! moments later. I propose to just
use TupleDescAttr(tupleDesc,i) on the two places where it matters,
around line 808.
There are no comments for init_compression_options_htab and
get_compression_options_info, so that needs to be fixed. Moreover, the
names are confusing because what we really get is not just 'options' but
the compression routines too.
4) gen_db_file_maps probably shouldn't do the fprints, right?
5) not sure why you modify src/tools/pgindent/exclude_file_patterns
6) I'm rather confused by AttributeCompression vs. ColumnCompression. I
mean, attribute==column, right? Of course, one is for data from parser,
the other one is for internal info. But can we make the naming clearer?
7) The docs in general are somewhat unsatisfactory, TBH. For example the
ColumnCompression has no comments, unlike everything else in parsenodes.
Similarly for the SGML docs - I suggest to expand them to resemble FDW
docs (https://www.postgresql.org/docs/10/static/fdwhandler.html) which
also follows the handler/routines pattern.
8) One of the unclear things if why we even need 'drop' routing. It
seems that if it's defined DropAttributeCompression does something. But
what should it do? I suppose dropping the options should be done using
dependencies (just like we drop columns in this case).
BTW why does DropAttributeCompression mess with att->attisdropped in
this way? That seems a bit odd.
9) configure routines that only check if (options != NIL) and then error
out (like tsvector_configure) seem a bit unnecessary. Just allow it to
be NULL in CompressionMethodRoutine, and throw an error if options is
not NIL for such compression method.
10) toast_compress_datum still does this:
if (!ac && (valsize < PGLZ_strategy_default->min_input_size ||
valsize > PGLZ_strategy_default->max_input_size))
which seems rather pglz-specific (the naming is a hint). Why shouldn't
this be specific to compression, exposed either as min/max constants, or
wrapped in another routine - size_is_valid() or something like that?
11) The comments in toast_compress_datum probably need updating, as it
still references to pglz specifically. I guess the new compression
methods do matter too.
12) get_compression_options_info organizes the compression info into a
hash table by OID. The hash table implementation assumes the hash key is
at the beginning of the entry, but AttributeCompression is defined like
this:
typedef struct
{
CompressionMethodRoutine *routine;
List *options;
Oid cmoptoid;
} AttributeCompression;
Which means get_compression_options_info is busted, will never lookup
anything, and the hash table will grow by adding more and more entries
into the same bucket. Of course, this has extremely negative impact on
performance (pretty much arbitrarily bad, depending on how many entries
you've already added to the hash table).
Moving the OID to the beginning of the struct fixes the issue.
13) When writing the experimental extension, I was extremely confused
about the regular varlena headers, custom compression headers, etc. In
the end I stole the code from tsvector.c and whacked it a bit until it
worked, but I wouldn't dare to claim I understand how it works.
This needs to be documented somewhere. For example postgres.h has a
bunch of paragraphs about varlena headers, so perhaps it should be
there? I see the patch tweaks some of the constants, but does not update
the comment at all.
Perhaps it would be useful to provide some additional macros making
access to custom-compressed varlena values easier. Or perhaps the
VARSIZE_ANY / VARSIZE_ANY_EXHDR / VARDATA_ANY already support that? This
part is not very clear to me.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
Re: [HACKERS] Custom compression methods
On 11/15/2017 02:13 PM, Robert Haas wrote:
On Wed, Nov 15, 2017 at 4:09 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:So in the next version of the patch I can just unlink the options from
compression methods and dropping compression method will not affect
already compressed tuples. They still could be decompressed.I guess I don't understand how that can work. I mean, if somebody
removes a compression method - i.e. uninstalls the library - and you
don't have a way to make sure there are no tuples that can only be
uncompressed by that library - then you've broken the database.
Ideally, there should be a way to add a new compression method via an
extension ... and then get rid of it and all dependencies thereupon.
I share your confusion. Once you do DROP COMPRESSION METHOD, there must
be no remaining data compressed with it. But that's what the patch is
doing already - it enforces this using dependencies, as usual.
Ildus, can you explain what you meant? How could the data still be
decompressed after DROP COMPRESSION METHOD, and possibly after removing
the .so library?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, 20 Nov 2017 00:23:23 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
On 11/15/2017 02:13 PM, Robert Haas wrote:
On Wed, Nov 15, 2017 at 4:09 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:So in the next version of the patch I can just unlink the options
from compression methods and dropping compression method will not
affect already compressed tuples. They still could be
decompressed.I guess I don't understand how that can work. I mean, if somebody
removes a compression method - i.e. uninstalls the library - and you
don't have a way to make sure there are no tuples that can only be
uncompressed by that library - then you've broken the database.
Ideally, there should be a way to add a new compression method via
an extension ... and then get rid of it and all dependencies
thereupon.I share your confusion. Once you do DROP COMPRESSION METHOD, there
must be no remaining data compressed with it. But that's what the
patch is doing already - it enforces this using dependencies, as
usual.Ildus, can you explain what you meant? How could the data still be
decompressed after DROP COMPRESSION METHOD, and possibly after
removing the .so library?
The removal of the .so library will broke all compressed tuples. I
don't see a way to avoid it. I meant that DROP COMPRESSION METHOD could
remove the record from 'pg_compression' table, but actually the
compressed tuple needs only a record from 'pg_compression_opt' where
its options are located. And there is dependency between an extension
and the options so you can't just remove the extension without CASCADE,
postgres will complain.
Still it's a problem if the user used for example `SELECT
<compressed_column> INTO * FROM *` because postgres will copy compressed
tuples, and there will not be any dependencies between destination and
the options.
Also thank you for review. I will look into it today.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 11/20/2017 10:44 AM, Ildus Kurbangaliev wrote:
On Mon, 20 Nov 2017 00:23:23 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:On 11/15/2017 02:13 PM, Robert Haas wrote:
On Wed, Nov 15, 2017 at 4:09 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:So in the next version of the patch I can just unlink the options
from compression methods and dropping compression method will not
affect already compressed tuples. They still could be
decompressed.I guess I don't understand how that can work. I mean, if somebody
removes a compression method - i.e. uninstalls the library - and you
don't have a way to make sure there are no tuples that can only be
uncompressed by that library - then you've broken the database.
Ideally, there should be a way to add a new compression method via
an extension ... and then get rid of it and all dependencies
thereupon.I share your confusion. Once you do DROP COMPRESSION METHOD, there
must be no remaining data compressed with it. But that's what the
patch is doing already - it enforces this using dependencies, as
usual.Ildus, can you explain what you meant? How could the data still be
decompressed after DROP COMPRESSION METHOD, and possibly after
removing the .so library?The removal of the .so library will broke all compressed tuples. I
don't see a way to avoid it. I meant that DROP COMPRESSION METHOD could
remove the record from 'pg_compression' table, but actually the
compressed tuple needs only a record from 'pg_compression_opt' where
its options are located. And there is dependency between an extension
and the options so you can't just remove the extension without CASCADE,
postgres will complain.
I don't think we need to do anything smart here - it should behave just
like dropping a data type, for example. That is, error out if there are
columns using the compression method (without CASCADE), and drop all the
columns (with CASCADE).
Leaving around the pg_compression_opt is not a solution. Not only it's
confusing and I'm not aware about any extension because the user is
likely to remove the .so file (perhaps not directly, but e.g. by
removing the rpm package providing it).
Still it's a problem if the user used for example `SELECT
<compressed_column> INTO * FROM *` because postgres will copy compressed
tuples, and there will not be any dependencies between destination and
the options.
This seems like a rather fatal design flaw, though. I'd say we need to
force recompression of the data, in such cases. Otherwise all the
dependency tracking is rather pointless.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Nov 20, 2017, at 18:18, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
I don't think we need to do anything smart here - it should behave just
like dropping a data type, for example. That is, error out if there are
columns using the compression method (without CASCADE), and drop all the
columns (with CASCADE).
What about instead of dropping column we leave data uncompressed?
Re: [HACKERS] Custom compression methods
On 11/20/2017 04:21 PM, Евгений Шишкин wrote:
On Nov 20, 2017, at 18:18, Tomas Vondra <tomas.vondra@2ndquadrant.com
<mailto:tomas.vondra@2ndquadrant.com>> wrote:I don't think we need to do anything smart here - it should behave just
like dropping a data type, for example. That is, error out if there are
columns using the compression method (without CASCADE), and drop all the
columns (with CASCADE).What about instead of dropping column we leave data uncompressed?
That requires you to go through the data and rewrite the whole table.
And I'm not aware of a DROP command doing that, instead they just drop
the dependent objects (e.g. DROP TYPE, ...). So per PLOS the DROP
COMPRESSION METHOD command should do that too.
But I'm wondering if ALTER COLUMN ... SET NOT COMPRESSED should do that
(currently it only disables compression for new data).
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Nov 20, 2017, at 18:29, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
What about instead of dropping column we leave data uncompressed?
That requires you to go through the data and rewrite the whole table.
And I'm not aware of a DROP command doing that, instead they just drop
the dependent objects (e.g. DROP TYPE, ...). So per PLOS the DROP
COMPRESSION METHOD command should do that too.
Well, there is no much you can do with DROP TYPE. But i'd argue that compression
is different. We do not drop data in case of DROP STATISTICS or DROP INDEX.
At least there should be a way to easily alter compression method then.
Re: [HACKERS] Custom compression methods
On Mon, 20 Nov 2017 16:29:11 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
On 11/20/2017 04:21 PM, Евгений Шишкин wrote:
On Nov 20, 2017, at 18:18, Tomas Vondra
<tomas.vondra@2ndquadrant.com
<mailto:tomas.vondra@2ndquadrant.com>> wrote:I don't think we need to do anything smart here - it should behave
just like dropping a data type, for example. That is, error out if
there are columns using the compression method (without CASCADE),
and drop all the columns (with CASCADE).What about instead of dropping column we leave data uncompressed?
That requires you to go through the data and rewrite the whole table.
And I'm not aware of a DROP command doing that, instead they just drop
the dependent objects (e.g. DROP TYPE, ...). So per PLOS the DROP
COMPRESSION METHOD command should do that too.But I'm wondering if ALTER COLUMN ... SET NOT COMPRESSED should do
that (currently it only disables compression for new data).
If the table is big, decompression could take an eternity. That's why i
decided to only to disable it and the data could be decompressed using
compression options.
My idea was to keep compression options forever, since there will not
be much of them in one database. Still that requires that extension is
not removed.
I will try to find a way how to recompress data first in case it moves
to another table.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 11/20/2017 04:43 PM, Евгений Шишкин wrote:
On Nov 20, 2017, at 18:29, Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
What about instead of dropping column we leave data uncompressed?
That requires you to go through the data and rewrite the whole table.
And I'm not aware of a DROP command doing that, instead they just drop
the dependent objects (e.g. DROP TYPE, ...). So per PLOS the DROP
COMPRESSION METHOD command should do that too.Well, there is no much you can do with DROP TYPE. But i'd argue that compression
is different. We do not drop data in case of DROP STATISTICS or DROP INDEX.
But those DROP commands do not 'invalidate' data in the heap, so there's
no reason to drop the columns.
At least there should be a way to easily alter compression method then.
+1
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, 20 Nov 2017 00:04:53 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
I did a review of this today, and I think there are some things that
need improvement / fixing.Firstly, some basic comments from just eye-balling the diff, then some
bugs I discovered after writing an extension adding lz4.1) formatRelOptions/freeRelOptions are no longer needed (I see Ildar
already pointer that out)
I removed freeRelOptions, but formatRelOptions is used in other place.
2) There's unnecessary whitespace (extra newlines) on a couple of
places, which is needlessly increasing the size of the patch. Small
difference, but annoying.
Cleaned up.
3) tuptoaster.c
Why do you change 'info' from int32 to uint32? Seems unnecessary.
That's because I use highest bit, and it makes number negative for
int32. I use right shifting to get that bit and right shift on negative
gives negative value too.
Adding new 'att' variable in toast_insert_or_update is confusing, as
there already is 'att' in the very next loop. Technically it's
correct, but I'd bet it'll lead to some WTF?! moments later. I
propose to just use TupleDescAttr(tupleDesc,i) on the two places
where it matters, around line 808.There are no comments for init_compression_options_htab and
get_compression_options_info, so that needs to be fixed. Moreover, the
names are confusing because what we really get is not just 'options'
but the compression routines too.
Removed extra 'att', and added comments.
4) gen_db_file_maps probably shouldn't do the fprints, right?
5) not sure why you modify src/tools/pgindent/exclude_file_patterns
My bad, removed these lines.
6) I'm rather confused by AttributeCompression vs. ColumnCompression.
I mean, attribute==column, right? Of course, one is for data from
parser, the other one is for internal info. But can we make the
naming clearer?
For now I have renamed AttributeCompression to CompressionOptions, not
sure that's a good name but at least it gives less confusion.
7) The docs in general are somewhat unsatisfactory, TBH. For example
the ColumnCompression has no comments, unlike everything else in
parsenodes. Similarly for the SGML docs - I suggest to expand them to
resemble FDW docs
(https://www.postgresql.org/docs/10/static/fdwhandler.html) which
also follows the handler/routines pattern.
I've added more comments. I think I'll add more documentation if the
committers will approve current syntax.
8) One of the unclear things if why we even need 'drop' routing. It
seems that if it's defined DropAttributeCompression does something.
But what should it do? I suppose dropping the options should be done
using dependencies (just like we drop columns in this case).BTW why does DropAttributeCompression mess with att->attisdropped in
this way? That seems a bit odd.
'drop' routine could be useful. An extension could do
something related with the attribute, like remove extra tables or
something else. The compression options will not be removed after
unlinking compression method from a column because there is still be
stored compressed data in that column.
That 'attisdropped' part has been removed.
9) configure routines that only check if (options != NIL) and then
error out (like tsvector_configure) seem a bit unnecessary. Just
allow it to be NULL in CompressionMethodRoutine, and throw an error
if options is not NIL for such compression method.
Good idea, done.
10) toast_compress_datum still does this:
if (!ac && (valsize < PGLZ_strategy_default->min_input_size ||
valsize > PGLZ_strategy_default->max_input_size))which seems rather pglz-specific (the naming is a hint). Why shouldn't
this be specific to compression, exposed either as min/max constants,
or wrapped in another routine - size_is_valid() or something like
that?
I agree, moved to the next block related with pglz.
11) The comments in toast_compress_datum probably need updating, as it
still references to pglz specifically. I guess the new compression
methods do matter too.
Done.
12) get_compression_options_info organizes the compression info into a
hash table by OID. The hash table implementation assumes the hash key
is at the beginning of the entry, but AttributeCompression is defined
like this:typedef struct
{
CompressionMethodRoutine *routine;
List *options;
Oid cmoptoid;
} AttributeCompression;Which means get_compression_options_info is busted, will never lookup
anything, and the hash table will grow by adding more and more entries
into the same bucket. Of course, this has extremely negative impact on
performance (pretty much arbitrarily bad, depending on how many
entries you've already added to the hash table).Moving the OID to the beginning of the struct fixes the issue.
Yeah, I fixed it before, but somehow managed to do not include it to the
patch.
13) When writing the experimental extension, I was extremely confused
about the regular varlena headers, custom compression headers, etc. In
the end I stole the code from tsvector.c and whacked it a bit until it
worked, but I wouldn't dare to claim I understand how it works.This needs to be documented somewhere. For example postgres.h has a
bunch of paragraphs about varlena headers, so perhaps it should be
there? I see the patch tweaks some of the constants, but does not
update the comment at all.
This point is good, I'm not sure how this documentation should look
like. I've just assumed that people should have deep undestanding of
varlenas if they're going to compress them. But now it's easy to make
mistake there. Maybe I should add some functions that help to construct
varlena, with different headers. I like the way is how jsonb is
constructed. It uses StringInfo and there are few helper functions
(reserveFromBuffer, appendToBuffer and others). Maybe
they should be not static.
Perhaps it would be useful to provide some additional macros making
access to custom-compressed varlena values easier. Or perhaps the
VARSIZE_ANY / VARSIZE_ANY_EXHDR / VARDATA_ANY already support that?
This part is not very clear to me.
These macros will work, custom compressed varlenas behave like old
compressed varlenas.
Still it's a problem if the user used for example `SELECT
<compressed_column> INTO * FROM *` because postgres will copy
compressed tuples, and there will not be any dependencies between
destination and the options.This seems like a rather fatal design flaw, though. I'd say we need to
force recompression of the data, in such cases. Otherwise all the
dependency tracking is rather pointless.
Fixed this problem too. I've added recompression for datum that use
custom compression.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v5.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 01acc2ef9d..f43f09cc19 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -59,6 +59,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 92db00f52d..207d6cf26e 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,8 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET NOT COMPRESSED
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +322,34 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. Setting a compression method doesn't change anything in the
+ table and affects only future table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>SET NOT COMPRESSED</literal>
+ </term>
+ <listitem>
+ <para>
+ This form removes compression from a column. Removing compresssion from
+ a column doesn't change already compressed tuples and affects only future
+ table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..663010ecd9
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,50 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="SQL-CREATECOMPRESSIONMETHOD">
+ <indexterm zone="sql-createcompressionmethod">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">compression_method_name</replaceable>
+ HANDLER <replaceable class="parameter">compression_method_handler</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method
+ with <replaceable class="parameter">compression_method_name</replaceable>.
+ </para>
+
+ <para>
+ A compression method links a name with a compression handler. And the
+ handler is a special function that returns collection of methods that
+ can be used for compression.
+ </para>
+
+ <para>
+ After a compression method is created, you can specify it in
+ <xref linkend="SQL-CREATETABLE"> or <xref linkend="SQL-ALTERTABLE">
+ statements.
+ </para>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 83eef7f10d..9631279cb0 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index 9000b3aaaa..cc0bd70be3 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -87,6 +87,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..629f3575a9 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -92,7 +92,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index aa9c0f1bb9..f740ce4304 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -945,11 +945,31 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca73a8..d206cce18e 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -242,6 +244,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -396,6 +399,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -458,6 +463,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -563,6 +569,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -675,7 +682,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index c74945a52a..99307cb5e6 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -39,6 +41,8 @@
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
#include "utils/typcache.h"
@@ -53,19 +57,52 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+static HTAB *compression_options_cache = NULL;
+static MemoryContext compression_options_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ ((toast_compress_header *) (ptr))->info &= 0xC0000000; \
+ ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +120,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_compression_options_cache(void);
+static CompressionOptions * get_cached_compression_options(Oid cmoptoid);
/* ----------
@@ -421,7 +460,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_SIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -686,8 +725,24 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
toast_oldexternal[i] = new_value;
if (att->attstorage == 'p')
new_value = heap_tuple_untoast_attr(new_value);
+ else if (VARATT_IS_EXTERNAL_ONDISK(new_value))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, new_value);
+
+ /*
+ * If we're trying to insert a custom compressed datum we
+ * should decompress it first.
+ */
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ new_value = heap_tuple_untoast_attr(new_value);
+ else
+ new_value = heap_tuple_fetch_attr(new_value);
+ }
else
new_value = heap_tuple_fetch_attr(new_value);
+
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +796,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +811,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +829,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +974,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1353,7 +1414,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,41 +1428,57 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoptoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize,
+ len = 0;
+ CompressionOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ if (OidIsValid(cmoptoid))
+ cmoptions = get_cached_compression_options(cmoptoid);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (cmoptions)
+ {
+ tmp = cmoptions->routine->compress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ /*
+ * No point in wasting a palloc cycle if value size is out of the
+ * allowed range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return PointerGetDatum(NULL);
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!cmoptions && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,10 +1486,20 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (cmoptions && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, cmoptions->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
- pfree(tmp);
+ if (tmp)
+ pfree(tmp);
+
return PointerGetDatum(NULL);
}
}
@@ -1510,19 +1596,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1617,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1629,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +1989,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_SIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2174,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_SIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2370,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = get_cached_compression_options(hdr->cmoptoid);
+ result = cmoptions->routine->decompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2491,65 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * init_compression_options_cache
+ *
+ * Initialize a local cache for compression options.
+ */
+static void
+init_compression_options_cache(void)
+{
+ HASHCTL ctl;
+
+ compression_options_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "compression options cache context",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionOptions);
+ ctl.hcxt = compression_options_mcxt;
+ compression_options_cache = hash_create("compression options cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+}
+
+/* ----------
+ * get_cached_compression_options
+ *
+ * Get cached compression options structure or create it if it's not in cache.
+ * Cache is required because we can't afford for each tuple create
+ * CompressionMethodRoutine and parse its options.
+ */
+static CompressionOptions *
+get_cached_compression_options(Oid cmoptoid)
+{
+ bool found;
+ CompressionOptions *result;
+
+ Assert(OidIsValid(cmoptoid));
+ if (!compression_options_cache)
+ init_compression_options_cache();
+
+ result = hash_search(compression_options_cache, &cmoptoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ Assert(compression_options_mcxt);
+ oldcxt = MemoryContextSwitchTo(compression_options_mcxt);
+ result->cmoptoid = cmoptoid;
+ PG_TRY();
+ {
+ result->routine = GetCompressionRoutine(cmoptoid);
+ result->options = GetCompressionOptionsList(cmoptoid);
+ }
+ PG_CATCH();
+ {
+ hash_search(compression_options_cache, &cmoptoid, HASH_REMOVE, &found);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+ return result;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 8287de97a2..be6b460aee 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 033c4358ea..e1bfc7c6bf 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2512,6 +2524,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 256a9c9c93..c5838fa779 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -451,6 +451,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9e14880b99..4f958be51f 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -707,6 +712,13 @@ AddNewAttributeTuples(Oid new_rel_oid,
referenced.objectSubId = 0;
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+
+ if (OidIsValid(attr->attcompression))
+ {
+ ObjectAddressSet(referenced, CompressionOptRelationId,
+ attr->attcompression);
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ }
}
/*
@@ -1453,6 +1465,24 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+/*
+ * CallCompressionDropCallback
+ *
+ * Call drop callback from compression routine.
+ */
+static void
+CallCompressionDropCallback(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ cmr->drop(att, options);
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1513,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ CallCompressionDropCallback(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1613,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ CallCompressionDropCallback(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1636,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0125c18bc1..15942564aa 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8d55c76fc4..9fd1cb763d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index e02d312008..531a820464 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..fa2ad8a864
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,489 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/compression.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine * get_compression_method_routine(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+LookupCompressionHandlerFunc(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+static ObjectAddress
+CreateCompressionMethod(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = LookupCompressionHandlerFunc(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+/*
+ * CREATE COMPRESSION METHOD .. HANDLER ..
+ */
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ return CreateCompressionMethod(cmName, (List *) handlerEl->arg);
+}
+
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, Oid cmid, List *options)
+{
+ Relation rel;
+ HeapTuple tup,
+ newtup;
+ Oid cmoptoid;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+
+ ObjectAddress myself,
+ ref1,
+ ref2,
+ ref3;
+
+ CompressionMethodRoutine *routine;
+ Form_pg_compression cmform;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ /* Get handler function OID for the compression method */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ routine = get_compression_method_routine(cmform->cmhandler, attr->atttypid);
+
+ if (routine->configure == NULL && options != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("the compression method \"%s\" does not take any options",
+ NameStr(cmform->cmname))));
+ else if (routine->configure && options != NIL)
+ routine->configure(attr, options);
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ cmoptoid = GetNewOidWithIndex(rel, CompressionOptionsOidIndexId,
+ Anum_pg_compression_opt_cmoptoid);
+ values[Anum_pg_compression_opt_cmoptoid - 1] = ObjectIdGetDatum(cmoptoid);
+ values[Anum_pg_compression_opt_cmname - 1] = NameGetDatum(&cmform->cmname);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmform->cmhandler);
+
+ if (options)
+ values[Anum_pg_compression_opt_cmoptions - 1] = optionListToArray(options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+
+ ReleaseSysCache(tup);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, cmoptoid);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmform->cmhandler);
+ ObjectAddressSubSet(ref2, RelationRelationId, attr->attrelid, attr->attnum);
+ ObjectAddressSet(ref3, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&ref2, &myself, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref3, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return cmoptoid;
+}
+
+/*
+ * Remove the compression options record from pg_compression_opt
+ */
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmoptform;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = pstrdup(NameStr(cmoptform->cmname));
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression * c1, ColumnCompression * c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->methodName, c2->methodName))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->methodName, c2->methodName)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+ Form_pg_compression_opt cmoptform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmoptform->cmname));
+ ReleaseSysCache(tup);
+
+ return result;
+}
+
+/* get_compression_options */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+CompressionMethodRoutine *
+GetCompressionRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return get_compression_method_routine(cmhandler, InvalidOid);
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+get_compression_method_routine(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 13eb9e34ba..f84588ed31 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2781,8 +2781,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 4d77411a68..aba2087839 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -112,7 +112,8 @@ create_ctas_internal(List *attrList, IntoClause *into)
* Create the relation. (This will error out if there's an existing view,
* so we don't need more code to complain if "replace" is false.)
*/
- intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL);
+ intoRelationAddr = DefineRelation(create, relkind, InvalidOid, NULL, NULL,
+ NULL);
/*
* If necessary, create a TOAST table for the target table. Note that
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index fa7d0d015a..a2eee78a64 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 5e1b0fe289..eb9f72dcba 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -212,7 +212,8 @@ DefineSequence(ParseState *pstate, CreateSeqStmt *seq)
stmt->tablespacename = NULL;
stmt->if_not_exists = seq->if_not_exists;
- address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL);
+ address = DefineRelation(stmt, RELKIND_SEQUENCE, seq->ownerId, NULL, NULL,
+ NULL);
seqoid = address.objectId;
Assert(seqoid != InvalidOid);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d19846d005..f31be71e65 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -459,6 +464,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression * compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -503,7 +510,8 @@ static ObjectAddress ATExecDetachPartition(Relation rel, RangeVar *name);
*/
ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString)
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt)
{
char relname[NAMEDATALEN];
Oid namespaceId;
@@ -523,6 +531,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
static char *validnsps[] = HEAP_RELOPT_NAMESPACES;
Oid ofTypeId;
ObjectAddress address;
+ AlterTableStmt *alterStmt = NULL;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -724,6 +733,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ transformColumnCompression(colDef, stmt->relation, &alterStmt);
}
/*
@@ -921,6 +932,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
relation_close(rel, NoLock);
+ if (pAlterStmt)
+ *pAlterStmt = (Node *) alterStmt;
+ else
+ Assert(!alterStmt);
+
return address;
}
@@ -1610,6 +1626,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1944,6 +1961,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1971,6 +2001,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2180,6 +2213,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3279,6 +3319,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3770,6 +3811,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4118,6 +4165,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5338,6 +5390,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6438,6 +6492,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9361,6 +9420,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9410,7 +9471,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9432,6 +9495,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attbyval = tform->typbyval;
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ attTup->attcompression = InvalidOid;
ReleaseSysCache(typeTuple);
@@ -12484,6 +12548,83 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression * compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName)
+ {
+ /* SET COMPRESSED */
+ Oid cmid,
+ cmoptoid;
+
+ cmid = get_compression_method_oid(compression->methodName, false);
+ cmoptoid = CreateCompressionOptions(atttableform, cmid, compression->options);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c054..be4a6c5f38 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -2182,7 +2183,7 @@ DefineCompositeType(RangeVar *typevar, List *coldeflist)
* Finally create the relation. This also creates the type.
*/
DefineRelation(createStmt, RELKIND_COMPOSITE_TYPE, InvalidOid, &address,
- NULL);
+ NULL, NULL);
return address;
}
diff --git a/src/backend/commands/view.c b/src/backend/commands/view.c
index c1e80e61d4..cc0182075e 100644
--- a/src/backend/commands/view.c
+++ b/src/backend/commands/view.c
@@ -251,7 +251,7 @@ DefineVirtualRelation(RangeVar *relation, List *tlist, bool replace,
* false).
*/
address = DefineRelation(createStmt, RELKIND_VIEW, InvalidOid, NULL,
- NULL);
+ NULL, NULL);
Assert(address.objectId != InvalidOid);
/* Make the new view relation visible */
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 5dfc49deb9..3a80e997a9 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -302,6 +302,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
var->vartypmod != -1))
return false; /* type mismatch */
+ if (OidIsValid(att_tup->attcompression))
+ return false;
+
tlist_item = lnext(tlist_item);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e48921fff1..c9de2344fe 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2806,6 +2806,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2824,6 +2825,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression * from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5473,6 +5485,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2a83da9aa9..d618cc637e 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2544,6 +2544,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2562,6 +2563,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression * a, const ColumnCompression * b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3615,6 +3625,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index c2a93b2d4c..81e334a1d3 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index c97ee24ade..5ae1db146e 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2799,6 +2799,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2815,6 +2816,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression * node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4106,6 +4116,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c301ca465d..122d3de4d6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -397,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +583,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +617,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2168,6 +2171,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3332,11 +3344,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3346,8 +3359,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3394,6 +3407,37 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5754,6 +5798,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6262,6 +6315,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6325,7 +6379,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6515,6 +6569,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14704,6 +14759,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 8461da490a..39f21d4da7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -494,6 +495,39 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+/*
+ * transformColumnCompression
+ *
+ * Build ALTER TABLE .. SET COMPRESSED command if a compression method was
+ * specified for the column
+ */
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -794,6 +828,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1003,6 +1047,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 0f607bab70..0ff169e200 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2878,7 +2878,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_SIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 82a707af7b..331d133660 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -998,6 +998,7 @@ ProcessUtilitySlow(ParseState *pstate,
foreach(l, stmts)
{
Node *stmt = (Node *) lfirst(l);
+ Node *alterStmt = NULL;
if (IsA(stmt, CreateStmt))
{
@@ -1008,7 +1009,9 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_RELATION,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1042,7 +1045,8 @@ ProcessUtilitySlow(ParseState *pstate,
address = DefineRelation((CreateStmt *) stmt,
RELKIND_FOREIGN_TABLE,
InvalidOid, NULL,
- queryString);
+ queryString,
+ &alterStmt);
CreateForeignTable((CreateForeignTableStmt *) stmt,
address.objectId);
EventTriggerCollectSimpleCommand(address,
@@ -1074,6 +1078,9 @@ ProcessUtilitySlow(ParseState *pstate,
NULL);
}
+ if (alterStmt)
+ lappend(stmts, alterStmt);
+
/* Need CCI between commands */
if (lnext(l) != NULL)
CommandCounterIncrement();
@@ -1283,6 +1290,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -1696,6 +1708,11 @@ ExecDropStmt(DropStmt *stmt, bool isTopLevel)
case OBJECT_FOREIGN_TABLE:
RemoveRelations(stmt);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ if (stmt->behavior == DROP_CASCADE)
+ {
+ /* TODO decompress columns instead of their deletion */
+ }
default:
RemoveObjects(stmt);
break;
@@ -2309,6 +2326,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2432,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index b0a9217d1e..e1314e1f2e 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,85 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(CompressionOptions * cmoptions, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+tsvector_decompress(CompressionOptions * cmoptions, const struct varlena *data)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = NULL;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1908420d82..de834c5593 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 888edbb325..c689f60e47 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ Anum_pg_compression_opt_cmoptoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 4b47951de1..7ba6d31251 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
static DumpableObject **nspinfoindex;
static DumpableObject **extinfoindex;
+static DumpableObject **cminfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
@@ -61,6 +62,7 @@ static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
+static int numCompressionMethods;
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
@@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+ CompressionMethodInfo *cminfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr)
write_msg(NULL, "reading subscriptions\n");
getSubscriptions(fout);
+ if (g_verbose)
+ write_msg(NULL, "reading compression methods\n");
+ cminfo = getCompressionMethods(fout, &numCompressionMethods);
+ cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo));
+
*numTablesPtr = numTables;
return tblinfo;
}
@@ -827,6 +836,17 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
+/*
+ * findCompressionMethodByOid
+ * finds the entry (in cminfo) of the compression method with the given oid
+ * returns NULL if not found
+ */
+CompressionMethodInfo *
+findCompressionMethodByOid(Oid oid)
+{
+ return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex,
+ numCompressionMethods);
+}
/*
* setExtensionMembership
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..5c1aaad48e 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,8 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +152,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +174,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index ec2fa8b9b9..77eb55eb69 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_options_data = ropt->compression_options_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d8fb356130..6df688adb8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -361,6 +361,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -581,6 +582,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_options_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -785,6 +789,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_options_data)
+ getCompressionOptions(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -3957,6 +3964,205 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getCompressionMethods
+ * get information about compression methods
+ */
+CompressionMethodInfo *
+getCompressionMethods(Archive *fout, int *numMethods)
+{
+ DumpOptions *dopt = fout->dopt;
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionMethodInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i,
+ ntups;
+
+ if (dopt->no_compression_methods || fout->remoteVersion < 110000)
+ return NULL;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ /* Get the compression methods in current database. */
+ appendPQExpBuffer(query, "SELECT c.tableoid, c.oid, c.cmname, "
+ "c.cmhandler::pg_catalog.regproc as cmhandler "
+ "FROM pg_catalog.pg_compression c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "oid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_METHOD;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+
+ /* Decide whether we want to dump it */
+ selectDumpableObject(&(cminfo[i].dobj), fout);
+ }
+ if (numMethods)
+ *numMethods = ntups;
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+
+ return cminfo;
+}
+
+/*
+ * dumpCompressionMethod
+ * dump the definition of the given compression method
+ */
+static void
+dumpCompressionMethod(Archive *fout, CompressionMethodInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER",
+ fmtId(cminfo->dobj.name));
+ appendPQExpBuffer(query, " %s;\n", cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION METHOD", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * getCompressionOptions
+ * get information about compression options.
+ * this information should be migrated at binary upgrade
+ */
+void
+getCompressionOptions(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionOptionsInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i_options;
+ int i,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ appendPQExpBuffer(query,
+ "SELECT c.tableoid, c.cmoptoid, c.cmname,"
+ " c.cmhandler::pg_catalog.regproc as cmhandler, c.cmoptions "
+ "FROM pg_catalog.pg_compression_opt c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "cmoptoid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+ i_options = PQfnumber(res, "cmoptions");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionOptionsInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_OPTIONS;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].cmoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpCompressionOptions
+ * dump the given compression options
+ */
+static void
+dumpCompressionOptions(Archive *fout, CompressionOptionsInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ if (strlen(cminfo->cmoptions) > 0)
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), '%s');\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler,
+ cminfo->cmoptions);
+ else
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), NULL);\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION OPTIONS", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -4484,7 +4690,47 @@ getTypes(Archive *fout, int *numTypes)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- if (fout->remoteVersion >= 90600)
+ if (fout->remoteVersion >= 110000)
+ {
+ PQExpBuffer acl_subquery = createPQExpBuffer();
+ PQExpBuffer racl_subquery = createPQExpBuffer();
+ PQExpBuffer initacl_subquery = createPQExpBuffer();
+ PQExpBuffer initracl_subquery = createPQExpBuffer();
+
+ buildACLQueries(acl_subquery, racl_subquery, initacl_subquery,
+ initracl_subquery, "t.typacl", "t.typowner", "'T'",
+ dopt->binary_upgrade);
+
+ appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
+ "t.typnamespace, "
+ "%s AS typacl, "
+ "%s AS rtypacl, "
+ "%s AS inittypacl, "
+ "%s AS initrtypacl, "
+ "(%s t.typowner) AS rolname, "
+ "t.typelem, t.typrelid, "
+ "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" "
+ "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, "
+ "t.typtype, t.typisdefined, "
+ "t.typname[0] = '_' AND t.typelem != 0 AND "
+ "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray "
+ "FROM pg_type t "
+ "LEFT JOIN pg_init_privs pip ON "
+ "(t.oid = pip.objoid "
+ "AND pip.classoid = 'pg_type'::regclass "
+ "AND pip.objsubid = 0) ",
+ acl_subquery->data,
+ racl_subquery->data,
+ initacl_subquery->data,
+ initracl_subquery->data,
+ username_subquery);
+
+ destroyPQExpBuffer(acl_subquery);
+ destroyPQExpBuffer(racl_subquery);
+ destroyPQExpBuffer(initacl_subquery);
+ destroyPQExpBuffer(initracl_subquery);
+ }
+ else if (fout->remoteVersion >= 90600)
{
PQExpBuffer acl_subquery = createPQExpBuffer();
PQExpBuffer racl_subquery = createPQExpBuffer();
@@ -7895,6 +8141,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7930,7 +8178,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.cmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.cmname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.cmoptoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7949,9 +8236,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2 "
"ORDER BY a.attnum",
@@ -7975,7 +8266,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7999,7 +8292,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8017,7 +8312,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8034,7 +8331,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8064,6 +8363,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8080,6 +8381,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8107,6 +8410,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9596,6 +9901,12 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_COMPRESSION_METHOD:
+ dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj);
+ break;
+ case DO_COMPRESSION_OPTIONS:
+ dumpCompressionOptions(fout, (CompressionOptionsInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -15513,6 +15824,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSED %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17778,6 +18098,8 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_COMPRESSION_METHOD:
+ case DO_COMPRESSION_OPTIONS:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da884ffd09..88b18ba15c 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,9 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_COMPRESSION_METHOD,
+ DO_COMPRESSION_OPTIONS
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -315,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +615,21 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+/* The CompressionMethodInfo struct is used to represent compression method */
+typedef struct _CompressionMethodInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+} CompressionMethodInfo;
+
+/* The CompressionOptionsInfo struct is used to represent compression options */
+typedef struct _CompressionOptionsInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+ char *cmoptions;
+} CompressionOptionsInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -654,6 +673,7 @@ extern OprInfo *findOprByOid(Oid oid);
extern CollInfo *findCollationByOid(Oid oid);
extern NamespaceInfo *findNamespaceByOid(Oid oid);
extern ExtensionInfo *findExtensionByOid(Oid oid);
+extern CompressionMethodInfo * findCompressionMethodByOid(Oid oid);
extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems);
extern ExtensionInfo *findOwningExtension(CatalogId catalogId);
@@ -711,5 +731,8 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern CompressionMethodInfo * getCompressionMethods(Archive *fout,
+ int *numMethods);
+void getCompressionOptions(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 48b6dd594c..f5db0280e2 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -80,7 +80,9 @@ static const int dbObjectTypePriority[] =
34, /* DO_POLICY */
35, /* DO_PUBLICATION */
36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 37, /* DO_SUBSCRIPTION */
+ 17, /* DO_COMPRESSION_METHOD */
+ 17 /* DO_COMPRESSION_OPTIONS */
};
static DumpId preDataBoundId;
@@ -1436,6 +1438,16 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_COMPRESSION_METHOD:
+ snprintf(buf, bufsize,
+ "COMPRESSION METHOD %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
+ case DO_COMPRESSION_OPTIONS:
+ snprintf(buf, bufsize,
+ "COMPRESSION OPTIONS %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..6b72ccf9ea 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 8cc4de3878..8b0dbfa45c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index b7b978a361..49160ded0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1716,6 +1779,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " WHERE c.cmoptoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1830,6 +1909,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1925,6 +2008,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1932,7 +2020,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1943,7 +2031,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a926c40b9b..25d2fbc125 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b3e3799c13..a323ca80c6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..6c0e5fda5f
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,66 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ Oid cmoptoid;
+ CompressionMethodRoutine *routine;
+ List *options;
+} CompressionOptions;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (CompressionOptions * cmoptions, const struct varlena *data);
+
+/*
+ * API struct for a compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionConfigureRoutine drop;
+ CompressionRoutine compress;
+ CompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+/* Compression method handler parameters */
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine * GetCompressionRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr, Oid cmid,
+ List *options);
+extern ColumnCompression * GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression * c1,
+ ColumnCompression * c2, const char *attributeName);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index cd43e3a52e..2e1c92123e 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1d3e..867adc1bd8 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -88,6 +90,9 @@ typedef struct tupleDesc
/* Accessor for the i'th attribute of tupdesc. */
#define TupleDescAttr(tupdesc, i) (&(tupdesc)->attrs[(i)])
+#define TupleDescAttrCompression(tupdesc, i) \
+ ((tupdesc)->tdcompression? &((tupdesc)->tdcompression[i]) : NULL)
+
extern TupleDesc CreateTemplateTupleDesc(int natts, bool hasoid);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..0632588e5d 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_SIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_SIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -210,7 +219,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..7fe9f1f059 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(cmoptoid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..40f5cc4f18 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..ddcef814a7
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420) BKI_WITHOUT_OIDS
+{
+ Oid cmoptoid; /* compression options oid */
+ NameData cmname; /* name of compression method */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 4
+#define Anum_pg_compression_opt_cmoptoid 1
+#define Anum_pg_compression_opt_cmname 2
+#define Anum_pg_compression_opt_cmhandler 3
+#define Anum_pg_compression_opt_cmoptions 4
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index c969375981..dc3fa66f2e 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3881,6 +3881,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4677,6 +4681,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e3551440a0..ec8c3df953 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bfead9af3d..a98ecc12ce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index da3ff5dbee..c50d9525d0 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -24,7 +24,8 @@
extern ObjectAddress DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
- ObjectAddress *typaddress, const char *queryString);
+ ObjectAddress *typaddress, const char *queryString,
+ Node **pAlterStmt);
extern void RemoveRelations(DropStmt *drop);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ffeeb4919b..6dc49a73b6 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -468,6 +468,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -498,7 +499,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 80c19b2a55..9fa905ae6e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSED <compression_method_name> WITH (<params>)
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +651,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1622,6 +1636,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1769,7 +1784,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..d21974696f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 254a811aff..6ad889af7a 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -210,6 +210,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a0be41929..ff7cb530fd 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..16d66a0263
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,145 @@
+-- test drop
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE droptest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table droptest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 2 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to table droptest column fts
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+-- check ALTER commands
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method "ts1" does not take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+-- create different types of tables
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest);
+NOTICE: merging column "fts" with inherited definition
+CREATE TABLE cmtest5(fts tsvector);
+-- we update usual datum with compressed datum
+INSERT INTO cmtest5
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest;
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+DROP TABLE cmtest3;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 4 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+-----------
+(0 rows)
+
+SELECT * FROM pg_compression_opt;
+ cmoptoid | cmname | cmhandler | cmoptions
+----------+--------+-----------+-----------
+(0 rows)
+
+-- check that moved tuples still can be decompressed
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest5;
+ length
+--------
+ 200
+(1 row)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest5;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 335cd37e18..b511daf9fa 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -590,10 +590,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -729,11 +729,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -756,11 +756,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -793,46 +793,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..2ac75c44b5 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 331f7a911f..530ce1b1d0 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index c698faff2f..a147b8217f 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 9d84ba4658..bbab453247 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -435,10 +435,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -750,74 +750,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index e996640593..62b06da011 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index a4fe96112e..adbe764196 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index aa5e6af621..a44cf1c910 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 3866314a92..5c72cb16f5 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..f4701a1dbf
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,58 @@
+-- test drop
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE droptest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+DROP COMPRESSION METHOD ts1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+
+SELECT length(fts) FROM cmtest;
+SELECT length(fts) FROM cmtest;
+
+-- check ALTER commands
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+-- create different types of tables
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest);
+CREATE TABLE cmtest5(fts tsvector);
+
+-- we update usual datum with compressed datum
+INSERT INTO cmtest5
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest;
+
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+DROP TABLE cmtest3;
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
+SELECT * FROM pg_compression;
+SELECT * FROM pg_compression_opt;
+
+-- check that moved tuples still can be decompressed
+SELECT length(fts) FROM cmtest2;
+SELECT length(fts) FROM cmtest5;
+DROP TABLE cmtest2;
+DROP TABLE cmtest5;
Re: [HACKERS] Custom compression methods
Hi,
On 11/21/2017 03:47 PM, Ildus Kurbangaliev wrote:
On Mon, 20 Nov 2017 00:04:53 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:...
6) I'm rather confused by AttributeCompression vs.
ColumnCompression. I mean, attribute==column, right? Of course, one
is for data from parser, the other one is for internal info. But
can we make the naming clearer?For now I have renamed AttributeCompression to CompressionOptions,
not sure that's a good name but at least it gives less confusion.
I propose to use either
CompressionMethodOptions (and CompressionMethodRoutine)
or
CompressionOptions (and CompressionRoutine)
7) The docs in general are somewhat unsatisfactory, TBH. For example
the ColumnCompression has no comments, unlike everything else in
parsenodes. Similarly for the SGML docs - I suggest to expand them to
resemble FDW docs
(https://www.postgresql.org/docs/10/static/fdwhandler.html) which
also follows the handler/routines pattern.I've added more comments. I think I'll add more documentation if the
committers will approve current syntax.
OK. Haven't reviewed this yet.
8) One of the unclear things if why we even need 'drop' routing. It
seems that if it's defined DropAttributeCompression does something.
But what should it do? I suppose dropping the options should be done
using dependencies (just like we drop columns in this case).BTW why does DropAttributeCompression mess with att->attisdropped in
this way? That seems a bit odd.'drop' routine could be useful. An extension could do something
related with the attribute, like remove extra tables or something
else. The compression options will not be removed after unlinking
compression method from a column because there is still be stored
compressed data in that column.
OK. So something like a "global" dictionary used for the column, or
something like that? Sure, seems useful and I've been thinking about
that, but I think we badly need some extension using that, even if in a
very simple way. Firstly, we need a "how to" example, secondly we need
some way to test it.
13) When writing the experimental extension, I was extremely
confused about the regular varlena headers, custom compression
headers, etc. In the end I stole the code from tsvector.c and
whacked it a bit until it worked, but I wouldn't dare to claim I
understand how it works.This needs to be documented somewhere. For example postgres.h has
a bunch of paragraphs about varlena headers, so perhaps it should
be there? I see the patch tweaks some of the constants, but does
not update the comment at all.This point is good, I'm not sure how this documentation should look
like. I've just assumed that people should have deep undestanding of
varlenas if they're going to compress them. But now it's easy to
make mistake there. Maybe I should add some functions that help to
construct varlena, with different headers. I like the way is how
jsonb is constructed. It uses StringInfo and there are few helper
functions (reserveFromBuffer, appendToBuffer and others). Maybe they
should be not static.
Not sure. My main problem was not understanding how this affects the
varlena header, etc. And I had no idea where to look.
Perhaps it would be useful to provide some additional macros
making access to custom-compressed varlena values easier. Or
perhaps the VARSIZE_ANY / VARSIZE_ANY_EXHDR / VARDATA_ANY already
support that? This part is not very clear to me.These macros will work, custom compressed varlenas behave like old
compressed varlenas.
OK. But then I don't understand why tsvector.c does things like
VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize
VARRAWSIZE_4B_C(data) - arrsize
instead of
VARSIZE_ANY_EXHDR(data) - arrsize
VARSIZE_ANY(data) - arrsize
Seems somewhat confusing.
Still it's a problem if the user used for example `SELECT
<compressed_column> INTO * FROM *` because postgres will copy
compressed tuples, and there will not be any dependencies
between destination and the options.This seems like a rather fatal design flaw, though. I'd say we need
to force recompression of the data, in such cases. Otherwise all
the dependency tracking is rather pointless.Fixed this problem too. I've added recompression for datum that use
custom compression.
Hmmm, it still doesn't work for me. See this:
test=# create extension pg_lz4 ;
CREATE EXTENSION
test=# create table t_lz4 (v text compressed lz4);
CREATE TABLE
test=# create table t_pglz (v text);
CREATE TABLE
test=# insert into t_lz4 select repeat(md5(1::text),300);
INSERT 0 1
test=# insert into t_pglz select * from t_lz4;
INSERT 0 1
test=# drop extension pg_lz4 cascade;
NOTICE: drop cascades to 2 other objects
DETAIL: drop cascades to compression options for lz4
drop cascades to table t_lz4 column v
DROP EXTENSION
test=# \c test
You are now connected to database "test" as user "user".
test=# insert into t_lz4 select repeat(md5(1::text),300);^C
test=# select * from t_pglz ;
ERROR: cache lookup failed for compression options 16419
That suggests no recompression happened.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Tue, 21 Nov 2017 18:47:49 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
I propose to use either
CompressionMethodOptions (and CompressionMethodRoutine)
or
CompressionOptions (and CompressionRoutine)
Sounds good, thanks.
OK. But then I don't understand why tsvector.c does things like
VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize
VARRAWSIZE_4B_C(data) - arrsizeinstead of
VARSIZE_ANY_EXHDR(data) - arrsize
VARSIZE_ANY(data) - arrsizeSeems somewhat confusing.
VARRAWSIZE_4B_C returns original size of data, before compression (from
va_rawsize in current postgres, and from va_info in my patch), not size
of the already compressed data, so you can't use VARSIZE_ANY here.
VARSIZE_ANY_EXHDR in current postgres returns VARSIZE-VARHDRSZ, despite
the varlena is compressed or not, so I just kept this behavior for
custom compressed varlenas too. If you look into tuptoaster.c you will
also see lines like 'VARSIZE(attr) - TOAST_COMPRESS_HDRSZ'. So I think
if VARSIZE_ANY_EXHDR will subtract different header sizes then it
should subtract them for usual compressed varlenas too.
Hmmm, it still doesn't work for me. See this:
test=# create extension pg_lz4 ;
CREATE EXTENSION
test=# create table t_lz4 (v text compressed lz4);
CREATE TABLE
test=# create table t_pglz (v text);
CREATE TABLE
test=# insert into t_lz4 select repeat(md5(1::text),300);
INSERT 0 1
test=# insert into t_pglz select * from t_lz4;
INSERT 0 1
test=# drop extension pg_lz4 cascade;
NOTICE: drop cascades to 2 other objects
DETAIL: drop cascades to compression options for lz4
drop cascades to table t_lz4 column v
DROP EXTENSION
test=# \c test
You are now connected to database "test" as user "user".
test=# insert into t_lz4 select repeat(md5(1::text),300);^C
test=# select * from t_pglz ;
ERROR: cache lookup failed for compression options 16419That suggests no recompression happened.
I will check that. Is your extension published somewhere?
Re: [HACKERS] Custom compression methods
On 11/21/2017 09:28 PM, Ildus K wrote:
Hmmm, it still doesn't work for me. See this:
test=# create extension pg_lz4 ;
CREATE EXTENSION
test=# create table t_lz4 (v text compressed lz4);
CREATE TABLE
test=# create table t_pglz (v text);
CREATE TABLE
test=# insert into t_lz4 select repeat(md5(1::text),300);
INSERT 0 1
test=# insert into t_pglz select * from t_lz4;
INSERT 0 1
test=# drop extension pg_lz4 cascade;
NOTICE: drop cascades to 2 other objects
DETAIL: drop cascades to compression options for lz4
drop cascades to table t_lz4 column v
DROP EXTENSION
test=# \c test
You are now connected to database "test" as user "user".
test=# insert into t_lz4 select repeat(md5(1::text),300);^C
test=# select * from t_pglz ;
ERROR: cache lookup failed for compression options 16419That suggests no recompression happened.
I will check that. Is your extension published somewhere?
No, it was just an experiment, so I've only attached it to the initial
review. Attached is an updated version, with a fix or two.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
Re: [HACKERS] Custom compression methods
On Tue, 21 Nov 2017 18:47:49 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
Hmmm, it still doesn't work for me. See this:
test=# create extension pg_lz4 ;
CREATE EXTENSION
test=# create table t_lz4 (v text compressed lz4);
CREATE TABLE
test=# create table t_pglz (v text);
CREATE TABLE
test=# insert into t_lz4 select repeat(md5(1::text),300);
INSERT 0 1
test=# insert into t_pglz select * from t_lz4;
INSERT 0 1
test=# drop extension pg_lz4 cascade;
NOTICE: drop cascades to 2 other objects
DETAIL: drop cascades to compression options for lz4
drop cascades to table t_lz4 column v
DROP EXTENSION
test=# \c test
You are now connected to database "test" as user "user".
test=# insert into t_lz4 select repeat(md5(1::text),300);^C
test=# select * from t_pglz ;
ERROR: cache lookup failed for compression options 16419That suggests no recompression happened.
Should be fixed in the attached patch. I've changed your extension a
little bit according changes in the new patch (also in attachments).
Also I renamed few functions, added more comments and simplified the
code related with DefineRelation (thanks to Ildar Musin suggestion).
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v6.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 01acc2ef9d..f43f09cc19 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -59,6 +59,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 92db00f52d..207d6cf26e 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,8 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET NOT COMPRESSED
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +322,34 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. Setting a compression method doesn't change anything in the
+ table and affects only future table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>SET NOT COMPRESSED</literal>
+ </term>
+ <listitem>
+ <para>
+ This form removes compression from a column. Removing compresssion from
+ a column doesn't change already compressed tuples and affects only future
+ table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..663010ecd9
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,50 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="SQL-CREATECOMPRESSIONMETHOD">
+ <indexterm zone="sql-createcompressionmethod">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">compression_method_name</replaceable>
+ HANDLER <replaceable class="parameter">compression_method_handler</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method
+ with <replaceable class="parameter">compression_method_name</replaceable>.
+ </para>
+
+ <para>
+ A compression method links a name with a compression handler. And the
+ handler is a special function that returns collection of methods that
+ can be used for compression.
+ </para>
+
+ <para>
+ After a compression method is created, you can specify it in
+ <xref linkend="SQL-CREATETABLE"> or <xref linkend="SQL-ALTERTABLE">
+ statements.
+ </para>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3bc155a775..5e0c06421a 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index 9000b3aaaa..cc0bd70be3 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -87,6 +87,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 5c035fb203..0c6cf937f8 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a1a9d9905b..534c11edaa 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -230,10 +230,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +267,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +787,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1470,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..e5299d3094 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index aa9c0f1bb9..f740ce4304 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -945,11 +945,31 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca73a8..d206cce18e 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -242,6 +244,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -396,6 +399,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -458,6 +463,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -563,6 +569,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -675,7 +682,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 3acef279f4..af4c7b2e13 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2656,7 +2656,8 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ else if (HeapTupleHasExternal(tup) || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
return toast_insert_or_update(relation, tup, NULL, options);
else
return tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index c74945a52a..4dd87a5d8f 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -39,8 +41,11 @@
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +58,52 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+static HTAB *compression_options_cache = NULL;
+static MemoryContext compression_options_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ ((toast_compress_header *) (ptr))->info &= 0xC0000000; \
+ ((toast_compress_header *) (ptr))->info |= ((uint32)(len) & RAWSIZEMASK); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +121,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_compression_options_cache(void);
+static CompressionMethodOptions *get_cached_compression_options(Oid cmoptoid);
/* ----------
@@ -421,7 +461,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -686,8 +726,32 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
toast_oldexternal[i] = new_value;
if (att->attstorage == 'p')
new_value = heap_tuple_untoast_attr(new_value);
+ else if (VARATT_IS_EXTERNAL_ONDISK(new_value))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, new_value);
+
+ /*
+ * If we're trying to insert a custom compressed datum we
+ * should decompress it first.
+ */
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ new_value = heap_tuple_untoast_attr(new_value);
+ else
+ new_value = heap_tuple_fetch_attr(new_value);
+ }
else
new_value = heap_tuple_fetch_attr(new_value);
+
+ toast_values[i] = PointerGetDatum(new_value);
+ toast_free[i] = true;
+ need_change = true;
+ need_free = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(new_value))
+ {
+ new_value = heap_tuple_untoast_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +805,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +820,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +838,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +983,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1116,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1345,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1425,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,41 +1439,57 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoptoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize,
+ len = 0;
+ CompressionMethodOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ if (OidIsValid(cmoptoid))
+ cmoptions = get_cached_compression_options(cmoptoid);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ if (cmoptions)
+ {
+ tmp = cmoptions->routine->compress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ /*
+ * No point in wasting a palloc cycle if value size is out of the
+ * allowed range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return PointerGetDatum(NULL);
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!cmoptions && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,10 +1497,20 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (cmoptions && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, cmoptions->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
- pfree(tmp);
+ if (tmp)
+ pfree(tmp);
+
return PointerGetDatum(NULL);
}
}
@@ -1510,19 +1607,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1628,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1640,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2000,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2185,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2381,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionMethodOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = get_cached_compression_options(hdr->cmoptoid);
+ result = cmoptions->routine->decompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2502,78 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * init_compression_options_cache
+ *
+ * Initialize a local cache for compression options.
+ */
+static void
+init_compression_options_cache(void)
+{
+ HASHCTL ctl;
+
+ compression_options_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "compression options cache context",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionMethodOptions);
+ ctl.hcxt = compression_options_mcxt;
+ compression_options_cache = hash_create("compression options cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+}
+
+/* ----------
+ * get_cached_compression_options
+ *
+ * Remove cached compression options from the local cache
+ */
+static inline void
+remove_cached_compression_options(Oid cmoptoid)
+{
+ bool found;
+
+ hash_search(compression_options_cache, &cmoptoid, HASH_REMOVE, &found);
+}
+
+/* ----------
+ * get_cached_compression_options
+ *
+ * Get cached compression options structure or create it if it's not in cache.
+ * Cache is required because we can't afford for each tuple create
+ * CompressionMethodRoutine and parse its options.
+ */
+static CompressionMethodOptions *
+get_cached_compression_options(Oid cmoptoid)
+{
+ bool found;
+ CompressionMethodOptions *result;
+
+ Assert(OidIsValid(cmoptoid));
+ if (!compression_options_cache)
+ init_compression_options_cache();
+
+ /* check if the compression options wasn't removed from the last check */
+ found = SearchSysCacheExists(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(cmoptoid), 0, 0, 0);
+ if (!found)
+ {
+ remove_cached_compression_options(cmoptoid);
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+ }
+
+ result = hash_search(compression_options_cache, &cmoptoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ Assert(compression_options_mcxt);
+ oldcxt = MemoryContextSwitchTo(compression_options_mcxt);
+ result->cmoptoid = cmoptoid;
+ result->routine = GetCompressionMethodRoutine(cmoptoid);
+ result->options = GetCompressionOptionsList(cmoptoid);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ return result;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 8287de97a2..be6b460aee 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 033c4358ea..e1bfc7c6bf 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2512,6 +2524,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 256a9c9c93..c5838fa779 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -451,6 +451,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9e14880b99..4439f5df03 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1453,6 +1458,24 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+/*
+ * CallCompressionDropCallback
+ *
+ * Call drop callback from compression routine.
+ */
+static void
+CallCompressionDropCallback(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionMethodRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ cmr->drop(att, options);
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1506,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ CallCompressionDropCallback(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1606,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ CallCompressionDropCallback(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1629,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0125c18bc1..15942564aa 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8d55c76fc4..9fd1cb763d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index e02d312008..531a820464 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..5780926179
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,522 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/compression.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine *invoke_compression_handler(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+lookup_compression_handler(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+/* Creates a record in pg_compression */
+static ObjectAddress
+create_compression_method(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = lookup_compression_handler(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+invoke_compression_handler(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
+
+/*
+ * CREATE COMPRESSION METHOD .. HANDLER ..
+ */
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ /* Extract the name of compression handler function */
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ /* Finally create a compression method */
+ return create_compression_method(cmName, (List *) handlerEl->arg);
+}
+
+/*
+ * RemoveCompressionMethodById
+ *
+ * Removes a compresssion method by its Oid.
+ */
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, ColumnCompression *compression)
+{
+ Relation rel;
+ HeapTuple tup,
+ newtup;
+ Oid cmoptoid,
+ cmid;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+
+ ObjectAddress myself,
+ ref1,
+ ref2;
+
+ CompressionMethodRoutine *routine;
+ Form_pg_compression cmform;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmid = get_compression_method_oid(compression->methodName, false);
+
+ /* Get handler function OID for the compression method */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ routine = invoke_compression_handler(cmform->cmhandler, attr->atttypid);
+
+ if (routine->configure == NULL && compression->options != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("the compression method \"%s\" does not take any options",
+ NameStr(cmform->cmname))));
+ else if (routine->configure && compression->options != NIL)
+ routine->configure(attr, compression->options);
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ cmoptoid = GetNewOidWithIndex(rel, CompressionOptionsOidIndexId,
+ Anum_pg_compression_opt_cmoptoid);
+ values[Anum_pg_compression_opt_cmoptoid - 1] = ObjectIdGetDatum(cmoptoid);
+ values[Anum_pg_compression_opt_cmname - 1] = NameGetDatum(&cmform->cmname);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmform->cmhandler);
+
+ if (compression->options)
+ values[Anum_pg_compression_opt_cmoptions - 1] =
+ optionListToArray(compression->options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+
+ ReleaseSysCache(tup);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, cmoptoid);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmform->cmhandler);
+ ObjectAddressSet(ref2, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref2, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return cmoptoid;
+}
+
+/*
+ * Create pg_depend record between attribute and its compression options
+ */
+void
+CreateColumnCompressionDependency(Form_pg_attribute attr, Oid cmoptoid)
+{
+ ObjectAddress optref,
+ attrref;
+
+ ObjectAddressSet(optref, CompressionOptRelationId, cmoptoid);
+ ObjectAddressSubSet(attrref, RelationRelationId, attr->attrelid, attr->attnum);
+ recordDependencyOn(&attrref, &optref, DEPENDENCY_NORMAL);
+}
+
+/*
+ * Remove the compression options record from pg_compression_opt
+ */
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmoptform;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = pstrdup(NameStr(cmoptform->cmname));
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->methodName, c2->methodName))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->methodName, c2->methodName)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+ Form_pg_compression_opt cmoptform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmoptform->cmname));
+ ReleaseSysCache(tup);
+
+ return result;
+}
+
+/*
+ * GetCompressionOptionsList
+ *
+ * Parse array with compression options and return it as list.
+ */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetCompressionMethodRoutine
+ *
+ * Determine compression handler by compression options Oid and return
+ * structure with compression methods
+ */
+CompressionMethodRoutine *
+GetCompressionMethodRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return invoke_compression_handler(cmhandler, InvalidOid);
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 13eb9e34ba..f84588ed31 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2781,8 +2781,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index fa7d0d015a..a2eee78a64 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d19846d005..f5e3522551 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -459,6 +464,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -724,6 +731,12 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ /* Create compression options */
+ if (colDef->compression)
+ attr->attcompression = CreateCompressionOptions(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -771,6 +784,19 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * We should also create dependencies between attributes and their
+ * compression options
+ */
+ for (attnum = 0; attnum < (RelationGetDescr(rel))->natts; attnum++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), attnum);
+ if (OidIsValid(attr->attcompression))
+ CreateColumnCompressionDependency(attr, attr->attcompression);
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1610,6 +1636,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1944,6 +1971,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1971,6 +2011,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2180,6 +2223,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3279,6 +3329,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3770,6 +3821,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4118,6 +4175,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5338,6 +5400,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6438,6 +6502,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9361,6 +9430,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9410,7 +9481,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9432,6 +9505,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attbyval = tform->typbyval;
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ attTup->attcompression = InvalidOid;
ReleaseSysCache(typeTuple);
@@ -12484,6 +12558,82 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName)
+ {
+ /* SET COMPRESSED */
+ Oid cmoptoid;
+
+ cmoptoid = CreateCompressionOptions(atttableform, compression);
+ CreateColumnCompressionDependency(atttableform, cmoptoid);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c054..a7e9111c8d 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
diff --git a/src/backend/executor/execScan.c b/src/backend/executor/execScan.c
index 5dfc49deb9..3a80e997a9 100644
--- a/src/backend/executor/execScan.c
+++ b/src/backend/executor/execScan.c
@@ -302,6 +302,9 @@ tlist_matches_tupdesc(PlanState *ps, List *tlist, Index varno, TupleDesc tupdesc
var->vartypmod != -1))
return false; /* type mismatch */
+ if (OidIsValid(att_tup->attcompression))
+ return false;
+
tlist_item = lnext(tlist_item);
}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e48921fff1..1f14818d32 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2806,6 +2806,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2824,6 +2825,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5473,6 +5485,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2a83da9aa9..4eddc69602 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2544,6 +2544,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2562,6 +2563,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3615,6 +3625,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index c2a93b2d4c..81e334a1d3 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index c97ee24ade..72e88de08f 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2799,6 +2799,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2815,6 +2816,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4106,6 +4116,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c301ca465d..122d3de4d6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -397,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +583,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +617,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2168,6 +2171,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3332,11 +3344,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3346,8 +3359,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3394,6 +3407,37 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5754,6 +5798,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6262,6 +6315,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6325,7 +6379,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6515,6 +6569,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14704,6 +14759,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 8461da490a..39f21d4da7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -494,6 +495,39 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+/*
+ * transformColumnCompression
+ *
+ * Build ALTER TABLE .. SET COMPRESSED command if a compression method was
+ * specified for the column
+ */
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -794,6 +828,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1003,6 +1047,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 0f607bab70..f9eecf041b 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2878,7 +2878,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 82a707af7b..8c8c51dada 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1009,6 +1009,7 @@ ProcessUtilitySlow(ParseState *pstate,
RELKIND_RELATION,
InvalidOid, NULL,
queryString);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1283,6 +1284,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -2309,6 +2315,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2421,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index b0a9217d1e..fcff9d3fd6 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,85 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(CompressionMethodOptions *cmoptions, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+tsvector_decompress(CompressionMethodOptions *cmoptions, const struct varlena *data)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = NULL;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1908420d82..de834c5593 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 888edbb325..c689f60e47 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ Anum_pg_compression_opt_cmoptoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 4b47951de1..7ba6d31251 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
static DumpableObject **nspinfoindex;
static DumpableObject **extinfoindex;
+static DumpableObject **cminfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
@@ -61,6 +62,7 @@ static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
+static int numCompressionMethods;
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
@@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+ CompressionMethodInfo *cminfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr)
write_msg(NULL, "reading subscriptions\n");
getSubscriptions(fout);
+ if (g_verbose)
+ write_msg(NULL, "reading compression methods\n");
+ cminfo = getCompressionMethods(fout, &numCompressionMethods);
+ cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo));
+
*numTablesPtr = numTables;
return tblinfo;
}
@@ -827,6 +836,17 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
+/*
+ * findCompressionMethodByOid
+ * finds the entry (in cminfo) of the compression method with the given oid
+ * returns NULL if not found
+ */
+CompressionMethodInfo *
+findCompressionMethodByOid(Oid oid)
+{
+ return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex,
+ numCompressionMethods);
+}
/*
* setExtensionMembership
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..5c1aaad48e 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,8 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +152,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +174,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index ec2fa8b9b9..77eb55eb69 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_options_data = ropt->compression_options_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d8fb356130..6df688adb8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -361,6 +361,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -581,6 +582,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_options_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -785,6 +789,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_options_data)
+ getCompressionOptions(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -3957,6 +3964,205 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getCompressionMethods
+ * get information about compression methods
+ */
+CompressionMethodInfo *
+getCompressionMethods(Archive *fout, int *numMethods)
+{
+ DumpOptions *dopt = fout->dopt;
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionMethodInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i,
+ ntups;
+
+ if (dopt->no_compression_methods || fout->remoteVersion < 110000)
+ return NULL;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ /* Get the compression methods in current database. */
+ appendPQExpBuffer(query, "SELECT c.tableoid, c.oid, c.cmname, "
+ "c.cmhandler::pg_catalog.regproc as cmhandler "
+ "FROM pg_catalog.pg_compression c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "oid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_METHOD;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+
+ /* Decide whether we want to dump it */
+ selectDumpableObject(&(cminfo[i].dobj), fout);
+ }
+ if (numMethods)
+ *numMethods = ntups;
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+
+ return cminfo;
+}
+
+/*
+ * dumpCompressionMethod
+ * dump the definition of the given compression method
+ */
+static void
+dumpCompressionMethod(Archive *fout, CompressionMethodInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER",
+ fmtId(cminfo->dobj.name));
+ appendPQExpBuffer(query, " %s;\n", cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION METHOD", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * getCompressionOptions
+ * get information about compression options.
+ * this information should be migrated at binary upgrade
+ */
+void
+getCompressionOptions(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionOptionsInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i_options;
+ int i,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ appendPQExpBuffer(query,
+ "SELECT c.tableoid, c.cmoptoid, c.cmname,"
+ " c.cmhandler::pg_catalog.regproc as cmhandler, c.cmoptions "
+ "FROM pg_catalog.pg_compression_opt c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "cmoptoid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+ i_options = PQfnumber(res, "cmoptions");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionOptionsInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_OPTIONS;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].cmoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpCompressionOptions
+ * dump the given compression options
+ */
+static void
+dumpCompressionOptions(Archive *fout, CompressionOptionsInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ if (strlen(cminfo->cmoptions) > 0)
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), '%s');\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler,
+ cminfo->cmoptions);
+ else
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), NULL);\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION OPTIONS", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -4484,7 +4690,47 @@ getTypes(Archive *fout, int *numTypes)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- if (fout->remoteVersion >= 90600)
+ if (fout->remoteVersion >= 110000)
+ {
+ PQExpBuffer acl_subquery = createPQExpBuffer();
+ PQExpBuffer racl_subquery = createPQExpBuffer();
+ PQExpBuffer initacl_subquery = createPQExpBuffer();
+ PQExpBuffer initracl_subquery = createPQExpBuffer();
+
+ buildACLQueries(acl_subquery, racl_subquery, initacl_subquery,
+ initracl_subquery, "t.typacl", "t.typowner", "'T'",
+ dopt->binary_upgrade);
+
+ appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
+ "t.typnamespace, "
+ "%s AS typacl, "
+ "%s AS rtypacl, "
+ "%s AS inittypacl, "
+ "%s AS initrtypacl, "
+ "(%s t.typowner) AS rolname, "
+ "t.typelem, t.typrelid, "
+ "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" "
+ "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, "
+ "t.typtype, t.typisdefined, "
+ "t.typname[0] = '_' AND t.typelem != 0 AND "
+ "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray "
+ "FROM pg_type t "
+ "LEFT JOIN pg_init_privs pip ON "
+ "(t.oid = pip.objoid "
+ "AND pip.classoid = 'pg_type'::regclass "
+ "AND pip.objsubid = 0) ",
+ acl_subquery->data,
+ racl_subquery->data,
+ initacl_subquery->data,
+ initracl_subquery->data,
+ username_subquery);
+
+ destroyPQExpBuffer(acl_subquery);
+ destroyPQExpBuffer(racl_subquery);
+ destroyPQExpBuffer(initacl_subquery);
+ destroyPQExpBuffer(initracl_subquery);
+ }
+ else if (fout->remoteVersion >= 90600)
{
PQExpBuffer acl_subquery = createPQExpBuffer();
PQExpBuffer racl_subquery = createPQExpBuffer();
@@ -7895,6 +8141,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7930,7 +8178,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.cmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.cmname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.cmoptoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7949,9 +8236,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2 "
"ORDER BY a.attnum",
@@ -7975,7 +8266,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7999,7 +8292,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8017,7 +8312,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8034,7 +8331,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8064,6 +8363,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8080,6 +8381,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8107,6 +8410,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9596,6 +9901,12 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_COMPRESSION_METHOD:
+ dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj);
+ break;
+ case DO_COMPRESSION_OPTIONS:
+ dumpCompressionOptions(fout, (CompressionOptionsInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -15513,6 +15824,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSED %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17778,6 +18098,8 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_COMPRESSION_METHOD:
+ case DO_COMPRESSION_OPTIONS:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da884ffd09..88b18ba15c 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,9 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_COMPRESSION_METHOD,
+ DO_COMPRESSION_OPTIONS
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -315,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +615,21 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+/* The CompressionMethodInfo struct is used to represent compression method */
+typedef struct _CompressionMethodInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+} CompressionMethodInfo;
+
+/* The CompressionOptionsInfo struct is used to represent compression options */
+typedef struct _CompressionOptionsInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+ char *cmoptions;
+} CompressionOptionsInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -654,6 +673,7 @@ extern OprInfo *findOprByOid(Oid oid);
extern CollInfo *findCollationByOid(Oid oid);
extern NamespaceInfo *findNamespaceByOid(Oid oid);
extern ExtensionInfo *findExtensionByOid(Oid oid);
+extern CompressionMethodInfo * findCompressionMethodByOid(Oid oid);
extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems);
extern ExtensionInfo *findOwningExtension(CatalogId catalogId);
@@ -711,5 +731,8 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern CompressionMethodInfo * getCompressionMethods(Archive *fout,
+ int *numMethods);
+void getCompressionOptions(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 48b6dd594c..f5db0280e2 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -80,7 +80,9 @@ static const int dbObjectTypePriority[] =
34, /* DO_POLICY */
35, /* DO_PUBLICATION */
36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 37, /* DO_SUBSCRIPTION */
+ 17, /* DO_COMPRESSION_METHOD */
+ 17 /* DO_COMPRESSION_OPTIONS */
};
static DumpId preDataBoundId;
@@ -1436,6 +1438,16 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_COMPRESSION_METHOD:
+ snprintf(buf, bufsize,
+ "COMPRESSION METHOD %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
+ case DO_COMPRESSION_OPTIONS:
+ snprintf(buf, bufsize,
+ "COMPRESSION OPTIONS %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..6b72ccf9ea 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 8cc4de3878..8b0dbfa45c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index b7b978a361..49160ded0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1716,6 +1779,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " WHERE c.cmoptoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1830,6 +1909,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1925,6 +2008,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1932,7 +2020,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1943,7 +2031,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a926c40b9b..25d2fbc125 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b3e3799c13..a323ca80c6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..bb4e299447
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,68 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ Oid cmoptoid;
+ CompressionMethodRoutine *routine;
+ List *options;
+} CompressionMethodOptions;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (CompressionMethodOptions *cmoptions, const struct varlena *data);
+
+/*
+ * API struct for a compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionConfigureRoutine drop;
+ CompressionRoutine compress;
+ CompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+/* Compression method handler parameters */
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine *GetCompressionMethodRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr,
+ ColumnCompression *compression);
+extern ColumnCompression *GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+extern void CreateColumnCompressionDependency(Form_pg_attribute attr,
+ Oid cmoptoid);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index b0d4c54121..7ff4cafacc 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contain custom compressed
+ * varlenas */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index cd43e3a52e..2e1c92123e 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1d3e..1a8c526ca4 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..115889c8f5 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -210,7 +219,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..7fe9f1f059 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(cmoptoid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..40f5cc4f18 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..ddcef814a7
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420) BKI_WITHOUT_OIDS
+{
+ Oid cmoptoid; /* compression options oid */
+ NameData cmname; /* name of compression method */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 4
+#define Anum_pg_compression_opt_cmoptoid 1
+#define Anum_pg_compression_opt_cmname 2
+#define Anum_pg_compression_opt_cmhandler 3
+#define Anum_pg_compression_opt_cmoptions 4
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index c969375981..dc3fa66f2e 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3881,6 +3881,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4677,6 +4681,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e3551440a0..ec8c3df953 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bfead9af3d..a98ecc12ce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ffeeb4919b..6dc49a73b6 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -468,6 +468,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -498,7 +499,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 80c19b2a55..56e42ce1f1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSED <compression_method_name> WITH (<params>)
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +651,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1622,6 +1636,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1769,7 +1784,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..d21974696f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 254a811aff..6ad889af7a 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -210,6 +210,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a0be41929..ff7cb530fd 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..119fac595a
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,154 @@
+-- test drop
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE droptest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table droptest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 2 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to table droptest column fts
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+-- check ALTER commands
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method "ts1" does not take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+-- create different types of tables
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest);
+NOTICE: merging column "fts" with inherited definition
+CREATE TABLE cmtest5(fts tsvector);
+CREATE TABLE cmtest6(fts tsvector);
+INSERT INTO cmtest6 SELECT * FROM cmtest;
+-- we update usual datum with compressed datum
+INSERT INTO cmtest5
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest;
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+DROP TABLE cmtest3;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 4 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+-----------
+(0 rows)
+
+SELECT * FROM pg_compression_opt;
+ cmoptoid | cmname | cmhandler | cmoptions
+----------+--------+-----------+-----------
+(0 rows)
+
+-- check that moved tuples still can be decompressed
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest5;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest6;
+ length
+--------
+ 200
+(1 row)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest5;
+DROP TABLE cmtest6;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 335cd37e18..b511daf9fa 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -590,10 +590,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -729,11 +729,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -756,11 +756,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -793,46 +793,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..2ac75c44b5 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 331f7a911f..530ce1b1d0 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index c698faff2f..a147b8217f 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 9d84ba4658..bbab453247 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -435,10 +435,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -750,74 +750,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index e996640593..62b06da011 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index a4fe96112e..adbe764196 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 1a3ac4c1f9..a69c485e8c 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index a205e5d05c..e72015a391 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..6bf0ea8267
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,62 @@
+-- test drop
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE droptest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+DROP COMPRESSION METHOD ts1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+
+SELECT length(fts) FROM cmtest;
+SELECT length(fts) FROM cmtest;
+
+-- check ALTER commands
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+-- create different types of tables
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest);
+CREATE TABLE cmtest5(fts tsvector);
+CREATE TABLE cmtest6(fts tsvector);
+INSERT INTO cmtest6 SELECT * FROM cmtest;
+
+-- we update usual datum with compressed datum
+INSERT INTO cmtest5
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest;
+
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+DROP TABLE cmtest3;
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
+SELECT * FROM pg_compression;
+SELECT * FROM pg_compression_opt;
+
+-- check that moved tuples still can be decompressed
+SELECT length(fts) FROM cmtest2;
+SELECT length(fts) FROM cmtest5;
+SELECT length(fts) FROM cmtest6;
+DROP TABLE cmtest2;
+DROP TABLE cmtest5;
+DROP TABLE cmtest6;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b422050a92..2fe9d7d434 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -340,6 +340,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -363,6 +364,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionMethodOptions
+CompressionMethodRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Hi,
On 11/23/2017 10:38 AM, Ildus Kurbangaliev wrote:
On Tue, 21 Nov 2017 18:47:49 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:Hmmm, it still doesn't work for me. See this:
test=# create extension pg_lz4 ;
CREATE EXTENSION
test=# create table t_lz4 (v text compressed lz4);
CREATE TABLE
test=# create table t_pglz (v text);
CREATE TABLE
test=# insert into t_lz4 select repeat(md5(1::text),300);
INSERT 0 1
test=# insert into t_pglz select * from t_lz4;
INSERT 0 1
test=# drop extension pg_lz4 cascade;
NOTICE: drop cascades to 2 other objects
DETAIL: drop cascades to compression options for lz4
drop cascades to table t_lz4 column v
DROP EXTENSION
test=# \c test
You are now connected to database "test" as user "user".
test=# insert into t_lz4 select repeat(md5(1::text),300);^C
test=# select * from t_pglz ;
ERROR: cache lookup failed for compression options 16419That suggests no recompression happened.
Should be fixed in the attached patch. I've changed your extension a
little bit according changes in the new patch (also in attachments).
Hmm, this seems to have fixed it, but only in one direction. Consider this:
create table t_pglz (v text);
create table t_lz4 (v text compressed lz4);
insert into t_pglz select repeat(md5(i::text),300)
from generate_series(1,100000) s(i);
insert into t_lz4 select repeat(md5(i::text),300)
from generate_series(1,100000) s(i);
\d+
Schema | Name | Type | Owner | Size | Description
--------+--------+-------+-------+-------+-------------
public | t_lz4 | table | user | 12 MB |
public | t_pglz | table | user | 18 MB |
(2 rows)
truncate t_pglz;
insert into t_pglz select * from t_lz4;
\d+
Schema | Name | Type | Owner | Size | Description
--------+--------+-------+-------+-------+-------------
public | t_lz4 | table | user | 12 MB |
public | t_pglz | table | user | 18 MB |
(2 rows)
which is fine. But in the other direction, this happens
truncate t_lz4;
insert into t_lz4 select * from t_pglz;
\d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+--------+-------+-------+-------+-------------
public | t_lz4 | table | user | 18 MB |
public | t_pglz | table | user | 18 MB |
(2 rows)
which means the data is still pglz-compressed. That's rather strange, I
guess, and it should compress the data using the compression method set
for the target table instead.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Thu, 23 Nov 2017 21:54:32 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
Hmm, this seems to have fixed it, but only in one direction. Consider
this:create table t_pglz (v text);
create table t_lz4 (v text compressed lz4);insert into t_pglz select repeat(md5(i::text),300)
from generate_series(1,100000) s(i);insert into t_lz4 select repeat(md5(i::text),300)
from generate_series(1,100000) s(i);\d+
Schema | Name | Type | Owner | Size | Description
--------+--------+-------+-------+-------+-------------
public | t_lz4 | table | user | 12 MB |
public | t_pglz | table | user | 18 MB |
(2 rows)truncate t_pglz;
insert into t_pglz select * from t_lz4;\d+
Schema | Name | Type | Owner | Size | Description
--------+--------+-------+-------+-------+-------------
public | t_lz4 | table | user | 12 MB |
public | t_pglz | table | user | 18 MB |
(2 rows)which is fine. But in the other direction, this happens
truncate t_lz4;
insert into t_lz4 select * from t_pglz;\d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+--------+-------+-------+-------+-------------
public | t_lz4 | table | user | 18 MB |
public | t_pglz | table | user | 18 MB |
(2 rows)which means the data is still pglz-compressed. That's rather strange,
I guess, and it should compress the data using the compression method
set for the target table instead.
That's actually an interesting issue. It happens because if tuple fits
to page then postgres just moves it as is. I've just added
recompression if it has custom compressed datums to keep dependencies
right. But look:
create table t1(a text);
create table t2(a text);
alter table t2 alter column a set storage external;
insert into t1 select repeat(md5(i::text),300) from
generate_series(1,100000) s(i);
\d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+------+-------+-------+------------+-------------
public | t1 | table | ildus | 18 MB |
public | t2 | table | ildus | 8192 bytes |
(2 rows)
insert into t2 select * from t1;
\d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+------+-------+-------+-------+-------------
public | t1 | table | ildus | 18 MB |
public | t2 | table | ildus | 18 MB |
(2 rows)
That means compressed datums now in the column with storage specified as
external. I'm not sure that's a bug or a feature. Lets insert them
usual way:
delete from t2;
insert into t2 select repeat(md5(i::text),300) from
generate_series(1,100000) s(i);
\d+
List of relations
Schema | Name | Type | Owner | Size | Description
--------+------+-------+-------+---------+-------------
public | t1 | table | ildus | 18 MB |
public | t2 | table | ildus | 1011 MB |
Maybe there should be more common solution like comparison of attribute
properties?
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
Hi,
On 11/24/2017 10:38 AM, Ildus Kurbangaliev wrote:
...
That means compressed datums now in the column with storage
specified as external. I'm not sure that's a bug or a feature.
Interesting. Never realized it behaves like this. Not sure if it's
intentional or not (i.e. bug vs. feature).
Lets insert them usual way:
delete from t2;
insert into t2 select repeat(md5(i::text),300) from
generate_series(1,100000) s(i);
\d+List of relations
Schema | Name | Type | Owner | Size | Description
--------+------+-------+-------+---------+-------------
public | t1 | table | ildus | 18 MB |
public | t2 | table | ildus | 1011 MB |Maybe there should be more common solution like comparison of
attribute properties?
Maybe, not sure what the right solution is. I just know that if we allow
inserting data into arbitrary tables without recompression, we may end
up with data that can't be decompressed.
I agree that the behavior with extended storage is somewhat similar, but
the important distinction is that while that is surprising the data is
still accessible.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
Hi,
I ran into another issue - after inserting some data into a table with a
tsvector column (without any compression defined), I can no longer read
the data.
This is what I get in the console:
db=# select max(md5(body_tsvector::text)) from messages;
ERROR: cache lookup failed for compression options 6432
and the stack trace looks like this:
Breakpoint 1, get_cached_compression_options (cmoptoid=6432) at
tuptoaster.c:2563
2563 elog(ERROR, "cache lookup failed for compression options %u",
cmoptoid);
(gdb) bt
#0 get_cached_compression_options (cmoptoid=6432) at tuptoaster.c:2563
#1 0x00000000004bf3da in toast_decompress_datum (attr=0x2b44148) at
tuptoaster.c:2390
#2 0x00000000004c0c1e in heap_tuple_untoast_attr (attr=0x2b44148) at
tuptoaster.c:225
#3 0x000000000083f976 in pg_detoast_datum (datum=<optimized out>) at
fmgr.c:1829
#4 0x00000000008072de in tsvectorout (fcinfo=0x2b41e00) at tsvector.c:315
#5 0x00000000005fae00 in ExecInterpExpr (state=0x2b414b8,
econtext=0x2b25ab0, isnull=<optimized out>) at execExprInterp.c:1131
#6 0x000000000060bdf4 in ExecEvalExprSwitchContext
(isNull=0x7fffffe9bd37 "", econtext=0x2b25ab0, state=0x2b414b8) at
../../../src/include/executor/executor.h:299
It seems the VARATT_IS_CUSTOM_COMPRESSED incorrectly identifies the
value as custom-compressed for some reason.
Not sure why, but the tsvector column is populated by a trigger that
simply does
NEW.body_tsvector
:= to_tsvector('english', strip_replies(NEW.body_plain));
If needed, the complete tool is here:
https://bitbucket.org/tvondra/archie
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Sat, 25 Nov 2017 06:40:00 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
Hi,
I ran into another issue - after inserting some data into a table
with a tsvector column (without any compression defined), I can no
longer read the data.This is what I get in the console:
db=# select max(md5(body_tsvector::text)) from messages;
ERROR: cache lookup failed for compression options 6432and the stack trace looks like this:
Breakpoint 1, get_cached_compression_options (cmoptoid=6432) at
tuptoaster.c:2563
2563 elog(ERROR, "cache lookup failed for
compression options %u", cmoptoid);
(gdb) bt
#0 get_cached_compression_options (cmoptoid=6432) at
tuptoaster.c:2563 #1 0x00000000004bf3da in toast_decompress_datum
(attr=0x2b44148) at tuptoaster.c:2390
#2 0x00000000004c0c1e in heap_tuple_untoast_attr (attr=0x2b44148) at
tuptoaster.c:225
#3 0x000000000083f976 in pg_detoast_datum (datum=<optimized out>) at
fmgr.c:1829
#4 0x00000000008072de in tsvectorout (fcinfo=0x2b41e00) at
tsvector.c:315 #5 0x00000000005fae00 in ExecInterpExpr
(state=0x2b414b8, econtext=0x2b25ab0, isnull=<optimized out>) at
execExprInterp.c:1131 #6 0x000000000060bdf4 in
ExecEvalExprSwitchContext (isNull=0x7fffffe9bd37 "",
econtext=0x2b25ab0, state=0x2b414b8)
at ../../../src/include/executor/executor.h:299It seems the VARATT_IS_CUSTOM_COMPRESSED incorrectly identifies the
value as custom-compressed for some reason.Not sure why, but the tsvector column is populated by a trigger that
simply doesNEW.body_tsvector
:= to_tsvector('english', strip_replies(NEW.body_plain));If needed, the complete tool is here:
Hi. This looks like a serious bug, but I couldn't reproduce
it yet. Did you upgrade some old database or this bug happened after
insertion of all data to new database? I tried using your 'archie'
tool to download mailing lists and insert them to database, but couldn't
catch any errors.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
Hi,
On 11/27/2017 04:52 PM, Ildus Kurbangaliev wrote:
...
Hi. This looks like a serious bug, but I couldn't reproduce it yet.
Did you upgrade some old database or this bug happened after
insertion of all data to new database? I tried using your 'archie'
tool to download mailing lists and insert them to database, but
couldn't catch any errors.
I can trigger it pretty reliably with these steps:
git checkout f65d21b258085bdc8ef2cc282ab1ff12da9c595c
patch -p1 < ~/custom_compression_methods_v6.patch
./configure --enable-debug --enable-cassert \
CFLAGS="-fno-omit-frame-pointer -O0 -DRANDOMIZE_ALLOCATED_MEMORY" \
--prefix=/home/postgres/pg-compress
make -s clean && make -s -j4 install
cd contrib/
make -s clean && make -s -j4 install
export PATH=/home/postgres/pg-compress/bin:$PATH
pg_ctl -D /mnt/raid/pg-compress init
pg_ctl -D /mnt/raid/pg-compress -l compress.log start
createdb archie
cd ~/archie/sql/
psql archie < create.sql
~/archie/bin/load.py --workers 4 --db archie */* > load.log 2>&1
I guess the trick might be -DRANDOMIZE_ALLOCATED_MEMORY (I first tried
without it, and it seemed working fine). If that's the case, I bet there
is a palloc that should have been palloc0, or something like that.
If you still can't reproduce that, I may give you access to this machine
so that you can debug it there.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, 27 Nov 2017 18:20:12 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
I guess the trick might be -DRANDOMIZE_ALLOCATED_MEMORY (I first tried
without it, and it seemed working fine). If that's the case, I bet
there is a palloc that should have been palloc0, or something like
that.
Thanks, that was it. I've been able to reproduce this bug. The attached
patch should fix this bug and I've also added recompression when
tuples moved to the relation with the compressed attribute.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v7.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..766ced401f 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 01acc2ef9d..f43f09cc19 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -59,6 +59,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 7bcf242846..e44f9eb94b 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,8 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET NOT COMPRESSED
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +322,34 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSED <replaceable class="parameter">compression_method_name</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. Setting a compression method doesn't change anything in the
+ table and affects only future table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term>
+ <literal>SET NOT COMPRESSED</literal>
+ </term>
+ <listitem>
+ <para>
+ This form removes compression from a column. Removing compresssion from
+ a column doesn't change already compressed tuples and affects only future
+ table updates.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..663010ecd9
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,50 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="SQL-CREATECOMPRESSIONMETHOD">
+ <indexterm zone="sql-createcompressionmethod">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">compression_method_name</replaceable>
+ HANDLER <replaceable class="parameter">compression_method_handler</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method
+ with <replaceable class="parameter">compression_method_name</replaceable>.
+ </para>
+
+ <para>
+ A compression method links a name with a compression handler. And the
+ handler is a special function that returns collection of methods that
+ can be used for compression.
+ </para>
+
+ <para>
+ After a compression method is created, you can specify it in
+ <xref linkend="SQL-CREATETABLE"> or <xref linkend="SQL-ALTERTABLE">
+ statements.
+ </para>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index a0c9a6d257..1eae358c8a 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSED <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method should be
+ created with <xref linkend="SQL-CREATECOMPRESSIONMETHOD">. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index d20eaa87e7..167090639f 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -87,6 +87,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 5c035fb203..0c6cf937f8 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a1a9d9905b..aa764b7b59 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 138671410a..e5299d3094 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index aa9c0f1bb9..f740ce4304 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -945,11 +945,31 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 9e37ca73a8..06dcf219bf 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,8 +19,10 @@
#include "postgres.h"
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -64,6 +66,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -86,8 +89,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about it
+ * to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -118,6 +130,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -180,6 +193,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -242,6 +256,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -346,6 +361,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -396,6 +413,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -458,6 +477,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -563,6 +583,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -675,7 +696,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 3acef279f4..7615ad310f 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2656,7 +2656,10 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
return toast_insert_or_update(relation, tup, NULL, options);
else
return tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index c74945a52a..b9a610fb6e 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,8 +30,10 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -39,8 +41,11 @@
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +58,57 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmoptoid; /* Oid from pg_compression_opt */
+} toast_compress_header_custom;
+
+static HTAB *compression_options_cache = NULL;
+static MemoryContext compression_options_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMOPTOID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoptoid = (oid))
+
+/*
+ * Marks varlena as custom compressed. Notice that this macro should be called
+ * after TOAST_COMPRESS_SET_RAWSIZE because it will clean flags.
+ */
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +126,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_compression_options_cache(void);
+static CompressionMethodOptions *get_cached_compression_options(Oid cmoptoid);
/* ----------
@@ -421,7 +466,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -678,21 +723,47 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* still in the tuple must be someone else's that we cannot reuse
* (this includes the case of an out-of-line in-memory datum).
* Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * PLAIN storage or it has a custom compression). If necessary,
+ * we'll push it out as a new external value below.
*/
if (VARATT_IS_EXTERNAL(new_value))
{
toast_oldexternal[i] = new_value;
if (att->attstorage == 'p')
new_value = heap_tuple_untoast_attr(new_value);
+ else if (VARATT_IS_EXTERNAL_ONDISK(new_value))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, new_value);
+
+ /*
+ * If we're trying to insert a custom compressed datum we
+ * should decompress it first.
+ */
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ new_value = heap_tuple_untoast_attr(new_value);
+ else
+ new_value = heap_tuple_fetch_attr(new_value);
+ }
else
new_value = heap_tuple_fetch_attr(new_value);
+
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
need_free = true;
}
+ else if (VARATT_IS_CUSTOM_COMPRESSED(new_value)
+ || OidIsValid(att->attcompression))
+ {
+ struct varlena *untoasted_value = heap_tuple_untoast_attr(new_value);
+
+ toast_values[i] = PointerGetDatum(untoasted_value);
+ need_free = (untoasted_value != new_value);
+ toast_free[i] = need_free;
+ need_change = need_free;
+ }
/*
* Remember the size of this attribute
@@ -741,12 +812,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +827,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +845,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +990,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1123,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1352,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1432,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,41 +1446,57 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoptoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize,
+ len = 0;
+ CompressionMethodOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ if (OidIsValid(cmoptoid))
+ cmoptions = get_cached_compression_options(cmoptoid);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (cmoptions)
+ {
+ tmp = cmoptions->routine->compress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
+ }
+ else
+ {
+ /*
+ * No point in wasting a palloc cycle if value size is out of the
+ * allowed range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return PointerGetDatum(NULL);
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ PGLZ_strategy_default);
+ }
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
+ if (!cmoptions && len >= 0 &&
len + TOAST_COMPRESS_HDRSZ < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
@@ -1410,10 +1504,20 @@ toast_compress_datum(Datum value)
/* successful compression */
return PointerGetDatum(tmp);
}
+ else if (cmoptions && VARSIZE(tmp) < valsize - 2)
+ {
+ TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_CMOPTOID(tmp, cmoptions->cmoptoid);
+ /* successful compression */
+ return PointerGetDatum(tmp);
+ }
else
{
/* incompressible data */
- pfree(tmp);
+ if (tmp)
+ pfree(tmp);
+
return PointerGetDatum(NULL);
}
}
@@ -1510,19 +1614,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1635,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1647,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2007,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2192,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2388,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionMethodOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = get_cached_compression_options(hdr->cmoptoid);
+ result = cmoptions->routine->decompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2509,78 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * init_compression_options_cache
+ *
+ * Initialize a local cache for compression options.
+ */
+static void
+init_compression_options_cache(void)
+{
+ HASHCTL ctl;
+
+ compression_options_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "compression options cache context",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionMethodOptions);
+ ctl.hcxt = compression_options_mcxt;
+ compression_options_cache = hash_create("compression options cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+}
+
+/* ----------
+ * get_cached_compression_options
+ *
+ * Remove cached compression options from the local cache
+ */
+static inline void
+remove_cached_compression_options(Oid cmoptoid)
+{
+ bool found;
+
+ hash_search(compression_options_cache, &cmoptoid, HASH_REMOVE, &found);
+}
+
+/* ----------
+ * get_cached_compression_options
+ *
+ * Get cached compression options structure or create it if it's not in cache.
+ * Cache is required because we can't afford for each tuple create
+ * CompressionMethodRoutine and parse its options.
+ */
+static CompressionMethodOptions *
+get_cached_compression_options(Oid cmoptoid)
+{
+ bool found;
+ CompressionMethodOptions *result;
+
+ Assert(OidIsValid(cmoptoid));
+ if (!compression_options_cache)
+ init_compression_options_cache();
+
+ /* check if the compression options wasn't removed from the last check */
+ found = SearchSysCacheExists(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(cmoptoid), 0, 0, 0);
+ if (!found)
+ {
+ remove_cached_compression_options(cmoptoid);
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+ }
+
+ result = hash_search(compression_options_cache, &cmoptoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ Assert(compression_options_mcxt);
+ oldcxt = MemoryContextSwitchTo(compression_options_mcxt);
+ result->cmoptoid = cmoptoid;
+ result->routine = GetCompressionMethodRoutine(cmoptoid);
+ result->options = GetCompressionOptionsList(cmoptoid);
+ MemoryContextSwitchTo(oldcxt);
+ }
+ return result;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 8287de97a2..be6b460aee 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index fd33426bad..c7cea974b1 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -46,7 +46,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_collation.h pg_partitioned_table.h pg_range.h pg_transform.h \
pg_sequence.h pg_publication.h pg_publication_rel.h pg_subscription.h \
pg_subscription_rel.h toasting.h indexing.h \
- toasting.h indexing.h \
+ pg_compression.h pg_compression_opt.h \
)
# location of Catalog.pm
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index ccde66a7dd..fd733a34a0 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3340,6 +3340,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 033c4358ea..e1bfc7c6bf 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -28,6 +28,8 @@
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_collation_fn.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -173,7 +175,9 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionMethodRelationId, /* OCLASS_COMPRESSION_METHOD */
+ CompressionOptRelationId, /* OCLASS_COMPRESSION_OPTIONS */
};
@@ -1271,6 +1275,14 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_COMPRESSION_METHOD:
+ RemoveCompressionMethodById(object->objectId);
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ RemoveCompressionOptionsById(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2512,6 +2524,12 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionMethodRelationId:
+ return OCLASS_COMPRESSION_METHOD;
+
+ case CompressionOptRelationId:
+ return OCLASS_COMPRESSION_OPTIONS;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 256a9c9c93..c5838fa779 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -451,6 +451,7 @@ sub emit_pgattr_row
attisdropped => 'f',
attislocal => 't',
attinhcount => '0',
+ attcompression=> '0',
attacl => '_null_',
attoptions => '_null_',
attfdwoptions => '_null_');
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4319fc6b8c..8c1cc48060 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,8 +29,10 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -44,6 +46,8 @@
#include "catalog/partition.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_foreign_table.h"
@@ -628,6 +632,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1453,6 +1458,24 @@ DeleteRelationTuple(Oid relid)
heap_close(pg_class_desc, RowExclusiveLock);
}
+/*
+ * CallCompressionDropCallback
+ *
+ * Call drop callback from compression routine.
+ */
+static void
+CallCompressionDropCallback(Form_pg_attribute att)
+{
+ CompressionMethodRoutine *cmr = GetCompressionMethodRoutine(att->attcompression);
+
+ if (cmr->drop)
+ {
+ List *options = GetCompressionOptionsList(att->attcompression);
+
+ cmr->drop(att, options);
+ }
+}
+
/*
* DeleteAttributeTuples
*
@@ -1483,7 +1506,14 @@ DeleteAttributeTuples(Oid relid)
/* Delete all the matching tuples */
while ((atttup = systable_getnext(scan)) != NULL)
+ {
+ Form_pg_attribute att = (Form_pg_attribute) GETSTRUCT(atttup);
+
+ if (OidIsValid(att->attcompression))
+ CallCompressionDropCallback(att);
+
CatalogTupleDelete(attrel, &atttup->t_self);
+ }
/* Clean up after the scan */
systable_endscan(scan);
@@ -1576,6 +1606,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
else
{
/* Dropping user attributes is lots harder */
+ if (OidIsValid(attStruct->attcompression))
+ CallCompressionDropCallback(attStruct);
/* Mark the attribute as dropped */
attStruct->attisdropped = true;
@@ -1597,6 +1629,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0125c18bc1..15942564aa 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8d55c76fc4..9fd1cb763d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -29,6 +29,8 @@
#include "catalog/pg_default_acl.h"
#include "catalog/pg_event_trigger.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_conversion.h"
@@ -490,6 +492,30 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ CompressionMethodRelationId,
+ CompressionMethodOidIndexId,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
+ {
+ CompressionOptRelationId,
+ CompressionOptionsOidIndexId,
+ COMPRESSIONOPTIONSOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false,
}
};
@@ -712,6 +738,10 @@ static const struct object_type_map
/* OBJECT_STATISTIC_EXT */
{
"statistics object", OBJECT_STATISTIC_EXT
+ },
+ /* OCLASS_COMPRESSION_METHOD */
+ {
+ "compression method", OBJECT_COMPRESSION_METHOD
}
};
@@ -876,6 +906,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_ACCESS_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
address = get_object_address_unqualified(objtype,
(Value *) object, missing_ok);
break;
@@ -1182,6 +1213,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_subscription_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionMethodRelationId;
+ address.objectId = get_compression_method_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
default:
elog(ERROR, "unrecognized objtype: %d", (int) objtype);
/* placate compiler, which doesn't know elog won't return */
@@ -2139,6 +2175,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
case OBJECT_SCHEMA:
case OBJECT_SUBSCRIPTION:
case OBJECT_TABLESPACE:
+ case OBJECT_COMPRESSION_METHOD:
if (list_length(name) != 1)
ereport(ERROR,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
@@ -2395,12 +2432,14 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3398,6 +3437,27 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *name = get_compression_method_name(object->objectId);
+
+ if (!name)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfo(&buffer, _("compression method %s"), name);
+ pfree(name);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ char *name = get_compression_method_name_for_opt(object->objectId);
+
+ appendStringInfo(&buffer, _("compression options for %s"), name);
+ pfree(name);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3919,6 +3979,14 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION_METHOD:
+ appendStringInfoString(&buffer, "compression method");
+ break;
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ appendStringInfoString(&buffer, "compression options");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4160,6 +4228,30 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_COMPRESSION_METHOD:
+ {
+ char *cmname = get_compression_method_name(object->objectId);
+
+ if (!cmname)
+ elog(ERROR, "cache lookup failed for compression method %u",
+ object->objectId);
+ appendStringInfoString(&buffer, quote_identifier(cmname));
+ if (objname)
+ *objname = list_make1(cmname);
+ else
+ pfree(cmname);
+ break;
+ }
+
+ case OCLASS_COMPRESSION_OPTIONS:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_CONSTRAINT:
{
HeapTuple conTup;
diff --git a/src/backend/catalog/pg_type.c b/src/backend/catalog/pg_type.c
index e02d312008..531a820464 100644
--- a/src/backend/catalog/pg_type.c
+++ b/src/backend/catalog/pg_type.c
@@ -22,6 +22,7 @@
#include "catalog/indexing.h"
#include "catalog/objectaccess.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_proc.h"
#include "catalog/pg_type.h"
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 4f8147907c..4f18e4083f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -385,6 +385,7 @@ ExecRenameStmt(RenameStmt *stmt)
case OBJECT_TSTEMPLATE:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
+ case OBJECT_COMPRESSION_METHOD:
{
ObjectAddress address;
Relation catalog;
@@ -500,6 +501,7 @@ ExecAlterObjectSchemaStmt(AlterObjectSchemaStmt *stmt,
case OBJECT_TSDICTIONARY:
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
+ case OBJECT_COMPRESSION_METHOD:
{
Relation catalog;
Relation relation;
@@ -627,6 +629,8 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..5780926179
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,522 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands that manipulate compression methods.
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/compression.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+
+static CompressionMethodRoutine *invoke_compression_handler(Oid cmhandler, Oid typeid);
+
+/*
+ * Convert a handler function name to an Oid. If the return type of the
+ * function doesn't match the given AM type, an error is raised.
+ *
+ * This function either return valid function Oid or throw an error.
+ */
+static Oid
+lookup_compression_handler(List *handlerName)
+{
+ static const Oid funcargtypes[1] = {INTERNALOID};
+ Oid handlerOid;
+
+ if (handlerName == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* handlers have one argument of type internal */
+ handlerOid = LookupFuncName(handlerName, 1, funcargtypes, false);
+
+ /* check that handler has the correct return type */
+ if (get_func_rettype(handlerOid) != COMPRESSION_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handlerName),
+ "compression_handler")));
+
+ return handlerOid;
+}
+
+/* Creates a record in pg_compression */
+static ObjectAddress
+create_compression_method(char *cmName, List *handlerName)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+
+ rel = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(COMPRESSIONMETHODNAME, CStringGetDatum(cmName));
+ if (OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ cmName)));
+
+ /*
+ * Get the handler function oid and compression method routine
+ */
+ cmhandler = lookup_compression_handler(handlerName);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(cmName));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ cmoid = CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ ObjectAddressSet(myself, CompressionMethodRelationId, cmoid);
+
+ /* Record dependency on handler function */
+ ObjectAddressSet(referenced, ProcedureRelationId, cmhandler);
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ heap_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+/*
+ * Call the specified compression method handler
+ * routine to get its CompressionMethodRoutine struct,
+ * which will be palloc'd in the caller's context.
+ */
+static CompressionMethodRoutine *
+invoke_compression_handler(Oid cmhandler, Oid typeid)
+{
+ Datum datum;
+ CompressionMethodRoutine *routine;
+ CompressionMethodOpArgs opargs;
+
+ opargs.typeid = typeid;
+ opargs.cmhanderid = cmhandler;
+
+ datum = OidFunctionCall1(cmhandler, PointerGetDatum(&opargs));
+ routine = (CompressionMethodRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionMethodRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionMethodRoutine struct",
+ cmhandler);
+
+ return routine;
+}
+
+/*
+ * CREATE COMPRESSION METHOD .. HANDLER ..
+ */
+ObjectAddress
+DefineCompressionMethod(List *names, List *parameters)
+{
+ char *cmName;
+ ListCell *pl;
+ DefElem *handlerEl = NULL;
+
+ if (list_length(names) != 1)
+ elog(ERROR, "compression method name cannot be qualified");
+
+ cmName = strVal(linitial(names));
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ cmName),
+ errhint("Must be superuser to create an compression method.")));
+
+ /* Extract the name of compression handler function */
+ foreach(pl, parameters)
+ {
+ DefElem *defel = (DefElem *) lfirst(pl);
+ DefElem **defelp;
+
+ if (pg_strcasecmp(defel->defname, "handler") == 0)
+ defelp = &handlerEl;
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method attribute \"%s\" not recognized",
+ defel->defname)));
+ break;
+ }
+
+ *defelp = defel;
+ }
+
+ if (!handlerEl)
+ ereport(ERROR,
+ (errcode(ERRCODE_SYNTAX_ERROR),
+ errmsg("compression method handler is not specified")));
+
+ /* Finally create a compression method */
+ return create_compression_method(cmName, (List *) handlerEl->arg);
+}
+
+/*
+ * RemoveCompressionMethodById
+ *
+ * Removes a compresssion method by its Oid.
+ */
+void
+RemoveCompressionMethodById(Oid cmOid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression methods")));
+
+ relation = heap_open(CompressionMethodRelationId, RowExclusiveLock);
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmOid);
+
+ CatalogTupleDelete(relation, &tup->t_self);
+
+ ReleaseSysCache(tup);
+
+ heap_close(relation, RowExclusiveLock);
+}
+
+/*
+ * Create new record in pg_compression_opt
+ */
+Oid
+CreateCompressionOptions(Form_pg_attribute attr, ColumnCompression *compression)
+{
+ Relation rel;
+ HeapTuple tup,
+ newtup;
+ Oid cmoptoid,
+ cmid;
+ Datum values[Natts_pg_compression_opt];
+ bool nulls[Natts_pg_compression_opt];
+
+ ObjectAddress myself,
+ ref1,
+ ref2;
+
+ CompressionMethodRoutine *routine;
+ Form_pg_compression cmform;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmid = get_compression_method_oid(compression->methodName, false);
+
+ /* Get handler function OID for the compression method */
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression method %u", cmid);
+ cmform = (Form_pg_compression) GETSTRUCT(tup);
+ routine = invoke_compression_handler(cmform->cmhandler, attr->atttypid);
+
+ if (routine->configure == NULL && compression->options != NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("the compression method \"%s\" does not take any options",
+ NameStr(cmform->cmname))));
+ else if (routine->configure && compression->options != NIL)
+ routine->configure(attr, compression->options);
+
+ rel = heap_open(CompressionOptRelationId, RowExclusiveLock);
+
+ cmoptoid = GetNewOidWithIndex(rel, CompressionOptionsOidIndexId,
+ Anum_pg_compression_opt_cmoptoid);
+ values[Anum_pg_compression_opt_cmoptoid - 1] = ObjectIdGetDatum(cmoptoid);
+ values[Anum_pg_compression_opt_cmname - 1] = NameGetDatum(&cmform->cmname);
+ values[Anum_pg_compression_opt_cmhandler - 1] = ObjectIdGetDatum(cmform->cmhandler);
+
+ if (compression->options)
+ values[Anum_pg_compression_opt_cmoptions - 1] =
+ optionListToArray(compression->options);
+ else
+ nulls[Anum_pg_compression_opt_cmoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+
+ ReleaseSysCache(tup);
+
+ ObjectAddressSet(myself, CompressionOptRelationId, cmoptoid);
+ ObjectAddressSet(ref1, ProcedureRelationId, cmform->cmhandler);
+ ObjectAddressSet(ref2, CompressionMethodRelationId, cmid);
+
+ recordDependencyOn(&myself, &ref1, DEPENDENCY_NORMAL);
+ recordDependencyOn(&myself, &ref2, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+ heap_close(rel, RowExclusiveLock);
+
+ return cmoptoid;
+}
+
+/*
+ * Create pg_depend record between attribute and its compression options
+ */
+void
+CreateColumnCompressionDependency(Form_pg_attribute attr, Oid cmoptoid)
+{
+ ObjectAddress optref,
+ attrref;
+
+ ObjectAddressSet(optref, CompressionOptRelationId, cmoptoid);
+ ObjectAddressSubSet(attrref, RelationRelationId, attr->attrelid, attr->attnum);
+ recordDependencyOn(&attrref, &optref, DEPENDENCY_NORMAL);
+}
+
+/*
+ * Remove the compression options record from pg_compression_opt
+ */
+void
+RemoveCompressionOptionsById(Oid cmoptoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("must be superuser to drop compression options")));
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ relation = heap_open(CompressionOptRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+ColumnCompression *
+GetColumnCompressionForAttribute(Form_pg_attribute att)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmoptform;
+ ColumnCompression *compression = makeNode(ColumnCompression);
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID,
+ ObjectIdGetDatum(att->attcompression));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u",
+ att->attcompression);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ compression->methodName = pstrdup(NameStr(cmoptform->cmname));
+ compression->options = GetCompressionOptionsList(att->attcompression);
+ ReleaseSysCache(tuple);
+
+ return compression;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->methodName, c2->methodName))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->methodName, c2->methodName)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * get_compression_method_oid
+ *
+ * If missing_ok is false, throw an error if compression method not found.
+ * If missing_ok is true, just return InvalidOid.
+ */
+Oid
+get_compression_method_oid(const char *cmname, bool missing_ok)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ oid = HeapTupleGetOid(tup);
+ ReleaseSysCache(tup);
+ }
+
+ if (!OidIsValid(oid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist", cmname)));
+
+ return oid;
+}
+
+/*
+ * get_compression_method_name
+ *
+ * given an compression method OID, look up its name.
+ */
+char *
+get_compression_method_name(Oid cmOid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+
+ tup = SearchSysCache1(COMPRESSIONMETHODOID, ObjectIdGetDatum(cmOid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ result = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+ return result;
+}
+
+/*
+ * get_compression_method_name_for_opt
+ *
+ * given an compression options OID, look up a name for used compression method.
+ */
+char *
+get_compression_method_name_for_opt(Oid cmoptoid)
+{
+ HeapTuple tup;
+ char *result = NULL;
+ Form_pg_compression_opt cmoptform;
+
+ tup = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptform = (Form_pg_compression_opt) GETSTRUCT(tup);
+ result = pstrdup(NameStr(cmoptform->cmname));
+ ReleaseSysCache(tup);
+
+ return result;
+}
+
+/*
+ * GetCompressionOptionsList
+ *
+ * Parse array with compression options and return it as list.
+ */
+List *
+GetCompressionOptionsList(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ List *result = NULL;
+ bool isnull;
+ Datum cmoptions;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmoptions = SysCacheGetAttr(COMPRESSIONOPTIONSOID, tuple,
+ Anum_pg_compression_opt_cmoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(cmoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetCompressionMethodRoutine
+ *
+ * Determine compression handler by compression options Oid and return
+ * structure with compression methods
+ */
+CompressionMethodRoutine *
+GetCompressionMethodRoutine(Oid cmoptoid)
+{
+ HeapTuple tuple;
+ Form_pg_compression_opt cmopt;
+ regproc cmhandler;
+
+ /* Get handler function OID for the compression method */
+ tuple = SearchSysCache1(COMPRESSIONOPTIONSOID, ObjectIdGetDatum(cmoptoid));
+
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for compression options %u", cmoptoid);
+
+ cmopt = (Form_pg_compression_opt) GETSTRUCT(tuple);
+ cmhandler = cmopt->cmhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(cmhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("could not find compression method handler for compression options %u",
+ cmoptoid)));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return invoke_compression_handler(cmhandler, InvalidOid);
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 13eb9e34ba..f84588ed31 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2781,8 +2781,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/dropcmds.c b/src/backend/commands/dropcmds.c
index 2b30677d6f..8611db22ec 100644
--- a/src/backend/commands/dropcmds.c
+++ b/src/backend/commands/dropcmds.c
@@ -275,6 +275,10 @@ does_not_exist_skipping(ObjectType objtype, Node *object)
name = NameListToString(castNode(List, object));
}
break;
+ case OBJECT_COMPRESSION_METHOD:
+ msg = gettext_noop("compression method \"%s\" does not exist, skipping");
+ name = strVal((Value *) object);
+ break;
case OBJECT_CONVERSION:
if (!schema_does_not_exist_skipping(castNode(List, object), &msg, &name))
{
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index fa7d0d015a..a2eee78a64 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -91,6 +91,7 @@ static event_trigger_support_data event_trigger_support[] = {
{"CAST", true},
{"CONSTRAINT", true},
{"COLLATION", true},
+ {"COMPRESSION METHOD", true},
{"CONVERSION", true},
{"DATABASE", false},
{"DOMAIN", true},
@@ -1085,6 +1086,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -1144,6 +1146,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_ROLE:
+ case OCLASS_COMPRESSION_OPTIONS:
/* no support for global objects */
return false;
case OCLASS_EVENT_TRIGGER:
@@ -1183,6 +1186,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 9ad991507f..b840309d03 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -61,7 +61,7 @@ static void import_error_callback(void *arg);
* processing, hence any validation should be done before this
* conversion.
*/
-static Datum
+Datum
optionListToArray(List *options)
{
ArrayBuildState *astate = NULL;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d979ce266d..ed1c5aa577 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/multixact.h"
@@ -33,6 +34,8 @@
#include "catalog/partition.h"
#include "catalog/pg_am.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
@@ -41,6 +44,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +94,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -459,6 +464,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecAlterColumnCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -724,6 +731,12 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ /* Create compression options */
+ if (colDef->compression)
+ attr->attcompression = CreateCompressionOptions(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -771,6 +784,19 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * We should also create dependencies between attributes and their
+ * compression options
+ */
+ for (attnum = 0; attnum < (RelationGetDescr(rel))->natts; attnum++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), attnum);
+ if (OidIsValid(attr->attcompression))
+ CreateColumnCompressionDependency(attr, attr->attcompression);
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1610,6 +1636,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1944,6 +1971,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ GetColumnCompressionForAttribute(attribute);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1971,6 +2011,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression =
+ GetColumnCompressionForAttribute(attribute);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2180,6 +2223,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3279,6 +3329,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_AlterColumnCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3770,6 +3821,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_AlterColumnCompression:
+ ATSimplePermissions(rel, ATT_TABLE);
+ /* FIXME This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4118,6 +4175,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_AlterColumnCompression:
+ ATExecAlterColumnCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5338,6 +5400,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -6438,6 +6502,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("cannot alter system column \"%s\"",
colName)));
+ if (attrtuple->attcompression && newstorage != 'x' && newstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("compressed columns can only have storage MAIN or EXTENDED")));
+
/*
* safety check: do not allow toasted storage modes unless column datatype
* is TOAST-aware.
@@ -9361,6 +9430,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION_METHOD:
+ case OCLASS_COMPRESSION_OPTIONS:
/*
* We don't expect any of these sorts of objects to depend on
@@ -9410,7 +9481,9 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ !(foundDep->refclassid == CompressionMethodRelationId &&
+ foundDep->refobjid == attTup->attcompression))
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9432,6 +9505,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
attTup->attbyval = tform->typbyval;
attTup->attalign = tform->typalign;
attTup->attstorage = tform->typstorage;
+ attTup->attcompression = InvalidOid;
ReleaseSysCache(typeTuple);
@@ -12484,6 +12558,82 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecAlterColumnCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ HeapTuple newtuple;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ if (compression->methodName)
+ {
+ /* SET COMPRESSED */
+ Oid cmoptoid;
+
+ cmoptoid = CreateCompressionOptions(atttableform, compression);
+ CreateColumnCompressionDependency(atttableform, cmoptoid);
+
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(cmoptoid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+
+ }
+ else
+ {
+ /* SET NOT COMPRESSED */
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(InvalidOid);
+ replace[Anum_pg_attribute_attcompression - 1] = true;
+ }
+
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ heap_freetuple(newtuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/commands/typecmds.c b/src/backend/commands/typecmds.c
index f86af4c054..a7e9111c8d 100644
--- a/src/backend/commands/typecmds.c
+++ b/src/backend/commands/typecmds.c
@@ -40,6 +40,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
#include "catalog/pg_depend.h"
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e48921fff1..1f14818d32 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2806,6 +2806,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2824,6 +2825,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(methodName);
+ COPY_NODE_FIELD(options);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5473,6 +5485,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2a83da9aa9..4eddc69602 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2544,6 +2544,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2562,6 +2563,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(methodName);
+ COMPARE_NODE_FIELD(options);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3615,6 +3625,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index c2a93b2d4c..81e334a1d3 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index c97ee24ade..72e88de08f 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2799,6 +2799,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2815,6 +2816,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(methodName);
+ WRITE_NODE_FIELD(options);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4106,6 +4116,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c301ca465d..122d3de4d6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -397,6 +397,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +583,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> columnCompression optColumnCompression compressedClause
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +617,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2168,6 +2171,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (NOT COMPRESSED | COMPRESSED <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET columnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_AlterColumnCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3332,11 +3344,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3346,8 +3359,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3394,6 +3407,37 @@ columnOptions: ColId ColQualList
}
;
+compressedClause:
+ COMPRESSED name optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = $2;
+ n->options = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
+columnCompression:
+ compressedClause |
+ NOT COMPRESSED
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->methodName = NULL;
+ n->options = NIL;
+ $$ = (Node *) n;
+ }
+ ;
+
+optColumnCompression:
+ compressedClause /* FIXME shift/reduce conflict on NOT COMPRESSED/NOT NULL */
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NIL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5754,6 +5798,15 @@ DefineStmt:
n->if_not_exists = true;
$$ = (Node *)n;
}
+ | CREATE COMPRESSION METHOD any_name HANDLER handler_name
+ {
+ DefineStmt *n = makeNode(DefineStmt);
+ n->kind = OBJECT_COMPRESSION_METHOD;
+ n->args = NIL;
+ n->defnames = $4;
+ n->definition = list_make1(makeDefElem("handler", (Node *) $6, @6));
+ $$ = (Node *) n;
+ }
;
definition: '(' def_list ')' { $$ = $2; }
@@ -6262,6 +6315,7 @@ drop_type_any_name:
/* object types taking name_list */
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
@@ -6325,7 +6379,7 @@ opt_restart_seqs:
* The COMMENT ON statement can take different forms based upon the type of
* the object associated with the comment. The form of the statement is:
*
- * COMMENT ON [ [ ACCESS METHOD | CONVERSION | COLLATION |
+ * COMMENT ON [ [ ACCESS METHOD | COMPRESSION METHOD | CONVERSION | COLLATION |
* DATABASE | DOMAIN |
* EXTENSION | EVENT TRIGGER | FOREIGN DATA WRAPPER |
* FOREIGN TABLE | INDEX | [PROCEDURAL] LANGUAGE |
@@ -6515,6 +6569,7 @@ comment_type_any_name:
/* object types taking name */
comment_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| DATABASE { $$ = OBJECT_DATABASE; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
@@ -14704,6 +14759,8 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 8461da490a..39f21d4da7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compression.h"
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/dependency.h"
@@ -494,6 +495,39 @@ generateSerialExtraStmts(CreateStmtContext *cxt, ColumnDef *column,
*sname_p = sname;
}
+/*
+ * transformColumnCompression
+ *
+ * Build ALTER TABLE .. SET COMPRESSED command if a compression method was
+ * specified for the column
+ */
+void
+transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt)
+{
+ if (column->compression)
+ {
+ AlterTableCmd *cmd;
+
+ cmd = makeNode(AlterTableCmd);
+ cmd->subtype = AT_AlterColumnCompression;
+ cmd->name = column->colname;
+ cmd->def = (Node *) column->compression;
+ cmd->behavior = DROP_RESTRICT;
+ cmd->missing_ok = false;
+
+ if (!*alterStmt)
+ {
+ *alterStmt = makeNode(AlterTableStmt);
+ (*alterStmt)->relation = relation;
+ (*alterStmt)->relkind = OBJECT_TABLE;
+ (*alterStmt)->cmds = NIL;
+ }
+
+ (*alterStmt)->cmds = lappend((*alterStmt)->cmds, cmd);
+ }
+}
+
/*
* transformColumnDefinition -
* transform a single ColumnDef within CREATE TABLE
@@ -794,6 +828,16 @@ transformColumnDefinition(CreateStmtContext *cxt, ColumnDef *column)
cxt->alist = lappend(cxt->alist, stmt);
}
+
+ if (cxt->isalter)
+ {
+ AlterTableStmt *stmt = NULL;
+
+ transformColumnCompression(column, cxt->relation, &stmt);
+
+ if (stmt)
+ cxt->alist = lappend(cxt->alist, stmt);
+ }
}
/*
@@ -1003,6 +1047,10 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (attribute->attcompression)
+ def->compression = GetColumnCompressionForAttribute(attribute);
+ else
+ def->compression = NULL;
/*
* Add to column list
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index fa95bab58e..28680be49c 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2832,7 +2832,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 82a707af7b..8c8c51dada 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1009,6 +1009,7 @@ ProcessUtilitySlow(ParseState *pstate,
RELKIND_RELATION,
InvalidOid, NULL,
queryString);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
@@ -1283,6 +1284,11 @@ ProcessUtilitySlow(ParseState *pstate,
stmt->definition,
stmt->if_not_exists);
break;
+ case OBJECT_COMPRESSION_METHOD:
+ Assert(stmt->args == NIL);
+ address = DefineCompressionMethod(stmt->defnames,
+ stmt->definition);
+ break;
default:
elog(ERROR, "unrecognized define stmt type: %d",
(int) stmt->kind);
@@ -2309,6 +2315,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_STATISTIC_EXT:
tag = "DROP STATISTICS";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "DROP COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
@@ -2412,6 +2421,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = "CREATE ACCESS METHOD";
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = "CREATE COMPRESSION METHOD";
+ break;
default:
tag = "???";
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index be793539a3..a5cfbe3d4c 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index b0a9217d1e..fcff9d3fd6 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,14 @@
#include "postgres.h"
+#include "access/compression.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
@@ -548,3 +551,85 @@ tsvectorrecv(PG_FUNCTION_ARGS)
PG_RETURN_TSVECTOR(vec);
}
+
+/*
+ * Compress tsvector using LZ compression.
+ * Instead of trying to compress whole tsvector we compress only text part
+ * here. This approach gives more compressibility for tsvectors.
+ */
+static struct varlena *
+tsvector_compress(CompressionMethodOptions *cmoptions, const struct varlena *data)
+{
+ char *tmp;
+ int32 valsize = VARSIZE_ANY_EXHDR(data);
+ int32 len = valsize + VARHDRSZ_CUSTOM_COMPRESSED,
+ lenc;
+
+ char *arr = VARDATA(data),
+ *str = STRPTR((TSVector) data);
+ int32 arrsize = str - arr;
+
+ Assert(!VARATT_IS_COMPRESSED(data));
+ tmp = palloc0(len);
+
+ /* we try to compress string part of tsvector first */
+ lenc = pglz_compress(str,
+ valsize - arrsize,
+ tmp + VARHDRSZ_CUSTOM_COMPRESSED + arrsize,
+ PGLZ_strategy_default);
+
+ if (lenc >= 0)
+ {
+ /* tsvector is compressible, copy size and entries to its beginning */
+ memcpy(tmp + VARHDRSZ_CUSTOM_COMPRESSED, arr, arrsize);
+ SET_VARSIZE_COMPRESSED(tmp, arrsize + lenc + VARHDRSZ_CUSTOM_COMPRESSED);
+ return (struct varlena *) tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+tsvector_decompress(CompressionMethodOptions *cmoptions, const struct varlena *data)
+{
+ char *tmp,
+ *raw_data = (char *) data + VARHDRSZ_CUSTOM_COMPRESSED;
+ int32 count,
+ arrsize,
+ len = VARRAWSIZE_4B_C(data) + VARHDRSZ;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(data));
+ tmp = palloc0(len);
+ SET_VARSIZE(tmp, len);
+ count = *((uint32 *) raw_data);
+ arrsize = sizeof(uint32) + count * sizeof(WordEntry);
+ memcpy(VARDATA(tmp), raw_data, arrsize);
+
+ if (pglz_decompress(raw_data + arrsize,
+ VARSIZE(data) - VARHDRSZ_CUSTOM_COMPRESSED - arrsize,
+ VARDATA(tmp) + arrsize,
+ VARRAWSIZE_4B_C(data) - arrsize) < 0)
+ elog(ERROR, "compressed tsvector is corrupted");
+
+ return (struct varlena *) tmp;
+}
+
+Datum
+tsvector_compression_handler(PG_FUNCTION_ARGS)
+{
+ CompressionMethodOpArgs *opargs = (CompressionMethodOpArgs *)
+ PG_GETARG_POINTER(0);
+ CompressionMethodRoutine *cmr = makeNode(CompressionMethodRoutine);
+ Oid typeid = opargs->typeid;
+
+ if (OidIsValid(typeid) && typeid != TSVECTOROID)
+ elog(ERROR, "unexpected type %d for tsvector compression handler", typeid);
+
+ cmr->configure = NULL;
+ cmr->drop = NULL;
+ cmr->compress = tsvector_compress;
+ cmr->decompress = tsvector_decompress;
+
+ PG_RETURN_POINTER(cmr);
+}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1908420d82..9ebfb909ca 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -30,6 +30,7 @@
#include <fcntl.h>
#include <unistd.h>
+#include "access/compression.h"
#include "access/hash.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -77,6 +78,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -561,6 +563,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 888edbb325..c689f60e47 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,8 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
+#include "catalog/pg_compression_opt.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -309,6 +311,39 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODOID */
+ CompressionMethodOidIndexId,
+ 1,
+ {
+ ObjectIdAttributeNumber,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionMethodRelationId, /* COMPRESSIONMETHODNAME */
+ CompressionMethodNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 8
+ },
+ {CompressionOptRelationId, /* COMPRESSIONOPTIONSOID */
+ CompressionOptionsOidIndexId,
+ 1,
+ {
+ Anum_pg_compression_opt_cmoptoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{ConversionRelationId, /* CONDEFAULT */
ConversionDefaultIndexId,
4,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 4b47951de1..7ba6d31251 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -54,6 +54,7 @@ static DumpableObject **oprinfoindex;
static DumpableObject **collinfoindex;
static DumpableObject **nspinfoindex;
static DumpableObject **extinfoindex;
+static DumpableObject **cminfoindex;
static int numTables;
static int numTypes;
static int numFuncs;
@@ -61,6 +62,7 @@ static int numOperators;
static int numCollations;
static int numNamespaces;
static int numExtensions;
+static int numCompressionMethods;
/* This is an array of object identities, not actual DumpableObjects */
static ExtensionMemberId *extmembers;
@@ -93,6 +95,8 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+ CompressionMethodInfo *cminfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -289,6 +293,11 @@ getSchemaData(Archive *fout, int *numTablesPtr)
write_msg(NULL, "reading subscriptions\n");
getSubscriptions(fout);
+ if (g_verbose)
+ write_msg(NULL, "reading compression methods\n");
+ cminfo = getCompressionMethods(fout, &numCompressionMethods);
+ cminfoindex = buildIndexArray(cminfo, numCompressionMethods, sizeof(CompressionMethodInfo));
+
*numTablesPtr = numTables;
return tblinfo;
}
@@ -827,6 +836,17 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
+/*
+ * findCompressionMethodByOid
+ * finds the entry (in cminfo) of the compression method with the given oid
+ * returns NULL if not found
+ */
+CompressionMethodInfo *
+findCompressionMethodByOid(Oid oid)
+{
+ return (CompressionMethodInfo *) findObjectByOid(oid, cminfoindex,
+ numCompressionMethods);
+}
/*
* setExtensionMembership
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..5c1aaad48e 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,8 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +152,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +174,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_options_data; /* dump compression options even
+ * in schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index ec2fa8b9b9..77eb55eb69 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_options_data = ropt->compression_options_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d8fb356130..6df688adb8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -361,6 +361,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -581,6 +582,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_options_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -785,6 +789,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_options_data)
+ getCompressionOptions(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -3957,6 +3964,205 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getCompressionMethods
+ * get information about compression methods
+ */
+CompressionMethodInfo *
+getCompressionMethods(Archive *fout, int *numMethods)
+{
+ DumpOptions *dopt = fout->dopt;
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionMethodInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i,
+ ntups;
+
+ if (dopt->no_compression_methods || fout->remoteVersion < 110000)
+ return NULL;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ /* Get the compression methods in current database. */
+ appendPQExpBuffer(query, "SELECT c.tableoid, c.oid, c.cmname, "
+ "c.cmhandler::pg_catalog.regproc as cmhandler "
+ "FROM pg_catalog.pg_compression c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "oid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionMethodInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_METHOD;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+
+ /* Decide whether we want to dump it */
+ selectDumpableObject(&(cminfo[i].dobj), fout);
+ }
+ if (numMethods)
+ *numMethods = ntups;
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+
+ return cminfo;
+}
+
+/*
+ * dumpCompressionMethod
+ * dump the definition of the given compression method
+ */
+static void
+dumpCompressionMethod(Archive *fout, CompressionMethodInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ appendPQExpBuffer(query, "CREATE COMPRESSION METHOD %s HANDLER",
+ fmtId(cminfo->dobj.name));
+ appendPQExpBuffer(query, " %s;\n", cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION METHOD", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * getCompressionOptions
+ * get information about compression options.
+ * this information should be migrated at binary upgrade
+ */
+void
+getCompressionOptions(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ CompressionOptionsInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_handler;
+ int i_name;
+ int i_options;
+ int i,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ appendPQExpBuffer(query,
+ "SELECT c.tableoid, c.cmoptoid, c.cmname,"
+ " c.cmhandler::pg_catalog.regproc as cmhandler, c.cmoptions "
+ "FROM pg_catalog.pg_compression_opt c");
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_oid = PQfnumber(res, "cmoptoid");
+ i_name = PQfnumber(res, "cmname");
+ i_handler = PQfnumber(res, "cmhandler");
+ i_options = PQfnumber(res, "cmoptions");
+
+ cminfo = pg_malloc(ntups * sizeof(CompressionOptionsInfo));
+
+ for (i = 0; i < ntups; i++)
+ {
+ cminfo[i].dobj.objType = DO_COMPRESSION_OPTIONS;
+ cminfo[i].dobj.catId.tableoid =
+ atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+ AssignDumpId(&cminfo[i].dobj);
+ cminfo[i].cmhandler = pg_strdup(PQgetvalue(res, i, i_handler));
+ cminfo[i].cmoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ cminfo[i].dobj.name = pg_strdup(PQgetvalue(res, i, i_name));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpCompressionOptions
+ * dump the given compression options
+ */
+static void
+dumpCompressionOptions(Archive *fout, CompressionOptionsInfo * cminfo)
+{
+ PQExpBuffer query;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ if (strlen(cminfo->cmoptions) > 0)
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), '%s');\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler,
+ cminfo->cmoptions);
+ else
+ appendPQExpBuffer(query, "INSERT INTO pg_compression_opt (cmoptoid, cmname, cmhandler,"
+ " cmoptions) VALUES (%d, '%s', CAST('%s' AS REGPROC), NULL);\n",
+ cminfo->dobj.catId.oid,
+ cminfo->dobj.name,
+ cminfo->cmhandler);
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "COMPRESSION OPTIONS", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -4484,7 +4690,47 @@ getTypes(Archive *fout, int *numTypes)
/* Make sure we are in proper schema */
selectSourceSchema(fout, "pg_catalog");
- if (fout->remoteVersion >= 90600)
+ if (fout->remoteVersion >= 110000)
+ {
+ PQExpBuffer acl_subquery = createPQExpBuffer();
+ PQExpBuffer racl_subquery = createPQExpBuffer();
+ PQExpBuffer initacl_subquery = createPQExpBuffer();
+ PQExpBuffer initracl_subquery = createPQExpBuffer();
+
+ buildACLQueries(acl_subquery, racl_subquery, initacl_subquery,
+ initracl_subquery, "t.typacl", "t.typowner", "'T'",
+ dopt->binary_upgrade);
+
+ appendPQExpBuffer(query, "SELECT t.tableoid, t.oid, t.typname, "
+ "t.typnamespace, "
+ "%s AS typacl, "
+ "%s AS rtypacl, "
+ "%s AS inittypacl, "
+ "%s AS initrtypacl, "
+ "(%s t.typowner) AS rolname, "
+ "t.typelem, t.typrelid, "
+ "CASE WHEN t.typrelid = 0 THEN ' '::\"char\" "
+ "ELSE (SELECT relkind FROM pg_class WHERE oid = t.typrelid) END AS typrelkind, "
+ "t.typtype, t.typisdefined, "
+ "t.typname[0] = '_' AND t.typelem != 0 AND "
+ "(SELECT typarray FROM pg_type te WHERE oid = t.typelem) = t.oid AS isarray "
+ "FROM pg_type t "
+ "LEFT JOIN pg_init_privs pip ON "
+ "(t.oid = pip.objoid "
+ "AND pip.classoid = 'pg_type'::regclass "
+ "AND pip.objsubid = 0) ",
+ acl_subquery->data,
+ racl_subquery->data,
+ initacl_subquery->data,
+ initracl_subquery->data,
+ username_subquery);
+
+ destroyPQExpBuffer(acl_subquery);
+ destroyPQExpBuffer(racl_subquery);
+ destroyPQExpBuffer(initacl_subquery);
+ destroyPQExpBuffer(initracl_subquery);
+ }
+ else if (fout->remoteVersion >= 90600)
{
PQExpBuffer acl_subquery = createPQExpBuffer();
PQExpBuffer racl_subquery = createPQExpBuffer();
@@ -7895,6 +8141,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7930,7 +8178,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.cmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.cmname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.cmoptoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7949,9 +8236,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_compression_opt c "
+ "ON a.attcompression = c.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2 "
"ORDER BY a.attnum",
@@ -7975,7 +8266,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7999,7 +8292,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8017,7 +8312,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8034,7 +8331,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8064,6 +8363,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8080,6 +8381,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8107,6 +8410,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9596,6 +9901,12 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_COMPRESSION_METHOD:
+ dumpCompressionMethod(fout, (CompressionMethodInfo *) dobj);
+ break;
+ case DO_COMPRESSION_OPTIONS:
+ dumpCompressionOptions(fout, (CompressionOptionsInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -15513,6 +15824,15 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSED %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17778,6 +18098,8 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_COMPRESSION_METHOD:
+ case DO_COMPRESSION_OPTIONS:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da884ffd09..88b18ba15c 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,9 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_COMPRESSION_METHOD,
+ DO_COMPRESSION_OPTIONS
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -315,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +615,21 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+/* The CompressionMethodInfo struct is used to represent compression method */
+typedef struct _CompressionMethodInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+} CompressionMethodInfo;
+
+/* The CompressionOptionsInfo struct is used to represent compression options */
+typedef struct _CompressionOptionsInfo
+{
+ DumpableObject dobj;
+ char *cmhandler;
+ char *cmoptions;
+} CompressionOptionsInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -654,6 +673,7 @@ extern OprInfo *findOprByOid(Oid oid);
extern CollInfo *findCollationByOid(Oid oid);
extern NamespaceInfo *findNamespaceByOid(Oid oid);
extern ExtensionInfo *findExtensionByOid(Oid oid);
+extern CompressionMethodInfo * findCompressionMethodByOid(Oid oid);
extern void setExtensionMembership(ExtensionMemberId *extmems, int nextmems);
extern ExtensionInfo *findOwningExtension(CatalogId catalogId);
@@ -711,5 +731,8 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern CompressionMethodInfo * getCompressionMethods(Archive *fout,
+ int *numMethods);
+void getCompressionOptions(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 48b6dd594c..f5db0280e2 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -80,7 +80,9 @@ static const int dbObjectTypePriority[] =
34, /* DO_POLICY */
35, /* DO_PUBLICATION */
36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 37, /* DO_SUBSCRIPTION */
+ 17, /* DO_COMPRESSION_METHOD */
+ 17 /* DO_COMPRESSION_OPTIONS */
};
static DumpId preDataBoundId;
@@ -1436,6 +1438,16 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_COMPRESSION_METHOD:
+ snprintf(buf, bufsize,
+ "COMPRESSION METHOD %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
+ case DO_COMPRESSION_OPTIONS:
+ snprintf(buf, bufsize,
+ "COMPRESSION OPTIONS %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 41c5ff89b7..6b72ccf9ea 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/psql/command.c b/src/bin/psql/command.c
index 8cc4de3878..8b0dbfa45c 100644
--- a/src/bin/psql/command.c
+++ b/src/bin/psql/command.c
@@ -735,7 +735,10 @@ exec_command_d(PsqlScanState scan_state, bool active_branch, const char *cmd)
success = listConversions(pattern, show_verbose, show_system);
break;
case 'C':
- success = listCasts(pattern, show_verbose);
+ if (cmd[2] == 'M')
+ success = describeCompressionMethods(pattern, show_verbose);
+ else
+ success = listCasts(pattern, show_verbose);
break;
case 'd':
if (strncmp(cmd, "ddp", 3) == 0)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 804a84a0c9..4e077aff06 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -200,6 +200,69 @@ describeAccessMethods(const char *pattern, bool verbose)
return true;
}
+/*
+ * \dCM
+ * Takes an optional regexp to select particular compression methods
+ */
+bool
+describeCompressionMethods(const char *pattern, bool verbose)
+{
+ PQExpBufferData buf;
+ PGresult *res;
+ printQueryOpt myopt = pset.popt;
+ static const bool translate_columns[] = {false, false, false};
+
+ if (pset.sversion < 100000)
+ {
+ char sverbuf[32];
+
+ psql_error("The server (version %s) does not support compression methods.\n",
+ formatPGVersionNumber(pset.sversion, false,
+ sverbuf, sizeof(sverbuf)));
+ return true;
+ }
+
+ initPQExpBuffer(&buf);
+
+ printfPQExpBuffer(&buf,
+ "SELECT cmname AS \"%s\"",
+ gettext_noop("Name"));
+
+ if (verbose)
+ {
+ appendPQExpBuffer(&buf,
+ ",\n cmhandler AS \"%s\",\n"
+ " pg_catalog.obj_description(oid, 'pg_compression') AS \"%s\"",
+ gettext_noop("Handler"),
+ gettext_noop("Description"));
+ }
+
+ appendPQExpBufferStr(&buf,
+ "\nFROM pg_catalog.pg_compression\n");
+
+ processSQLNamePattern(pset.db, &buf, pattern, false, false,
+ NULL, "cmname", NULL,
+ NULL);
+
+ appendPQExpBufferStr(&buf, "ORDER BY 1;");
+
+ res = PSQLexec(buf.data);
+ termPQExpBuffer(&buf);
+ if (!res)
+ return false;
+
+ myopt.nullPrint = NULL;
+ myopt.title = _("List of compression methods");
+ myopt.translate_header = true;
+ myopt.translate_columns = translate_columns;
+ myopt.n_translate_columns = lengthof(translate_columns);
+
+ printQuery(res, &myopt, pset.queryFout, false, pset.logfile);
+
+ PQclear(res);
+ return true;
+}
+
/*
* \db
* Takes an optional regexp to select particular tablespaces
@@ -1716,6 +1779,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname || "
+ " (CASE WHEN cmoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(cmoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_compression_opt c "
+ " WHERE c.cmoptoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1830,6 +1909,10 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1925,6 +2008,11 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1932,7 +2020,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1943,7 +2031,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/describe.h b/src/bin/psql/describe.h
index 14a5667f3e..0ab8518a36 100644
--- a/src/bin/psql/describe.h
+++ b/src/bin/psql/describe.h
@@ -18,6 +18,9 @@ extern bool describeAccessMethods(const char *pattern, bool verbose);
/* \db */
extern bool describeTablespaces(const char *pattern, bool verbose);
+/* \dCM */
+extern bool describeCompressionMethods(const char *pattern, bool verbose);
+
/* \df, \dfa, \dfn, \dft, \dfw, etc. */
extern bool describeFunctions(const char *functypes, const char *pattern, bool verbose, bool showSystem);
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index a926c40b9b..25d2fbc125 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -227,6 +227,7 @@ slashUsage(unsigned short int pager)
fprintf(output, _(" \\db[+] [PATTERN] list tablespaces\n"));
fprintf(output, _(" \\dc[S+] [PATTERN] list conversions\n"));
fprintf(output, _(" \\dC[+] [PATTERN] list casts\n"));
+ fprintf(output, _(" \\dCM[+] [PATTERN] list compression methods\n"));
fprintf(output, _(" \\dd[S] [PATTERN] show object descriptions not displayed elsewhere\n"));
fprintf(output, _(" \\dD[S+] [PATTERN] list domains\n"));
fprintf(output, _(" \\ddp [PATTERN] list default privileges\n"));
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b3e3799c13..a323ca80c6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -889,6 +889,11 @@ static const SchemaQuery Query_for_list_of_statistics = {
" AND d.datname = pg_catalog.current_database() "\
" AND s.subdbid = d.oid"
+#define Query_for_list_of_compression_methods \
+" SELECT pg_catalog.quote_ident(cmname) "\
+" FROM pg_catalog.pg_compression "\
+" WHERE substring(pg_catalog.quote_ident(cmname),1,%d)='%s'"
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
@@ -1011,6 +1016,7 @@ static const pgsql_thing_t words_after_create[] = {
* CREATE CONSTRAINT TRIGGER is not supported here because it is designed
* to be used only by pg_dump.
*/
+ {"COMPRESSION METHOD", NULL, NULL},
{"CONFIGURATION", Query_for_list_of_ts_configurations, NULL, THING_NO_SHOW},
{"CONVERSION", "SELECT pg_catalog.quote_ident(conname) FROM pg_catalog.pg_conversion WHERE substring(pg_catalog.quote_ident(conname),1,%d)='%s'"},
{"DATABASE", Query_for_list_of_databases},
@@ -1424,8 +1430,8 @@ psql_completion(const char *text, int start, int end)
"\\a",
"\\connect", "\\conninfo", "\\C", "\\cd", "\\copy",
"\\copyright", "\\crosstabview",
- "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dd", "\\ddp", "\\dD",
- "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
+ "\\d", "\\da", "\\dA", "\\db", "\\dc", "\\dC", "\\dCM", "\\dd", "\\ddp",
+ "\\dD", "\\des", "\\det", "\\deu", "\\dew", "\\dE", "\\df",
"\\dF", "\\dFd", "\\dFp", "\\dFt", "\\dg", "\\di", "\\dl", "\\dL",
"\\dm", "\\dn", "\\do", "\\dO", "\\dp",
"\\drds", "\\dRs", "\\dRp", "\\ds", "\\dS",
@@ -1954,11 +1960,17 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSED", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED") ||
+ Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSED", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSED", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
@@ -2177,12 +2189,14 @@ psql_completion(const char *text, int start, int end)
"SCHEMA", "SEQUENCE", "STATISTICS", "SUBSCRIPTION",
"TABLE", "TYPE", "VIEW", "MATERIALIZED VIEW", "COLUMN", "AGGREGATE", "FUNCTION",
"OPERATOR", "TRIGGER", "CONSTRAINT", "DOMAIN", "LARGE OBJECT",
- "TABLESPACE", "TEXT SEARCH", "ROLE", NULL};
+ "TABLESPACE", "TEXT SEARCH", "ROLE", "COMPRESSION METHOD", NULL};
COMPLETE_WITH_LIST(list_COMMENT);
}
else if (Matches4("COMMENT", "ON", "ACCESS", "METHOD"))
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
+ else if (Matches4("COMMENT", "ON", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (Matches3("COMMENT", "ON", "FOREIGN"))
COMPLETE_WITH_LIST2("DATA WRAPPER", "TABLE");
else if (Matches4("COMMENT", "ON", "TEXT", "SEARCH"))
@@ -2255,6 +2269,14 @@ psql_completion(const char *text, int start, int end)
else if (Matches6("CREATE", "ACCESS", "METHOD", MatchAny, "TYPE", MatchAny))
COMPLETE_WITH_CONST("HANDLER");
+ /* CREATE COMPRESSION METHOD */
+ /* Complete "CREATE COMPRESSION METHOD <name>" */
+ else if (Matches4("CREATE", "COMPRESSION", "METHOD", MatchAny))
+ COMPLETE_WITH_CONST("HANDLER");
+ /* Complete "CREATE COMPRESSION METHOD <name> HANDLER" */
+ else if (Matches5("CREATE", "COMPRESSION", "METHOD", MatchAny, "HANDLER"))
+ COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_functions, NULL);
+
/* CREATE DATABASE */
else if (Matches3("CREATE", "DATABASE", MatchAny))
COMPLETE_WITH_LIST9("OWNER", "TEMPLATE", "ENCODING", "TABLESPACE",
@@ -2687,6 +2709,7 @@ psql_completion(const char *text, int start, int end)
Matches4("DROP", "ACCESS", "METHOD", MatchAny) ||
(Matches4("DROP", "AGGREGATE|FUNCTION", MatchAny, MatchAny) &&
ends_with(prev_wd, ')')) ||
+ Matches4("DROP", "COMPRESSION", "METHOD", MatchAny) ||
Matches4("DROP", "EVENT", "TRIGGER", MatchAny) ||
Matches5("DROP", "FOREIGN", "DATA", "WRAPPER", MatchAny) ||
Matches4("DROP", "FOREIGN", "TABLE", MatchAny) ||
@@ -2775,6 +2798,12 @@ psql_completion(const char *text, int start, int end)
else if (Matches5("DROP", "RULE", MatchAny, "ON", MatchAny))
COMPLETE_WITH_LIST2("CASCADE", "RESTRICT");
+ /* DROP COMPRESSION METHOD */
+ else if (Matches2("DROP", "COMPRESSION"))
+ COMPLETE_WITH_CONST("METHOD");
+ else if (Matches3("DROP", "COMPRESSION", "METHOD"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
+
/* EXECUTE */
else if (Matches1("EXECUTE"))
COMPLETE_WITH_QUERY(Query_for_list_of_prepared_statements);
@@ -3407,6 +3436,8 @@ psql_completion(const char *text, int start, int end)
COMPLETE_WITH_QUERY(Query_for_list_of_access_methods);
else if (TailMatchesCS1("\\db*"))
COMPLETE_WITH_QUERY(Query_for_list_of_tablespaces);
+ else if (TailMatchesCS1("\\dCM*"))
+ COMPLETE_WITH_QUERY(Query_for_list_of_compression_methods);
else if (TailMatchesCS1("\\dD*"))
COMPLETE_WITH_SCHEMA_QUERY(Query_for_list_of_domains, NULL);
else if (TailMatchesCS1("\\des*"))
diff --git a/src/include/access/compression.h b/src/include/access/compression.h
new file mode 100644
index 0000000000..bb4e299447
--- /dev/null
+++ b/src/include/access/compression.h
@@ -0,0 +1,68 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2016, PostgreSQL Global Development Group
+ *
+ * src/include/access/compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSION_H
+#define COMPRESSION_H
+
+#include "postgres.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
+
+/* parsenodes.h */
+typedef struct ColumnCompression ColumnCompression;
+typedef struct CompressionMethodRoutine CompressionMethodRoutine;
+
+typedef struct
+{
+ Oid cmoptoid;
+ CompressionMethodRoutine *routine;
+ List *options;
+} CompressionMethodOptions;
+
+typedef void (*CompressionConfigureRoutine)
+ (Form_pg_attribute attr, List *options);
+typedef struct varlena *(*CompressionRoutine)
+ (CompressionMethodOptions *cmoptions, const struct varlena *data);
+
+/*
+ * API struct for a compression method.
+ * Note this must be stored in a single palloc'd chunk of memory.
+ */
+typedef struct CompressionMethodRoutine
+{
+ NodeTag type;
+
+ CompressionConfigureRoutine configure;
+ CompressionConfigureRoutine drop;
+ CompressionRoutine compress;
+ CompressionRoutine decompress;
+} CompressionMethodRoutine;
+
+/* Compression method handler parameters */
+typedef struct CompressionMethodOpArgs
+{
+ Oid cmhanderid;
+ Oid typeid;
+} CompressionMethodOpArgs;
+
+extern CompressionMethodRoutine *GetCompressionMethodRoutine(Oid cmoptoid);
+extern List *GetCompressionOptionsList(Oid cmoptoid);
+extern Oid CreateCompressionOptions(Form_pg_attribute attr,
+ ColumnCompression *compression);
+extern ColumnCompression *GetColumnCompressionForAttribute(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+extern void CreateColumnCompressionDependency(Form_pg_attribute attr,
+ Oid cmoptoid);
+
+#endif /* COMPRESSION_H */
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index b0d4c54121..7ff4cafacc 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contain custom compressed
+ * varlenas */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index cd43e3a52e..2e1c92123e 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 2be5af1d3e..9356a1f13e 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ bool tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fd9f83ac44..115889c8f5 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -210,7 +219,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b9f98423cc..36cadd409e 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,12 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION_METHOD, /* pg_compression */
+ OCLASS_COMPRESSION_OPTIONS /* pg_compression_opt */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION_OPTIONS
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef8493674c..7fe9f1f059 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -120,6 +120,14 @@ DECLARE_UNIQUE_INDEX(pg_collation_name_enc_nsp_index, 3164, on pg_collation usin
DECLARE_UNIQUE_INDEX(pg_collation_oid_index, 3085, on pg_collation using btree(oid oid_ops));
#define CollationOidIndexId 3085
+DECLARE_UNIQUE_INDEX(pg_compression_oid_index, 3422, on pg_compression using btree(oid oid_ops));
+#define CompressionMethodOidIndexId 3422
+DECLARE_UNIQUE_INDEX(pg_compression_name_index, 3423, on pg_compression using btree(cmname name_ops));
+#define CompressionMethodNameIndexId 3423
+
+DECLARE_UNIQUE_INDEX(pg_compression_opt_oid_index, 3424, on pg_compression_opt using btree(cmoptoid oid_ops));
+#define CompressionOptionsOidIndexId 3424
+
DECLARE_INDEX(pg_constraint_conname_nsp_index, 2664, on pg_constraint using btree(conname name_ops, connamespace oid_ops));
#define ConstraintNameNspIndexId 2664
DECLARE_INDEX(pg_constraint_conrelid_index, 2665, on pg_constraint using btree(conrelid oid_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index bcf28e8f04..caadd61031 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression;
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index b256657bda..40f5cc4f18 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..1d5f9ac479
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,55 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the system "compression method" relation (pg_compression)
+ * along with the relation's initial contents.
+ *
+ *
+ * Portions Copyright (c) 1996-2016, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+#define CompressionMethodRelationId 3419
+
+CATALOG(pg_compression,3419)
+{
+ NameData cmname; /* compression method name */
+ regproc cmhandler; /* compression handler */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression * Form_pg_compression;
+
+/* ----------------
+ * compiler constants for pg_compression
+ * ----------------
+ */
+#define Natts_pg_compression 2
+#define Anum_pg_compression_cmname 1
+#define Anum_pg_compression_cmhandler 2
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_compression_opt.h b/src/include/catalog/pg_compression_opt.h
new file mode 100644
index 0000000000..ddcef814a7
--- /dev/null
+++ b/src/include/catalog/pg_compression_opt.h
@@ -0,0 +1,56 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression_opt.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression_opt.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_OPT_H
+#define PG_COMPRESSION_OPT_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_compression_opt definition. cpp turns this into
+ * typedef struct FormData_pg_compression_opt
+ * ----------------
+ */
+#define CompressionOptRelationId 3420
+
+CATALOG(pg_compression_opt,3420) BKI_WITHOUT_OIDS
+{
+ Oid cmoptoid; /* compression options oid */
+ NameData cmname; /* name of compression method */
+ regproc cmhandler; /* compression handler */
+ text cmoptions[1]; /* specific options from WITH */
+} FormData_pg_compression_opt;
+
+/* ----------------
+ * Form_pg_compression_opt corresponds to a pointer to a tuple with
+ * the format of pg_compression_opt relation.
+ * ----------------
+ */
+typedef FormData_pg_compression_opt * Form_pg_compression_opt;
+
+/* ----------------
+ * compiler constants for pg_compression_opt
+ * ----------------
+ */
+#define Natts_pg_compression_opt 4
+#define Anum_pg_compression_opt_cmoptoid 1
+#define Anum_pg_compression_opt_cmname 2
+#define Anum_pg_compression_opt_cmhandler 3
+#define Anum_pg_compression_opt_cmoptions 4
+
+#endif /* PG_COMPRESSION_OPT_H */
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index c969375981..dc3fa66f2e 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -3881,6 +3881,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
@@ -4677,6 +4681,8 @@ DATA(insert OID = 3646 ( gtsvectorin PGNSP PGUID 12 1 0 0 0 f f f f t f i s
DESCR("I/O");
DATA(insert OID = 3647 ( gtsvectorout PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3642" _null_ _null_ _null_ _null_ _null_ gtsvectorout _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3453 ( tsvector_compression_handler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ tsvector_compression_handler _null_ _null_ _null_ ));
+DESCR("tsvector compression handler");
DATA(insert OID = 3616 ( tsvector_lt PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_lt _null_ _null_ _null_ ));
DATA(insert OID = 3617 ( tsvector_le PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "3614 3614" _null_ _null_ _null_ _null_ _null_ tsvector_le _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index e3551440a0..ec8c3df953 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_handler_in compression_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bfead9af3d..a98ecc12ce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -140,6 +140,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -152,6 +153,14 @@ extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern ObjectAddress DefineCompressionMethod(List *names, List *parameters);
+extern void RemoveCompressionMethodById(Oid cmOid);
+extern void RemoveCompressionOptionsById(Oid cmoptoid);
+extern Oid get_compression_method_oid(const char *cmname, bool missing_ok);
+extern char *get_compression_method_name(Oid cmOid);
+extern char *get_compression_method_name_for_opt(Oid cmoptoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 03dc5307e8..367d6b189b 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -469,6 +469,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -499,7 +500,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionMethodRoutine, /* in access/compression.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 80c19b2a55..56e42ce1f1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSED <compression_method_name> WITH (<params>)
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *methodName;
+ List *options;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +651,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1622,6 +1636,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -1769,7 +1784,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_AlterColumnCompression /* ALTER COLUMN name COMPRESSED cm WITH (...) */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f50e45e886..7bfc6e6be4 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,8 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compressed", COMPRESSED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index e749432ef0..5cab77457a 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,10 +22,12 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
PartitionBoundSpec *spec);
+extern void transformColumnCompression(ColumnDef *column, RangeVar *relation,
+ AlterTableStmt **alterStmt);
#endif /* PARSE_UTILCMD_H */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 1ca9b60ea1..d21974696f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmoptoid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 254a811aff..6ad889af7a 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -210,6 +210,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 8a0be41929..ff7cb530fd 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -48,6 +48,9 @@ enum SysCacheIdentifier
CLAOID,
COLLNAMEENCNSP,
COLLOID,
+ COMPRESSIONMETHODOID,
+ COMPRESSIONMETHODNAME,
+ COMPRESSIONOPTIONSOID,
CONDEFAULT,
CONNAMENSP,
CONSTROID,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..119fac595a
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,154 @@
+-- test drop
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE droptest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+ERROR: cannot drop compression method ts1 because other objects depend on it
+DETAIL: compression options for ts1 depends on compression method ts1
+table droptest column fts depends on compression options for ts1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 2 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to table droptest column fts
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+------------------------------
+ ts1 | tsvector_compression_handler
+(1 row)
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+(1 row)
+
+\dCM
+List of compression methods
+ Name
+------
+ ts1
+(1 row)
+
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest;
+ length
+--------
+ 200
+(1 row)
+
+-- check ALTER commands
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | | |
+
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ERROR: the compression method "ts1" does not take any options
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+-- create different types of tables
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest);
+NOTICE: merging column "fts" with inherited definition
+CREATE TABLE cmtest5(fts tsvector);
+CREATE TABLE cmtest6(fts tsvector);
+INSERT INTO cmtest6 SELECT * FROM cmtest;
+-- we update usual datum with compressed datum
+INSERT INTO cmtest5
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest;
+\d+ cmtest3
+ Table "public.cmtest3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+
+\d+ cmtest4
+ Table "public.cmtest4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ fts | tsvector | | | | extended | ts1 | |
+ a | integer | | | | plain | | |
+Inherits: cmtest
+
+DROP TABLE cmtest CASCADE;
+NOTICE: drop cascades to table cmtest4
+DROP TABLE cmtest3;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+ cmhandler | cmoptions
+------------------------------+-----------
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+ tsvector_compression_handler |
+(4 rows)
+
+DROP COMPRESSION METHOD ts1 CASCADE;
+NOTICE: drop cascades to 4 other objects
+DETAIL: drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+drop cascades to compression options for ts1
+SELECT * FROM pg_compression;
+ cmname | cmhandler
+--------+-----------
+(0 rows)
+
+SELECT * FROM pg_compression_opt;
+ cmoptoid | cmname | cmhandler | cmoptions
+----------+--------+-----------+-----------
+(0 rows)
+
+-- check that moved tuples still can be decompressed
+SELECT length(fts) FROM cmtest2;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest5;
+ length
+--------
+ 200
+(1 row)
+
+SELECT length(fts) FROM cmtest6;
+ length
+--------
+ 200
+(1 row)
+
+DROP TABLE cmtest2;
+DROP TABLE cmtest5;
+DROP TABLE cmtest6;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8e745402ae..0e9b9a36fb 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -592,10 +592,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -731,11 +731,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -758,11 +758,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -795,46 +795,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..b5ca8b820b 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | | | A
+ b | text | | | | extended | | | B
+ c | text | | | | extended | | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | |
+ b | text | | | | extended | | |
+ c | text | | | | external | | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A3
+ b | text | | | | extended | | |
+ c | text | | | | external | | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | | | A
+ b | text | | | | extended | | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..2ac75c44b5 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index d2c184f2cf..ab21907a49 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index c698faff2f..a147b8217f 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 7481bebd83..98de0c4cff 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -452,10 +452,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -767,74 +767,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..f805e13d75 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..1526437bf8 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | | |
+ keyb | text | | not null | | extended | | |
+ nonkey | text | | | | extended | | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index e996640593..62b06da011 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,8 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
+pg_compression_opt|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index a4fe96112e..adbe764196 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 1a3ac4c1f9..a69c485e8c 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index a205e5d05c..e72015a391 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..6bf0ea8267
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,62 @@
+-- test drop
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE droptest(fts tsvector COMPRESSED ts1);
+DROP COMPRESSION METHOD ts1;
+DROP COMPRESSION METHOD ts1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
+CREATE TABLE cmtest(fts tsvector COMPRESSED ts1);
+SELECT * FROM pg_compression;
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+\dCM
+\d+ cmtest
+
+INSERT INTO cmtest
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+
+SELECT length(fts) FROM cmtest;
+SELECT length(fts) FROM cmtest;
+
+-- check ALTER commands
+ALTER TABLE cmtest ALTER COLUMN fts SET NOT COMPRESSED;
+\d+ cmtest
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1 WITH (format 'lz');
+ALTER TABLE cmtest ALTER COLUMN fts SET COMPRESSED ts1;
+\d+ cmtest
+
+-- create different types of tables
+SELECT * INTO cmtest2 FROM cmtest;
+CREATE TABLE cmtest3 (LIKE cmtest);
+CREATE TABLE cmtest4(fts tsvector, a int) INHERITS (cmtest);
+CREATE TABLE cmtest5(fts tsvector);
+CREATE TABLE cmtest6(fts tsvector);
+INSERT INTO cmtest6 SELECT * FROM cmtest;
+
+-- we update usual datum with compressed datum
+INSERT INTO cmtest5
+ SELECT to_tsvector(string_agg(repeat(substr(i::text,1,1), i), ' '))
+ FROM generate_series(1,200) i;
+UPDATE cmtest5 SET fts = cmtest.fts FROM cmtest;
+
+\d+ cmtest3
+\d+ cmtest4
+DROP TABLE cmtest CASCADE;
+DROP TABLE cmtest3;
+
+SELECT cmhandler, cmoptions FROM pg_compression_opt;
+
+DROP COMPRESSION METHOD ts1 CASCADE;
+SELECT * FROM pg_compression;
+SELECT * FROM pg_compression_opt;
+
+-- check that moved tuples still can be decompressed
+SELECT length(fts) FROM cmtest2;
+SELECT length(fts) FROM cmtest5;
+SELECT length(fts) FROM cmtest6;
+DROP TABLE cmtest2;
+DROP TABLE cmtest5;
+DROP TABLE cmtest6;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b422050a92..2fe9d7d434 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -340,6 +340,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -363,6 +364,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionMethodOptions
+CompressionMethodRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
On 11/28/2017 02:29 PM, Ildus Kurbangaliev wrote:
On Mon, 27 Nov 2017 18:20:12 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:I guess the trick might be -DRANDOMIZE_ALLOCATED_MEMORY (I first
tried without it, and it seemed working fine). If that's the case,
I bet there is a palloc that should have been palloc0, or something
like that.Thanks, that was it. I've been able to reproduce this bug. The
attached patch should fix this bug and I've also added recompression
when tuples moved to the relation with the compressed attribute.
I've done many tests with fulltext search on the mail archive, using
different compression algorithm, and this time it worked fine. So I can
confirm v7 fixes the issue.
Let me elaborate a bit about the benchmarking I did. I realize the patch
is meant to provide only an "API" for custom compression methods, and so
benchmarking of existing general-purpose algorithms (replacing the
built-in pglz) may seem a bit irrelevant. But I'll draw some conclusions
from that, so please bear with me. Or just skip the next section.
------------------ benchmark / start ------------------
I was curious how much better we could do than the built-in compression,
so I've whipped together a bunch of extensions for a few common
general-purpose compression algorithms (lz4, gz, bz2, zstd, brotli and
snappy), loaded the community mailing list archives using "archie" [1]
and ran a bunch of real-world full-text queries on it. I've used
"default" (or "medium") compression levels for all algorithms.
For the loads, the results look like this:
seconds size
-------------------------
pglz 1631 9786
zstd 1844 7102
lz4 1582 9537
bz2 2382 7670
gz 1703 7067
snappy 1587 12288
brotli 10973 6180
According to those results the algorithms seem quite comparable, with
the exception of snappy and brotli. Snappy supposedly aims for fast
compression and not compression ratio, but it's about as fast as the
other algorithms and compression ratio is almost 2x worse. Brotli is
much slower, although it gets better compression ratio.
For the queries, I ran about 33k of real-world queries (executed on the
community mailing lists in the past). Firstly, a simple
-- unsorted
SELECT COUNT(id) FROM messages WHERE body_tsvector @@ $1::tsquery
and then
-- sorted
SELECT id FROM messages WHERE body_tsvector @@ $1::tsquery
ORDER BY ts_rank(body_tsvector, $1::tsquery) DESC LIMIT 100;
Attached are 4 different charts, plotting pglz on x-axis and the other
algorithms on y-axis (so below diagonal => new algorithm is faster,
above diagonal => pglz is faster). I did this on two different machines,
one with only 8GB of RAM (so the dataset does not fit) and one much
larger (so everything fits into RAM).
I'm actually surprised how well the built-in pglz compression fares,
both on compression ratio and (de)compression speed. There is a bit of
noise for the fastest queries, when the alternative algorithms perform
better in non-trivial number of cases.
I suspect those cases may be due to not implementing anything like
PGLZ_strategy_default->min_comp_rate (requiring 25% size reduction), but
I'm not sure about it.
For more expensive queries, pglz pretty much wins. Of course, increasing
compression level might change the results a bit, but it will also make
the data loads slower.
------------------ benchmark / end ------------------
While the results may look differently for other datasets, my conclusion
is that it's unlikely we'll find another general-purpose algorithm
beating pglz (enough for people to switch to it, as they'll need to
worry about testing, deployment of extensions etc).
That doesn't necessarily mean supporting custom compression algorithms
is pointless, of course, but I think people will be much more interested
in exploiting known features of the data (instead of treating the values
as opaque arrays of bytes).
For example, I see the patch implements a special compression method for
tsvector values (used in the tests), exploiting from knowledge of
internal structure. I haven't tested if that is an improvement (either
in compression/decompression speed or compression ratio), though.
I can imagine other interesting use cases - for example values in JSONB
columns often use the same "schema" (keys, nesting, ...), so can I
imagine building a "dictionary" of JSON keys for the whole column ...
Ildus, is this a use case you've been aiming for, or were you aiming to
use the new API in a different way?
I wonder if the patch can be improved to handle this use case better.
For example, it requires knowledge the actual data type, instead of
treating it as opaque varlena / byte array. I see tsvector compression
does that by checking typeid in the handler.
But that fails for example with this example
db=# create domain x as tsvector;
CREATE DOMAIN
db=# create table t (a x compressed ts1);
ERROR: unexpected type 28198672 for tsvector compression handler
which means it's a few brick shy to properly support domains. But I
wonder if this should be instead specified in CREATE COMPRESSION METHOD
instead. I mean, something like
CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler
TYPE tsvector;
When type is no specified, it applies to all varlena values. Otherwise
only to that type. Also, why not to allow setting the compression as the
default method for a data type, e.g.
CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler
TYPE tsvector DEFAULT;
would automatically add 'COMPRESSED ts1' to all tsvector columns in new
CREATE TABLE commands.
BTW do you expect the tsvector compression to be generally useful, or is
it meant to be used only by the tests? If generally useful, perhaps it
should be created in pg_compression by default. If only for tests, maybe
it should be implemented in an extension in contrib (thus also serving
as example how to implement new methods).
I haven't thought about the JSONB use case very much, but I suppose that
could be done using the configure/drop methods. I mean, allocating the
dictionary somewhere (e.g. in a table created by an extension?). The
configure method gets the Form_pg_attribute record, so that should be
enough I guess.
But the patch is not testing those two methods at all, which seems like
something that needs to be addresses before commit. I don't expect a
full-fledged JSONB compression extension, but something simple that
actually exercises those methods in a meaningful way.
Similarly for the compression options - we need to test that the WITH
part is handled correctly (tsvector does not provide configure method).
Which reminds me I'm confused by pg_compression_opt. Consider this:
CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler;
CREATE TABLE t (a tsvector COMPRESSED ts1);
db=# select * from pg_compression_opt ;
cmoptoid | cmname | cmhandler | cmoptions
----------+--------+------------------------------+-----------
28198689 | ts1 | tsvector_compression_handler |
(1 row)
DROP TABLE t;
db=# select * from pg_compression_opt ;
cmoptoid | cmname | cmhandler | cmoptions
----------+--------+------------------------------+-----------
28198689 | ts1 | tsvector_compression_handler |
(1 row)
db=# DROP COMPRESSION METHOD ts1;
ERROR: cannot drop compression method ts1 because other objects
depend on it
DETAIL: compression options for ts1 depends on compression method
ts1
HINT: Use DROP ... CASCADE to drop the dependent objects too.
I believe the pg_compression_opt is actually linked to pg_attribute, in
which case it should include (attrelid,attnum), and should be dropped
when the table is dropped.
I suppose it was done this way to work around the lack of recompression
(i.e. the compressed value might have ended in other table), but that is
no longer true.
A few more comments:
1) The patch makes optionListToArray (in foreigncmds.c) non-static, but
it's not used anywhere. So this seems like change that is no longer
necessary.
2) I see we add two un-reserved keywords in gram.y - COMPRESSION and
COMPRESSED. Perhaps COMPRESSION would be enough? I mean, we could do
CREATE TABLE t (c TEXT COMPRESSION cm1);
ALTER ... SET COMPRESSION name ...
ALTER ... SET COMPRESSION none;
Although I agree the "SET COMPRESSION none" is a bit strange.
3) heap_prepare_insert uses this chunk of code
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)Shouldn't that be rather
+ else if (HeapTupleHasExternal(tup)
+ || (RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ && HeapTupleHasCustomCompressed(tup))
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
data-sorted-exceeds-ram.pngimage/png; name=data-sorted-exceeds-ram.pngDownload
�PNG
IHDR � � �W�3 bKGD � � ����� IDATx���w`SU����H��HAJK�dXVKYe��C�"�RE��>��dIJ�(����a���U�@i�����5�t��I�M���EOO��e��� g�DQ `]Nr ��H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� BnnnXX��Q ����
������qc�5N�:%�~�~��U���[7�F���[�����)S���������(�g��9�����.��(���dgg <xp�� 3 ��d�__�J�*���3g�DDD�����D�TmExx�����2���n���e�5jt�����~*e5�������������[�V�_|q���{���'#���7n��SO�/_��IU�V :t�� ��g�}�������V�Z���F����Nzz��9s�7o^�V�&M�L�<���� ,����U������u-��YSwi OO�#G������y���[����o��=!!��W)��}��w�^a������o<|�0� >�>}z��Mk����]�%K�h�Z�o��[W�b�2e�l��������n�2e�T�����,��[���U�J__�3f����o���[�B���~���!11����.W�\@@��K�A����j�����^�f���� �;���[I�s� x��������E+�i����^���e���{�_�~c�����c�f�������n��6lNJJ�tHzm������I�7�u�2e���QC_�k��qqq��2o?�������_~��s������W�2e�~���k�� �� ��������������c|,�O�~���r�����i���k�N��=�-%��jC��m��[o����-�� ��;��H��Hg������}��2e����?����B�����`��� �r��/�����ks�� 0`�a`%��B��)� ?n����� ���EQ����)S<==���K�j_~���9s� {���!C����5h����Jl������� }���;wDQ������
�X�bjj����� ?��3��333������#���h� =*Q�s��u���� ������I������edd,\��i�����Q���{����.]z��������/� ?>$$�����: .�2eJ�-���������?"""&&��g �3g��� ���3�����q���o���Q��={�>}��Db��y�~s��Q�F��m!&�S�Z�j�����<v�X�J���h4���l1����j��i�k�
)7UQr�1��������O-[����4��y��DF����%&&��[������>����C��h��6m��<y�������4���<y�j��YYYF��q�����
�w��������K��y���<RF %"ed������Q�e�W��[3� Jb�������{������� ;v4���L�s�����Y�f��� �P�R�e���h�B������#F�Z�j���U�TA�V 8p���� =�W+W�\��� l�R�J���������4�477�Y�f���K�7o�j��a��}�����M{���U*������~ n� u���\������%111]�t�X����$����������w�����c��III��W�N~����G������]��K���w��mT�l������
��~S5"�����4m�411����'N�(}����*��]�vo����
�_{�������X}�'�|��Y�������0�p��q���)SF�@d�M��x{{ 8p����m��wK)C ��������P;��}���iS�re}I�v��_�~��M��$el��+W6lhX�R�f���� /��{��%'' �P�^�a���2��Z���o���o8_W�����?��,��7h��[^�ti��MK��G�4h�����q�"j������w�U�V���{��������Z�w�����d�oo����l��pn��
�����nS�N��� ��7�����/Y�D�oQW�Z5y�����oT�Xq��5#G��Q�Fxx����������Q��O>1���������^�z�>�lPP���+����7{���>}������Q�Q�F�~���E����R-##�?������T�z�
|��G�K�� 9��S��5k6n�x������9rd��U����)���G�GAX�~���c�3TK��)iw6������ >|8$$���' ���.\�_���;��� ���m���;�������Z���W�^*��f�� |���p���5j�(S�����[�����������kX��M���cH�a��U�Pa����U������MJJ2��]�n��K�����O=z�������w||�o�����V�V���D����s�� 2~�x___�c���&�g��?�E�-Z��})��{w�N��W����� ���%���Wt��)[����K������rss=<< K���>���7�x�������+�����wJ���>�RH� ��||��'U�T�>}��+Wz��U�~}??������� 9r���������w����U�r�Jll��1c
�`�wK�����Mu������7a e�Js��2��8���Y�� @It��%�D�J�j������M��#�;; g�� h��eFFF�� gFGG/X����%]�TXk3g�,�"�NX-b��(�_~�e���� ?������{����m��X������xy�����������M��%���/l��e�V�\���n�b^^^�f�t�� :��gO��W~���>>>�6m��������4i���|��M�?��C_s�����5;r�HNNN^^^RR����CCC333
�~�z�5�o�~��}]5��.��^-%%e����O�����������&O�����5k���N�<�������+V��W/88�h�y��?�����E�M�6+W� 3f���
�b5a�R)?o�[��Q�F�V������s��ZmVV���{��� gX����-]NKKE�������
6�w��QlR�i������c�_|��[o��,v��Q�F��_~y����n�*�����;wta���KYM�L{����?��q���}�]NNNNN���G_x���C���N, 2d���G
K$�����[�n
6l��=�'O���� ���jW�^���_����=�xS-���e���������S��;���Jd��V����m�������W���O� ��j��o����������c)����? �w�n��]������OQ���.\ l��=�������;p�@�q�^��N�:��>��qS5����jNN������j���n)e�c�w�b�q�RI��=��3� ���^����X����z�h��j:���!)c{V�ZU�~��S�n�������C�����_V�P����������z���
kMbR���t�9���X-)s���^x�h����������������������w/!!!,,l��Z�V_�A��/^�%$$�����111���E�[�n�o��%e�.]��wFFF���D�O��u�V��/^���q��+r���g��"66V���;v���R���KII
2,�r�J��� >|hX�f�A �2f
R��u�i���(>x��i�����~~~yyyZ�����7��{������?���=R�\�7��I��Q�v�����^�z�^�z <�����x�����_��_�~� ��� ��i����>�V��I�zr�<~�8..n���-Z�x��
�)�������������� �|z/_���s����G� ��U��������j�����^��D���o���.S�0C�^x����6l���T"����UK�V;;;6��j�!!!����0a� �2d��eI�A�/��Qw����m[��npp�� �j��illl��
����{���G�����_���R"n�z6qSm������[���3��R�0�1a�Q,�&e��|uW`Rf���F�cQ����~�z��I����K�g���?��C�� O�:��+�n��1��_�����/_^�pa�/Zlk*�*//������w� =�h�ww��S���������?������;h���3g�_�����~'tH����KVV�Qa��}������o�n������|���U�T������X�lY}H�������P���cg���i�&��;$$$##c����V�JHH�M_�z�����%VEq���l��Y�f��
�t�R����v��8q��������� �?)�T���O��v���B�
�<��������g��Vg�������C��m���V��$z������
Kj����cG�Y 'O�
j����G�q�����t������/�X�r�����Kv����M�zr�xxxt����?<q���M�&M�t���[a����� ����Y�M��w��]������t//��3g�pE�=Ko���s��U�T6h���D7U�]F"����^������g\\��U��M������STT�� W4X�o���� I�����iS�ZmX���k��F���?�9s��C����Wlx�o�.b�}�-%��Z"��T�z�)��+�x�,v�������������9�!V999F�2�����O� I������s�>����G� �r��W_}UX�����������_�c��EEE�����{��?�������~{�����7h����� �M�8����W�^�q�F�^����+�o�
4(�u���_�3sqq��������)S
�Kl.����7o�o>���� ddd�kzzz&$$�������?x���#
��RJ5QG��n��q�����/11q���/��b�����cT���l�F��)H��������� .B���������o�����[�Z�����;�[�?�����`U*U�z��[*�����7O�k� ���l��7�x��� >|���kXR�N����_~����O���������?�� W�����n�\�i�n�%�2zyyyiii�GP:���999U�V�C�{������ :��N��5+U�d�M��xS5T�I���y����'<==���'�~��m�n)7U�(�M�r����-���<V}12*�5���]���o��mTx���g�}��j(I[���3x�`����z�O�>�v�*�Q���/�����aaaRv#OJJ���?K�Z��mw��]�Q��V�Z������_�x���� E�P����k�j��_����u������j7m�T`�g��� ,�����1K8g������}���??b�������7...::���e��� ;����c���m�>��S �=�\���N�:g��5*E������S(�3�<s��� ~�Aw���2�sa�7�Z+Z���0kV�w��/��][�����-[zzz >|X����������� s��1Jj����/���7��?����+�]����M�VI{��zOM��
����O?����_��k����rvv����Y�*=s�T�����D��+�y���������J6R�J�����[[�[J�M��k�~S�w�^��UM�J�L�[J �Xm�Q��H
�}u7���k����%�(������&TCH����������{����{��Nff�i �Z�6m<==?��s�:�f������H���W�^�?�X'33s��U� C�R-^����G� ��5�� 4,�Y�|yPPP��Iyyy���kV8n���_��aC���� 4(--���|��]�t1�j"��o����Y3�?�F�Az�������h��e��m���$������o������ AAA����K��{ v������������7��������?$$D�c�-~������l��% @���������� �����|�M"�����={���~������s���h����m
��|����;�������uUt#R>��z��������j� =����Z���0j����F}Pz�*���&�Tu�w����I�&��7OJ<����?��_����4/�>J���I�RU�T)>>^J^�I�&E��������PPP�ON*n�F�)������� X� ����<J }��Ww� OLL4�5 =z�Z�jF_h%V�7n����5W�) I�������������gdd�X���O>�3g����T���Wo�����^� gp������
*4i�D�����;i�� :����yyyyyy�O�����+��b�H�jutt��1c�3Q]\\6n�8v���?�Xw/�v��� ����h�"+�g��g�}��w���cccu&u��x���={������h�"����������[����>)���i�q�F��>�������S�N���/�b����:ujhh�nO�����>�l����;w��S����'**J�vI''���[����7)S���7K
����n�����������/[�L?W���}��u�����e��`� |���-�����S�~��U+''� ���09� :���/))�{������2e�n����1�{���i�����L�R�j������������s������o������O�����o�z�� �V��ju��t�����-Z������?|JJ���[���k4����J�,�iin��vQ���v��u��)88��Kz�������/� 9���/������ci^�|�F��Y�jU�y� :\�x���[�V�w�7n\�r%""����8�T���������sss[�je���+��R��G������1s���_~�����M~5j�� L��w���N�:=�����Xta�Wnn�G }t���q��5i��v���=��� ����
�R��~csCe������t�R�y
���F��T�2s��������n��� ;���������IG� ��������e�����h���-[�Z���5|"�}�
�<y�k��~~~5j������3g��;���V�vm�C����y������aTm������<y�3�<�q�F}Ijj���S��������m�t�R�f����������
��>�����|��U�T1<��h��O�T�R���?��C]���K``���9s��v�����G7j������g�
_�f����F�:����c{��Q�N�z��� =���KF��X�����Z��]�v�z��������������m��� ������=z���wG����4�S�H7��}�����/���_~���1����._�|�z�t����j�����^�z���g��;���[I�s��n�����������o�������������������ys��1�?������M���_�!,%�����NNN^�E @ �?�F�
(����7h��r��-Z���+J�����)S���[7 ((h��AQQQF��H�������u�]�v��U[�n�o��s���?�n����7���k����o����1d��j��u�b{��[�4h������U�jU�����>*����=-��Z�.S�L___]�u����'�a}��S�O���������
:4�!�c���U�z/^,��4b��e������>|�������+���������\� �b��������g�yF�R��cd�[�nF�����r������Fg �7U=��TG� ��ks���n)J��I }K��H'**JWn8���� uJ�������5�K�7m���Y��5k�o����Vb���$A���oZ��J%*r 0A��=���S�9���}��g�}c��4M����;n:��~�m��Y������������ 4h���W�T�������
��HipSU�b�����������3~~~� k��-s�����?M>v�.�� ���N�>]���K .\�x��Zsvv����(�y�S(VNN������/_���+i��������;-�P����#G���l,#�T����>::�{��dd`5G� 3f #� l^^^������W�3f�vHl�/���V�u������k���j���B�����V�Fb�F9T*��e����/w �*�g=z����Gw\�qS�����������o���7nX[O�c� [u���.]�����T��������}�c��� IDAT� ,k��)�i����NNN�*U��c�x�������J�Z�n��+�=,��ee�T��� � X� �2 2 ) �2 2 ) �2 2 ) �2 2 ) �2 2 ) �2 2 ) �2 2 ) �2 2 ) �2 2 ) �2 2 ) %&e���w��1h��y��Xa���!!!���'N�����o �EqIQ'O�|����#G�����p�����7'$$ >|�U�V�&M��o �HqI�J =m���5kXa��� ����� �y�������[ 3R\R�XYYY>>>� ���N�:e�� �����LVVV�2e K�}��k��Y��Epuu���-�� V%���!���LNN�Q�D�VgeeY��E���U���X*���^.y����{ ��fr#&<������l�����n��K�`iZ�t��O��l���[�A��������;��/as_��?�����:u���7M��ll�����Q�D����)��- �9;v��:ujBB������2e�����1,IJJ�o l�� ��s��I�&���7h�@�X
fcIA\\\n����1!!��^��oa����;���7`+\���0Kk&7b�m�i[l�����n��K�`iZ��+�����w+7H�6���wg7�%l��|����k��������y����� <�����W�XaT��w�}���[�nuvv��k������]k�����U0� ptv�q��{�o��Q�F����I�&�e�w�m����7t��G� =~����k�/_a���/����B�6mn����cGgg���k������o ������5��o��gdK��"���d K����8�;`���?r��o����_4,Wf��=e ���� 9rdll�QFF�H� �T6�� ����A�IBB��#�����Y3�c�J��wl�2g@ �8 80x���;w�i���
� �+1&���� q����}�~��Wm��-��2�,_ ����c}����m[�") l��������u��v��� �)H� ��������;&&&$$D�XLDR� T�����; ���'z���e�������DFF����M:%�sc[��W �������w��aC��]%>D��wf� �q����_~y����32�ER ����Ow��5**�[�nr�b$e �
8s�Lxx��5k�w�.w,��"w �t��f���TM^E���� �����6��_�������{��!w,fCR `~��5���4ZQ�� m��4�(��!/S�={�S�N+W���������� ���R�.#�����/��l�������W�X1`� �c13�2 ��;SkTr?_ P�������-^�x���r�b~$e ����lTR)_ P� .t��a��E��;� ) 0�~ jg��G��j@������\�x1,,l���C� �;Ka�_ ����Q�*����`N_�d�.]
{���� *w,�E��Z(�J�k ��\�|9$$d��y�jS��w%�d[��� `��]����o���kflV��w�� �p������3f�7#�X�) @�����H�� I ����L�>}��qr�b%J��c[�9
�u��5k��i���e[������� `�n��2e�� &X�}e�Y� @��]J�gdA�h����e���$%%���L�<�B�")c�DFF� `'������W��22c���8q�y[���������(q��mQ� ( ��M>r�v�Y��=�W���+ {��=���;wBBB� >k�,�^H��wf� P�~ j��s[���!c<v�x�&�lZJ�67OL��F�M;���;(8�;w���� :�����% J��G-��I ��6�C��P
�����a w��m������g��-w,�!) @�ZWS+0�!��dM�?�G�&��DAi�6��\RRR��o?`��9s�� ��X� �g�Gy�9�T�W��.#��_�y��� ��H� ����$6������v���K�.o��������K `~�n�F�G)p� l������ :���.Z�H�XA�'B�e�� @^��5Q��)#��Y���=e k�ed��m�d��_]��wf� ��1 0����� ;�n�Z���b)1Qd[��l @!t��^zi���r����;� Ky��Q�N�Z�l)cFF���(�-�L� ��������u��Y�F�R� �Q����2 ������v�Z�N �32�ER �YFFF�n�j��MF�$e �9�22~~~k��ur"�P(^ `6�����w�Y�fTT��)q������ �������{W�\y��u���(s����l�2�W �,;;�w�� �7ovqq�;�'(s���� �Q���}��qwwW`FF�H���� ���r� ��egg���W�Vo��E9���H��]�X
���;�E�3� �����~�����n��U9#� �3S �(''g��...111���(� 0�V� >|xvv��]�\]]����� %��j�
������w�-[V�plI P2�92���ddJ�� (�V;r��{�����G�V� �
#) ���M�� �����j�u=ZWc� 3�j������;wbcc����O��-�<U ��;��Y{.M������Y5&���Ch�J^^�+��r�����X777��)e�9 ���K���� ���R������8~�������������b�| p\,o�cgj�J��+��ed~�����8����� ��dM���� mn�����>�v,Y#wP0o7g��J�J �DQ�0a��s����<==��~�� �[�[� ��J���Y5 ��
0�(�'N<s� �c� ��X�b�Z��E�`�<-�*��6��<7�|�#�O9��.��4���Ea9�(N�4���S���^^^r�coH� ����v�Y�������V�'(�T������rA���1v�Y�J T��1��f��u�������/ ��-�t��fB��!q)� ��v<� ��/�22:�Y���/�Ji��Y <p�@�����>1S pPr-o���5���t����IV�B+ G� '�n�+���={vBB�����XI �qYy��(p�d�25*�/��0,�^��_5g������V�TI�X��� ���&�@�t��Q�����(��^��2����7o��� ����;;GR J�;���l�,����k�4�������[�}M��������������dd�C%�b��P8��� p8��5Q��)#��Y�����`�"##�m����X�jU�c13e��S J�m�a>����[� 9r��22���D�mQ��=Dp�������� �)�,Y �����3�� ��DFF���;� � !)SZ�� �D��-[�fMbbb�j����R�9xg� �k����W�>r�� gd��� pP+V������{���1S G�j��+V 9r���G�X 3e p8QQQ��-KLL����;�ER � ����'&&��UK�X �� p ���{�����(3e p�����w >���'w,`� �a��
s��=t�������@A%���1�6��� �t��m�6m����{�9�c��2�,_ ��}��W��M;p��cfd�� �l���S�NMHH�_����� �) ����c����4h w,0�L ���]�&M� �����
�L E9zS��Bz�&���i@]����rG@���w���kqqq���r���)q�a���
� �8��Y{.M���o�jg��@�V>�e ����{��W�����M�� �"(s���% ��v)]��A��_J�1 R�����W^���o��( I ��;SkTr?_ E��� 9rdll��/�(w,({� (������'�0����
&cc ����0b��={�4k�L�XP<�2 �5����_�G��{� ��7$���dM�?obJ�6�l�J��. 8p`���_�u�������� ,�vG��|��J0L'�H: zn d ?���� �s��-Z� �") X�M��[WS��� ��� ���G� �}��6m�� J�� `qG�6��
g���2�l~|j�6#����3h����] :�� ������[��k'w,(N_ ,�;�8���x�&�lZJ�67O�-q:���V��g�<��*���(�J��cs _�rr _}xbl��[,t-� ?~�O�>111!!!r��#) X��Qq�K�� �Z+ ���^���]�T���G7�
�����g��9�~n����g�[�Z���������-[BCC���`� `qR��e�X��6�MO2.��a��:�'N�����/�h������D��1�?"##��
�������]*�]��X�'@�\�k�xT�%�w���^�zm��!<<\�X�+22R?`�;���DQ,�
�R� � �%k��96[��*"���L K��s����L*W��-������9}�txx���k�w�.w,6C��w�/ � e��pn������Kdd����3���k��!#c��(�-�L� ���3g�t��z���={� ��Q���=e ����kxx�g�}FF�n�| �;{�l�N�V�X���/� ���2 (��������/_>p�@�c�9�� @�.\��h��A��5���� @�.^���� 2D�X`~�) �PGoj O�n�)��]�t�}�� |����C��AR @��'k���i�� )���i*Qh�C^��K�.�����7o��ar�Ka� ��m������h���K�2�8�k��u��q���r� ") �`gj�J��+`v��] y��7�� +w,�,�/ ���b���o�?�����l� Gt����������7N�X`q�� ��dM���� mn���/�X���N� ��J���Y5 ���a����!!!��M?~����H� �H�~1�|��zUvwvqRUvw ��+�]~��ed�L������ ���K �#��_L�jj��� )))$$$""b���r��a� ������~1�\�����c���G����r��") 8"�����;�����
�5k�����X� 8�V>jQ%���2%�����:x����zK�X �(���B�T*^C X[��YP>]Ff���s���;�����c�-�|_ `��'k��K����vV� �j��H�MIII
�����o�-w,A��w�� l����(��{���o��O 22�� `c$�f
@� <x��s���;GFF� dFR �1�f
�� t���]�v�/�;��� `c8��Q >���c�6m�,Y"w,P�� L!��G�f
�"]F&88x���r��P����E�8 ��8�@�<z��c��-Z�X�b���8(e�Y� �� �.==�{���5Z�|���@YH� %��G $JOO���k��uW�^�R�� {� %���|;��,L� ��� yeddt���v��ddP f� %&��������i)��<1%C}6�X��Za��.#����v�Z''F�( ��Z��G7����������<��Wp!���� ��222�w����EF���a�DFF� ��u5�'!���T�$���I�>8���� =z��Q#::����"##�v�c)�O��-�<U
1��}��g��p^��[�x XZVVV�>}<==7o���\�nS�2e��� $}� v ;;�o�� �6m"#�bq� `A�|��J0<})��� ;��������7��0�F��8{��(s ������������u�V22
���;�� (����������������t$e fsq����.~V����.^��@�p�A��qrr��u���������� ������co��� ��v5;q�MA��3���qY�V� >|xVV���������) 0� ���edtr��~�G�x,M��y��������-+w8�=�� ���� ���%vC���1����_�5��� `��75�g]�.�Y�fo�2��/g?QR�>W�h���#G������G����D$e S O�D�K�hEAR2��g�T��������
���r4{����d�
&�f�=-oH����7j��;w���A)�� `�m��u �V�~)��
O�D�MK������T��dMi� J����!k�y��qrUy�� YS�� {��W�q�����_�w�vss�; �6f� ��;SkTr?_�,0�S� ;@)�R���
rGa)�(� ?��?��������; �<f� ��vs6*����
�=7�0�(��������������; ��2 �)�x��U� �����os�4{n@�DQ�0a����JFf��% �9 ����G-����K��i
� ���g{`� �! ���8i����O���{yy� ��J��k�p*�! �r O��}2g1&����\��%��DQ�<y�O?� _�\9�����9xWbL�E��+ t& �'��=V�z8�l�-W< l��Y� 8p����+� L���;�� `� d \&� ��[o%$$������/ ��#��{<Y}6-%C��'�dh��� K�� `�������V�TI�X`�H� �����$�v)]�i� ���R��� �a���{��%#�b� �Y�G��� Y�X��o���7� :t���-�`A$e `�ZWS�m9�����`&�� �(���L�n��������h�4���������O=�������| �nV�8�-�`�/�� *���Z m�%Y�pl����������22�f� @Y������~d\ U�-8��SIcw�K���jj���gU*qpPu��R�%K��[�.11�j��r��@R
b�%Ef\�#�J���h�a��������g����@R�hK�.���JLL|��g����� ��%E�f7L���/�#��L�P�U?���8�cCe�����X��J`h���k��9r�H�j����� ��%E&�R1�� �?*���� S�
���2 K�W�5/��+>�����D22�2�2 P��f����~��~���GrMW��iD�I�w>�����zA�zC��� � �;��F�8�_��Q���Nu�
I�V�\�r����D E�wp(�� 1� !��R9���>������uS?>UF�� ���X�����4��SIcw������������T��A � ���OC�����N����i8�
e
�b����D___�c�#b� ��%E�������{��z)��#����4"�)y?�!M|2mG������}22�I (KI��q�^�
������d��b�7���O�v}��� ,8|�p�Z������K �m�|��zUvwvqRUvw ��+��@��M�P��c��dM5����~�6a��u�����C������M%�b��P8��� �� K�M��Tz��c� [N%���7��[n��
���?t�P���������;�� ��1�Y�Vc�1���A�EQex��� �m���s�l�_|1{�l22P%&�l�2�m lZ���1O��Y�7P![�����j��� �_�������9xgO P��u�1 ;�}���S�&$$���r�| �s��k�� �'ONHHh����� �") X�����]�|�6�U����� �O�K IDAT��u2���;'M� ����� ��/ Vr<Y}6-%C��'�dh��� Kf�
�N�� e��Ot��.�u�+c<�k��]������o��0FR ��m����R �����/���lpP�5}�Vtwuv�����OCN�6��}��� �o���� �c
��% �t%Z��w����~�@oH ��.����Q�F}���M�4�;�`��1�?"##�� �S��H�n�F%��� 0�������+�|�M��M�����������`J<���(��s X��#���?1������v���?���:��_��vVEz��{+`f �����m�����@)�9xg� `�b�#/n�Q�*��$��� s;p��������CF�GR 0�����'g�.G:����g^�nq�JZ��9��� <x�����7�;�x�) ��_�����}
��� � 9k ���G�2d�� m���;@f� &jU�r$�Z6��<7�|�#�O9��.�,z���c�����m���m�Z�*�y�� L��Z����^����SIcw�K���jj���gU*qpPuK\����}����uk�v�,�>`!,_ ,���M���A���I��� ��������b���a� `E/n�^��Q��|%�w�����{o��944����FR 0�����X������e�T�`��p����={�_��}���m��2 p,�S'fi���[�����z�S�� ��+�<�������?u�T�^���[��kW36 X{� �� O�D�MK���������d���� l�����k�4�������[�}M�����������v��v��n����M���) R`��,�Y8 �oH K �}������5k�t�������L 8��N�� w���W���);j��X&eG��+���l~��������W���C�X��b�
`��Wd���|;��,�Q���'�/�#��=e���2��<<1V�I!����'�
�����q����g;u��r���={� `*Q�����T�� �� O��}2�0&��>��=��1J�Dz������ ��:
��)15c��,���]�s �iX��U�r�Kr�c��� [�d������G��wf� ���6^�]+ �� S'���'.���<��(q 6�#7=��$��,���� ���/^LF��� �����E�N�?q��rR���+� ����Hvv�� Z��7D��A\�k��]~���l' Y�.#�h����m~`��~ `�a�����u�)'��m����������%��-r%���� �����\=<�g�0r2����f��;Cr2��������.\8d�"^��� ��~ jg��G�������9HN
&x|f�n3] 1'����2����?�|�5.��*'WO��-���
�r 9��3R����������Kr2���K�.�������C��D�������mQ�^A ���{� �������������6 ��R-h��(r.��0�L�(���^���a�*�\����p������y��EDD�;:8 e��S p�=kuO�('��/����[�K
�'������*�w���� ;��3�� �I ��aN��M�����=i�a'F��g�w ����`�k�e5Y*w��%R�]�~=44t������B��� �g��@��u<Y}.MwNvJ�V���6������T���/Y{����Jw�uU9���L�V���.���������������l��������(sY �
9��Y�OR@��jL����%���#���?q�RU����-�\ �9B�ny|�-�Z�Ue��Y�9%y�Xl����!!!S�L�0a�u���P����2 ���K��,� ���R��I��'���Z�; ����v��h�2�l~r�m �������%g<�q/U#�������fd���BCC'O�LF�� df��@�+qJz-o7��O�"���� �9���+1O���V�����cQ��?X���s����w��d �p�q�����m�s������}�C��G��8q�����I� ����� LN
8�4����v�����j@��i�)�#<G[���<���� �9�����1 ����������d�?� ��w��
1b���3- �8$e 33&��S�k��Q�n�U����IU��ytC�`�����-�MO2.��!K$%������F��b ��� :t��Y�� P(�/ �P�9���ZT �
6�B���1�Z��c�+Gx�6���Fn��'J<���t5�+\N�gXR��|��{�n��� <{�lK�(�7 �-��� ��� Td���v,Y�d�����M��s����L*W��-�����?OE���_���Z&��~��^(�~JJJhhh������g����9xWbL�E��+ @�I<)��Z�(%$�B2D(���$��;��H�SN��K�!M|���dJ� ����I�!~��������}��}��������I��w%�d[��� �����9yO��: ��T1�&1wc!���`91����8��������������l` ����oi���aaaaaa�-�;8e��� �xP��6A��-�v�����h����;?��������oQ <���Chh(88�2 (�����x�� ���R���E%� {w e}��������`� J�.CK=�%@��Q�M(��t5��m=CI��� [�@Q�!R�c-��p�0�n ����$��#s=�? ��<� ��<3��L�����<�w&�< >������Y�2��?z�������{�9�� 2e �7��� h nH��"��x��9�����9+R&��Ns�� ��[]o8i���/^|�-�<�������I��J^�Y� K|�/EiB�1�����X �eS��]M ��{�l_>�X>=Yd��7�a"27�t��-[�t ���.�=��4� ��v�%u��
��
�����h�O_��i����5{����>�0�Z�x�
7��u��h\ 0i>�KqO�"��+ ��DuB�#G���/NQ���0� (���F ^A�D��������{<v�&Y{��UzMM
EQ���4 ��S �.��VL����h�` Qm|S�z������~����y������������ D]T'4���(8���Xn ��*��OT�_�U��fu��M�Mv���CAU�:(����2 u����!�Xn �*)��Y6k�D�Z��2QW�l����;����v��P���� Df^�����������X1]����0��O�fB�H���(u ���� ��Z��D�F ��� DQl� �}�� �C9=f���a�&h ������f�w�r(/�g�� � nwW R#����"�� R�I����L(���f��{��z����_����y(K7�z��������4 ��) ����� �!�X��;gM� �Du&��e���-[��� �J�Z5�;b�@��� �E�Q�����l6����5��]�T*$ �� D��� �� 0��l�+V$$$���""�~� @��=���[G{&��b�����>�J�� R�s#/�� c�B$Q��$�� ����\�r����������(�v |�����$/��� �R�C$� N�s��U��� "2 Y�|xGR @��R�
�)K�nF�-�����z����~wD�n>d �I;{)e�&y�Z�H�=H�2 ���h�q9{�����<L�$J�+AC$�o&M������T a�q��t>����� |��F�!��-��[ m!���.��V E�� ��)�Da� @|����{��$�f08 �����-/'|�fnK�a@��;��5k�tww�#2���N&"s m��!����2 �)�>AC$�of^�f��� �R��2t����oF�_ -��
e��m6�� n�5�u����.��� ���o<#2����:�v�x_�8��% ��$l�O��0AgK ���5���O7v;L�������/$;S��?�f�c�w�8M�=���_~YWW��j=��������.�M��2 �i�t�v��Ha���sK�&@|D����Q���t��z�����U�!})����=���� ;�+3�/�����N�g�F���o��2V�Di5)k"�#@ C� @|��'��0Bm&��r "gi5����z3�gO����['���|s?�a��\�����4M������uuuIII�������*T�)J�PM���T�]�M�2e �V����#�f0 $��0�I�����R�P}e~i*�ni� <QF�M��H���2BS��%�����I������\������'�O������ C�[�1� <!SF �e���b� @xA�+��� 0�� 9����0h�i����m,�$%K��Z�N
?0J���
������������hO�������.�^|CPF �e� @\���i������u�t�\�n������s{�vF���K�8V�OTm.����x��'N���/��L�'@UYY�~`{/�!( AHg��tv����W�/��t9��T�H����� �f��)uj�b�D]��Y��v�}��' :��� O�8��Y"V ��$.� ��w 0N�5�e���s%ej��s����]���wW0Qj}����y�"�22O=����� :t�W�t�+�/��^�J�D����� _�|xG� q��Z��WZ�S��w�3�F� qfn�$�~��J�W��<���vjI��5��)�B�J��rc� #2O?�t]]�� jD����f�d�W �9)��E��6 B��N��3�~�k�Fq� }x����m��Qk������P41ai5�4U8�F�.+��Z�W*��|�7}I.|�7n�����������r�����Fw�J�(���_�"��B$��w)�I^��s *���H_��k��d�����<����Z;xb�W��
�b6R:Z�*N��CY� �s���p� �%�����������������#VO���.�=��4� ��)��hV�]���V�
���+��e /=�Ns ��\Q%O�XvV���Km��u��1�/��������d*;.��K�����{wCC����c�c�(����*�! C-N�J���F�u��$� �sG�0EI4iZe�wp�
�2��RH���#�EI������� !�d�m:u����%?��so����#G��6e �S�`���&wY!����9k��Y1]��� ���z��8�i}���!��dV �@��q ��Z��H�� '�u�0Y�&h��\[:'K����6�u2
Y��l
e����K�6d�0�V�����_��-��W_E� @l`� @|Z1]�QR����� I4\��4kg%g��*��S���|3��4�F���awD����������k�1� �$����c@J�O��Y����6u��mn�7����~s���]M���H|����"��0�>����� ;�&[�#���� t��} � ����B�/ �A ��4/KsK�F� ������qS�8�G�+�n���T�fX��RaZH�,�� 7&�3a���!�I|���gL��K��c��3^���^ig�Pu�b�:��R�N�\p�0�/����7 ^Q��_333��3 �
�K ���h=�nu�!��"G��������+��� qSrL (�� ����DN�W*�1����vbW;g%I9��s6��l5��K��qO_���Is��u�R�tM��n��H�K��u� �����Dd b A ���q��,
M�x���7s��-����x��i! ��m�(��9����P]R ��-����9�k�\�N�)���I
���
&�^1w����P�W����[���rX!���<�UGQj�B�������n�z����,iE� � �� ���NkyC_i]O�9P J�In|��X}m(B&$*�]it����vN|�*��O �Gn}�js� ��_�RX��<%A����$�d�����b �Ih��k�b |��j��/����:�~X��CkS���� S ~xf�p ������I�y��#�6E��D��Jr��I��HD� '��d�4�Y�U\ p�I~ijHc������� ��_�������'.W�H��=M��� J� � ���(���,>R���� MhB�G]�*'�~4?�v2�a�"�8q�tN�����D)3h��{%���k���?C_��^7�m�N DP fP� c�b������Vn�
EH����Y|��� ��%"�8 ��� � {Mi5)kv��YUU�Hg�D��b�9��
�2 r�m��W+F]��\�W�~~Z4�����g�S�&�K1(������;ig/�L��>[�<�� ��'v+�n��uEEE}}��{���WqS�.��5 e �����(���|�5O�dj�� ��r�0l��K��n9l ����� ]����g[��|5L'�L)>����C�����*���'����fo�� � nu� � �� ���0�]kg%g��Q*Y
�UH�^�Y����>�a� �bM\JT���&��+��59����o}�[����57-��4Q�PNI�Xs�������5���L �����'{Ep�B*��/�S�D��cJ�a�;q��\��AG�:��K�
J���@ � ����z��6�����0 ���h�X
�Ud8�; � ��0B������5����a��%�`����o����`��N{���w����_{��$�Q��z� ��2 2Rh#@�Q�����A��O�9CkS��:�y(K7�z��������:�f�-6�����=]�[N����>�����<�UGQj�B�? ��F� ���"��Oo4Z�v���ZA�# B������i����m#�e���:����E*�����,
�?7�{g^��o���������y��1�~���X��S @`�� ����|�p���w�5y�d�.A���h|p 9����=��b��6�:X25�dKk���k�����[ 7�ovi!_�^Q~T�0��� ���L;{��> ��L �i4Zw4����
3Z��L������
��m.�(}p 9�0�V��!]���S�VB�sJ�� �4��t� �kJ��:�R���W �2 "8�i-o�+��)o�;���;DXg����z��r� uZ���#��x���� ]*k%[�������&2�6v��}�� ;��� � {��jR���U �2 �4 $�:#�ei��J��)U
*C�\;+9�"#�G��� GU�/���/������<���?^�z�B����T�+��S/��Wk�R�PM�(�B5Y��S�vA�{ ��S �|���������4���E8����aL��'M��2yEa0�I
'�u�0Y�&h��\[:'K� � %S h�xN_*�������_V�\��;��U ��L���~>������L�D�������E�
���? BAQ� 4+�.�:���_gM�vy�4!�"��7 ��Z?3������O������K���P�6u����i����'��-���� :z�����w��=��i{������I�dj�����v��8��I�#���4 ���'y��� ���#}�|�Iz%!��{Q� ��4y�o� e� ����<�w�#� iF���i����5{����>�P��@ �� [�l��o�]XXH�������y@�Re]��v 2i>��| �VL���A����y�u�������D��h��k��_K��S<O�qy�LC �&�:AEX`��`7�n���466.[�������Brt����&^��* b �~ b�g�\g�����/����b@�x�����esV@t�� ���{w���`�X��HZ@l S @�|��3��5w� % �I�MU����=e<{���8q�?��o��p�WYY��� f� �g��pD�? �I�MIA6MS���� �#VB��������_}����wK�
J�
b�+ �I�}n�E��� @vvL�� IDATb���5������������; � Dt����K�n����;�{/ R'��w)�I^��s yi4Z�yG:B�����`@ ��ji�p��J]VRA�6�4�w?}���%Kjjj����X�@����.�=��4� /����aO�+_��&�fb !O��F�B��9(�.Kk���u��R�kJ�O�a�vj��1v�!��N��K)�5�k��EB�������+?����&@|���;�/ �OR�s1_I,�M e{����v����\��yWS����FNotGd!��4rz��v�a��V���� .G�y`��R/��?����K������� � �2 ���<���������������V�I*B4�l<p�=��bul:pF��@`;d�0�{���$�e�5m���}������~����}���_
���K ��?;�~.�F�����1;w4S4�Y���Jb�`G��9+ *]�c����>�������Gp��B�7��?��t��-[���G?��R �� ��,��Y�:�JAe��kg��g�h��h �1; .� � 3�}h�_��b�^���/D� $����������V�N�����������U�z����'���y��3�u���J`v����������R����3g-Z�������8�m�� S @n��M{�X����Q��,
M�|��1_)b�M e{?g���~1E�x%VT�X��sw�>Q��h�{~�SKE{O_���4�n�w�w�t��K�9������{}�%
S E;�L����~������t� D�2 ��)�n/!%��AH|��aeJ
�i����T\��K�u�;m^i�o�����y����>��AKW��go��|���Z�P�����3���h� 1� ���O�t{�����g���9Y� �mi� <Q��r��
�(#4�� a�[5�a��/���R�Wn���t���ZI���@�AO ���� �I��mk J��/&�Pg�D~IJaMf����J��PX��_���tJ��YIt�����my9��7s[Z �� ��L ��\���_*M$�^P�$"i�� :�"�_��_Zs_7M����/P�RKBi5)k��e-��
e���2�fk(�$�_B� "�� ���V'-D������_LPA�:��������;���)I�QJ��I^��.�.�n�f"2 �����ne bA � ���K���J��Yy�q�>�fW
~��]�� � bioo/,,|��G3�? ��t��+�� � e ��:)x!EM�+ �
:��b8i�L��q\��0�I�����R�P}e��@��Dd~���>��`�dR�z������l�P7��(��� �Q�C `{�H �:i�^���i��P����_1I�*C�
����u��p�-���� =-��
�����^Q�-+�+k:::����~��� {L�k�&[v
6��~k{VH
��B���� �H�$12t�Y�4!=f��E3Mj�1 �n��3�� !�4��t�L�����I��.�������]�V�� �MF8"
"��% �E8&�p��P�x�� �I
@4t �c����o�N���,X�j��'�xB�+�&# BP *" ��
�\�:�.��8mk�xt7�s9��s�f������2���\�pa��%%%O>��� W�$���| @�n���T�fX��RaZ��]�tg% �F���aTE�?UE3��c���OTm.���������=������'�����Dd����~��h\�|� ��� � D���'���#�,������e��L����)u5�f���/��v'����w~����������������� ���?�����6����WRl>,/�l� �'uF+�.�����
���+#�2������^m��u��1�/�
�� �n\^2������� �{�����"l`<�����$/��� @4�_���Gw�|�;�z<������}^+��g��5� X�h�� �}����`����;�� x���+Q���0�4�V�'<�/���p��� w O��(u�YG���� ����?�|Dd �3L_ ��g��R�����nEmSG��KI.m�����E���"����Q�z��� �F�m���� �&L�R�.��p�����K�\/^|���>�����3 H2e x��A-�Pj"j����8(�'Ir�� ��������i��k !�d�m:u��u������[zu���������7��� /�x� 5� ���P���������������>�-��!$�E�q�q#@�R������OI��V(�$M��iyq��CCCEEE7�x��-[b�Y � �/ ��b�~����� ��+�F���'2Y0MXuOQM������9�k�\ #��� ��)��K���/YZ
#MN�Q��J*�������x�F�R�WP�76n�d2�y������� �2 ���~�O3�4��5�����L��*���{pRxI.�s��������(��&B�c�m�D�)���0n���_��� ���L���d����g����+�P�� o� ���LM!>Y0A�p�5r�'���'�Dn��F&"��������q+�Z@S��d��<���P=gIq�����f� w�1m��GdZ �T\0��,�
�W������ ���t��4G� �t<r���3I����4�a�.�F�6��������\x~��W&��`�~�
G������z���/G� '��a��)'2��j�s�kI�x�*�i
������}�B���-��
����y��+
�e���> ^I�� �2 ���M�4�hO��V
��6 �X�0�m�#�PjYP�r ���V�q�1#a7 ������2]��F�[����l���;�L��� !������ !�ar}��A I��% �����Y;+9C�T)� �r����/�S�Neb�1r�
$y����l�HfZ�K�����4��R��)�>i�f�#5v�a��V���� .G�y`��R�~�b��u�]999;v��qD�2�ag���W @\�� �8!b���������� � sI��A��gZ�N������ZB(�{�R������NB[�^���j�BB�����+���_}���Gd!I9��s6��lu�� ( � �~���q�nU��()���(���$O��R�3�D'�Pj.i����Jd'D;{9+=����|�r�^��[o)��DQ�VMjX��Sf��I�� �AP �AH]W�0�E��E��N�u�`����E.�7��H�<*�XA��fZ�N����0|Z�(e�4z��3�Fc0T*����KRM{M_*FC iAP �� �g���I���!�"�|��O�#���(E.��B�HbM_�K:L �J���r�I����e�������V�X���P[[+bD��_��1� R�� ��V���&a����5�`��EIa� E��0;�t�P���9Y���v�:�+�v����.���{��Z��G���R��� ���% �+��5J��2@�#pZ��@��:h��h������
��k*SH�� ��=f��E3Y9��|;��w�Q\R�r�+>-j���I� �t�o>�n�B�x����j����� �x`�4 7 �j���[v��(.q��ZA�(?��>�rx��?�1�����t:W�\944T[[�� ���: �<�������H��:pi���\")A���#y1�4zv��W������t:W�^=00����'&&�� �
e `|a�!��H�K�U�i��� �vr �a0�L� �HmSG��Ks������>>����_|��!�>-/�����uF�`�������W�CD B�� �;�0D��h��vDFa�a0D��g��� !f���_}�n������Se����6B�]�<s�����L-{_ 'M���>(
�! �����
���f�_F��� 4��u�<W({�?/S%O�XvV�]A���y��p��Jr���O�� L���� �5�;��I�rR5�z��+� �����}x����p�(; �BP LrJ��I�hr�VR�
�(�GU���{>wW0��_\i w-s4MO���fxM:����A
Fb ��v��Z��WZ�S��w����,�Lk'Mz�� ����|� �GIAv��Y�$��
G������'�/2wmi5����z3�gO��� �v$���� ~x���N �G��Q���]/�� @��XR%/�,K >��m��]��L�3�z���n��yI����� �7���?�V���kLY����XG�/��Pj}�
��yb�����O�>]����Ig�|w������;�/Xt������� �&��w�/ ������ �!�X��;gM|�2�~&XG2��S����r�����h�gc� �3��p�XQ���5k��z���s�b�7FNotGd!��4rz�x��4�a��S�N 8p��]������c�R�s�Ow{ ;� ��u�C�Ui4Z].�oE2��S��"V4����7�>b��� ���m�(�{��L[��lo3E�%b��q�:�+��?1M��<��_���$'''�B��T\0��,�
�W����G �e `�J�*�����DUv�5��~�(��� ��3�h�Ds�.b u���mb)���8���w?`B�i����q�2*]�c����8�O���O666 :th�� �J~ij~i��� YC�_ ���:��g�t�FI�_���p#&�� �r����
%�qbC<sp|�����D�����$�~D�1���������1��>��}�����F��4��R����Qb�=��� ����'N�(�^ ~ S $$�I��44E< gn�Q8�����W��O��
sVL�o��@�3��7OB��V�iAR l ��T��^��Jvj��`�Z���ZB(��?q� W����~z������W\q��{��"����"�� 2��j$�<#� 3Z=#&j��>���=����a��%@�HD� �� ������u{>w �U����$���c���/�Y �����?� ����;�����,��������d��% �#��wd� ����4R���ZEY]4E��r��!?}[�������Bn������3�G���v��rOb[�+�� � ��� ��)�������7����.Zx�Q�S�c������ ~�/"c7 ������2]��F�[� ��!( ^���T3�y0M�����n�a�W� eOi�A��3��
6�)������� �0����Y���d�^��������w5u
YZkO�1� #m}��m���U���|���i�;��s�����t��v�a��VB[!���<�UGQj-r� � ����4��i���<h&��h�~��#[!1&O����^��)l���Z��3��oW����Giz�e��&|rvFNod��P������]?�/�\gBy��g���?544L�4�����LD��b |A �/ ����������S������f*�x#v�76[.�9�����los[���t1}pw5ur��������'�6���Ay�S�M�o%�������0�����O7u �C�{���_{����O�<��1������}��n>4�]:�Y4�]j7 {; ��L ��P�&�k��)��)��!���FI};#������#v�y$����_��k#V<%;�������#gG��q ��\�^�4�nd���� /l���������
p����F� UN"B���!S d��h��<�cv:\4���1�� +��5�����5S���?E�N��5[��v��� !�&���&�oA�D���[���/+�������BUE����x����Z���~�3g'iv���s�t�j��^yR�Z�}���-[jjj233 �I��P����V��&�� ��+�� > ( 2�� L� �f�}��"
���� ��f�
K2^*L����o 5#6���S'g$f�_�'iv� .K��K��)uj�b�D]��YAiB�SKl�-}Y.��������o>�c�f��S���[�����|"2��v�.��
�d�R)T�u�?Uk�`�����
`�C� ��(#�o��(h����&� g ���f������)��_�G;��P��4��V<�$����\�bX~����h�g� k��/ �]~_|��_|���!+�o��Z�H�c��_` � ��L�����{�N�6m���������K/Qu�u���� B� R#���Fb3�
l���Sl�I�-��<���K$���C����6�4
��K�W� }~���������1��%e���[�nmhh�2e��{�&�~s���
&�X����h�~��G���� x`xx�{��c� ��� ><o��
6�. HPH�`� ���O�k?��
�aR��;�d����`&f(�c��v� #m�'�,�wEg����*)��� ;v��W�:|�p<Ed�
� @fA��v������^s�5>��e� ���ZM����:;;���y $H��|j��vZ��J�z��^����=��R��{?��(�r�� �yd��`&��z�&��v���M�OP�|X����W_}����������b�Exj���I� ��'�\i0n��|)���Q��E�555-]�4�w @�B ����U����V/_�w���G��E^W����D��#�H3��RB �v�5Q�B%�[��k�������������� �G2�� ltt����������"70������? ��3��hg�|Ak�XY$4g��BAX:+��� �R+H����Z��������� fb
�&�|`�Anu}�c�r��
'�>���S�$; ���_�������:u��{ aTVV�{B{k��U���ngV4���h��Fs�j 0���&�O�^;����)������:S)�����/�]������+���_���{s��G6��L�t���_���'�.>>�-�J� �J��La��P]�c.u��8�J���x����z���~��ib� SYY�/=B�q��
���jV�j��C-�� ��1�HXAc�w��[�DyY|�/v�5�=���]$pLDp��#�LIYmSG��K��!Df
|���I&���n���o���j�}� �m���2aXY���?��OO=���C����o�� ��*(���h��=W:::���"y 5/#>^|� <��9��w�b����!w�E� �WkO]��_���%�a��w���>�<"�s������3�lf #��)n�}X�����w����~v�����{/ 0��UOB�J����o�/ <x��� �. y �gG�R�-Qtj���C���I�w�Mw���~�0��$���4�� �[�2,\,���������,��\i�,{��y��G 8p�u��� �� �<���6lp:���w�}w�������� D�������fn ����;/�R.Bxv�Y���~(Bz��P;GB�a���m�(����o�;]L-��&��#���������}�X�D�� ���{7l�p�����{/ ���|��r�\�rhhhdd����:w� !���������n��������+��i�����<=�w �����~����m�R��P�;R�8]��O�0�����hq��i��c�mG��Ha�"���,��[���%��)j1 |Rq�d����7T_�_�*�������>��Cuuu3g�{/ �P� !��w @Hd��V�]�{W���%Wr�|�H �U�$������ ��Q$(�~��&���v���}]$��rm8���}n�{���}�� O�Ja�����
o�R$B����0�T�4�����jm^i��'�����uF���E�Wn��/IwW����� |��}����3G�� �8���.�L ��H�2�w�����=��k����uX�.�3_"�������j��zu��S��>���hi� <QF�M��H���2BS��%Q��`>�����B &���������{�����?#" Ro=e @��w���*%���v� �?��o1��p�;��NU��8k�2rz#�a�v���M"�'T# v�J;{E�����f��?��;����{ `C� ����;d��J��#��|�yS��\LQ� IDAT������Z S m��5I%�t|5gq���� �vQv�� ��9��J�:fw7�6U��3���t��,)�+���>����_����~7J; ��K��E�ei X$ p�i4Z�y�z<))BQ�����;�1|��������<��#=f�o"��k�a�wO��Y���(_�m=���;&����^�����%S��e>���U�V}��������G �
i>�KqO�"��+ � �������0 X���d=S��������������OTm[>�s��4� ���i{�9?�����~v�|�=t�Pqq���{o����� dF��(_ ������[�����I����T �������#2�3���v�����In�5�u��BK���TQ�C �A�J�y���G� ---��g"2 q� �;C��>2�VmSG���M�B���+ ��7����b�B���_����Y ^w� 4W\�^����~���{��K�.��p��J�.��q����������_��_��5 a`�� ��*++�� ��5C����"<�5�|Ns�� ��[]o8i�|k��3LD�au�}������>(��J�aT(�� c��U�����������zu���^�������466.[������?~t7 rPYY�~`{/�I��J^�Y� 0E�wo�w�� @\�M ��|����z�h �7�����VR *������,�||��k���<�r���G���$.��K�� ���{jkk,X� �\H��]�{�i�\ ���#��Z73y^4�SbG��i����5{����>�����g��y�;5]W��Z������:_�Pd��x�$f>`%�XZ
#MN�Q��J*�������d���w�}��`X�h��{ )���;z� @<�}��9�����9k�j�$�w�h #��c� 1��SU4c�w ����Y��`��s<�gB��} ���Z;x�����i8���}<���zb<9y��=����k�!" ���2 ������B�� !��� ��=f��E���;���������7�=bJ
�k���2Q�V*�L��,�U\(��4��R�����������a��F&"s M�[^���������t� w������o{/ �A� ���\���hO��� !�X�O-�������l�"#O>sa<(��7/�K;��P�w!Rq�� ���9�I;���}e���S��~��m������ 2e @|<�F�� ���0� M�������;���PG��n�;� y�>}����3m����o.��LQt��$�{R�{(��_}[�#-SO���Y�|C0>i�JC������!s{$�����O/]������;�{/ �@P ��h����� �\M���S7& ������,
M��������$^��� S��O�����0L��M�p�2L3� hM?���B��)���u~J����c��5�I���F��C������R�k���uc-c>���%K�����w�u��; ��2 -�F���!��BhB�G]5�
q.�;�n�rK��U pq�� |�d�3��l���m�4�%�f.��$J��vj��B����4���:�f4�g� 6l%��B;��[u��.$�477m������{� �C�_ ���gM6��'vy���uX�� j���0�8M[��q ��� |5�)K��gOn�� ={r-�����������S�T�6��n6��/���E�����#���T%M�jU���k�4����0�fn�� _����b�"|~\���LD��b |����c���[�l���~�~ $�2 -���5������(� ��K�v��4N�&F�(�}&����!+���;�i��on���6s�h(�f4|��6�u:L.B�p�����PT~IJH��M�I)������3g�,Z���g�-.,� d� @�p[����sw������S���|s�j����X��3�F�
������9�
��~���b���jb'�� e��`�O�Lnu}�c�r��
'���I9�D�*�n�f"2 ������P��������2���p�,Z��7��MI���- q�2 -+��_i �{T0������c'����GBm�� oT�'Ei���Ud���R�s�:�__>��Y�?J�M�_�L�f�kf� )'�/]e���^g���T�u^�����[�L4��T������W�\� � A �& ���� Q�/8"T\�_ 'J���vZ��bd�r7�0Z3 ���IS���Q"� V�Q�-�u��&�<y
m~����0X� �)'� ����8)G=x�������a�y�� u4�L_r�R�����[ _�jU� �,e ��$�� �D>I����s�n�I�MH4?�8�4�j����xDVB����=���7 ���g�G���H���2�jR�z�;���+�n���Y?���C�u���E���+,,���X�zu�w �2e @|���9�$��>'�tq������y���:��*2���)�Q�5�U�_L|���%J]VRA�6��p�XQ�n��rR5�z��+�� ARN"^�J~I
��O*.��v}����+��C��a��
�C��W_-^�����.++ �� �� H��Ha���91��� 8EBi!��
� �"�Qg\
�d���]���4h��6+:�~�h2���� ��hF��M���D"�t�������� �p����z��?����� G� ��% �"���|�����C����S!��y��P�*���e�B������G��p��N�M��L��
����e�l�/I)��L���PS�S
k2�H9��vjI��5��)�B�J��rcMH�*����� }��� zH�� D�2�.���{/ q"�(k�95a���Y������3����N�7n���Qv��_`�xiK�a���N\��8���%]�������_������_�G�{m^i��/'��e,�2���Xjoo_�`��~��� ~X�� �\UVV� ����o(_ M�( �H� ���eO���T���P/��v����W`M_d2T ���[���8p��d%|i��\���;^��}���z��#�b�j�V�G��DCGGGaa��������� �Xee�;yB�q
�Q�C 1 3ZY����������/�di� <���[J�
��D,�w����O�Y��$��0���,����yw�c���
�P�.�iJ��i������j� lX����p[V���H4tuu>���?���{��!��w�/ ����������S�T�N��V���a_ bi��Fw`�B�M#�7�9��f���V� ���{bEd���a�tc�;�Bq�\�n��"����^�`�������@��"y�f�
BaO���o�SC��^K���V��C��h={����|d�L��q�9 �h�p�z�T�H��z]�/"� .���>��S"n ��4 ��) ���������������~C rT��Q�����lw��������!�����W�~�X ���fWQ�����Z������s�tx�r���l�JH��kM��_{�C_�V�N#�_X.\��`����bDd `�@P b*n�F���� ����{�� ����2�8,� {2t�qC$"�|4n�S��/�r����I*����Uz�������_� y���})'E�����Y�p�}��WQQ�[ �H��;�"� ( ��vZ# �F�u�G�\��h�
����)B��N���0l8���=�i�^���4��#,���F .��D������wG���Ms�n7= �(%���b���D��K?_����U{5-]w�����B �M'����,���9KJ�
<�e"2��-���W3 �0H�� #� ��F�u�����A�d^l���>krGd!v���!��[���0\�8Y+}���I���5{��� -�������3 ,m�����k�Mi�?[Z�W��N+����� ����A#�<�������\d��g��k�}@�����Se����6BH�H��=MJ�^�����/Y����nCD �!�/ �����I�s��=,��E�@���IM�V�Z���Q�yU��'����>Q��h�P�b�����w]���?J��eL� ��[j�:��l.;=�fk(�l�5�g��VC����7z��ZZ
�?Z��p�R���66��=����/z�n������6�:��500�x�����g�}V�; ��2 ����h�I�2|�R�������%�~�QR�M�8>���*)��3Z( ���N{���!���2&������������(s���.;�d9������h���N�r�u�Wkmj}�����������[o������v 2��% ��"M����������MCSD�N4+���{���')'�Wo�������c�
���K����9Y1��-)�L�u�:�(%.n|�[��G�w��;k}���B8�nFNo��c�`L P���h+)��i��D��@� ��S;��� �^�zrj���`QQ��y�^x�� /� ������n6
�*�����)��ew���$�$@'�h�/n�����8n��T�X��swB�N9��+/���:[Z�n8�.��� <g�\��f�b�( �C;m�cB������2����D[QQ�M7�����.J7 ��/ ���4kg%g��*��S���|3'$����"|:��wK����4��� ����=��Or�"9Q`c|������'���� ���?
]+��o����8�.�g(����������$S*������{�CD �2 �H��n6���.50���G�A��sa�To���'��dj�0��^�����%)����/���u�:X+4M(�� � 5H�4)��������ZI���[�n�(n�
`|���ny���s ���C$*%u����$��������Ri4Z�y���L��he�V1�ZA�Zr%��cF++~�~f�)9�"THK��M��V��dnBH�����s��&LM�]���8f7 ��� =�=�C���?��� ��4 ���'y��� B;�6|�+ ��p�XF�!�
G���X&��/z}�{�H +F����E���!��6u��hICQ���x�,��[[ZkO���|I�a��q�����O�{�{��������l���;rss�o��P�� ��4 �Q� c��Z5E��in�!$C�d���{.�V/Ac
AkyD��+"��Oy�m�(�{)&��o.��LQtI4Gh���V�4�����s'F�� �/��]c<�)i����������y��S�LAD ��2 ��������A�y����y�\��^����� �)3Q �����F ���eP�;��E�H����Dh���jn��4%m^�8)P
�b��u�]W_}�� ;� ��?@]VYY)�^ B�z\����TH��vZ��J�z���v^z�^1]�Q�������gL�{�{��K�i�B-��4����g����������V�N} ����sV�*U��E � |���<R��a���iJ! ]�|yzz:"2 {����v����X }�2 G��u.����h��<C*�(���n�� {��N� oxvmSG����~���b��v5�C!9�� Y6g%zj�:�v�z${����4��R��7?N�)�g���-[�����[o)���w �SYY�~`{/��| `������BA^�]s]2�}�C�g��zp�ux��0*�$U$��l�uIUE3�{���'�6�h��m<pf�5v+�=�t9��I~�_xv�i1 x
�.M
{o2�Ddt:��`P��� ��p �X��nj��f=Q��1�Gy�g��S]��)�F|'�.�Qm���3O<��J
�i����T\��K�MR��,�`A��&��bMM
���kLK�`CY���"� ���: E����\�bhm�8Yg4e�&T�YR�W����b�
�F�k�.Dd ���� �����{��NMY��!��S�0�%���Oa�w������� c�TN��\��s�_MP�����v�n2/]`3|��
;5����LD��0�>��-��L�'����iBM�F�����hR25P\�n����?T�T�� �?" �����#}C�c/#�)D��"�@�0�;a���GD��%N=�i�
�����C����4��ny3��� �4���z� ��~�R�b��6�: (�Dd
��o��V�M� �� �$xLA�T���� #�G�&8"�.����Z\4a�,2=�)�dlv��d5�}��o�����}���<%�����V�%�\���i��� ��<��jN�s��������� "2 A!( >D#� T�K����#` qEX�T��Q��s�n� =���&=���7p�c��]B\���|v��[5�a��]���+�n��-�U�z��� ��F����� ��l���+���j�������?111J{ �'�
>��u���NkyC_i]OyC��N �b%����O�K��M�|� ��!��](5��~.L�(�+3�j #m���t����E�/I)��L���PS�S
k2���o(�bx3����/��mi1��da������0��$�rt�AQ
�l.��{)��y���_�x����CD �'J������� @|v�s����;sd��dq�B h �7�`��fLM��{ra���p��Li�=J{e�'��e�g3����4��y�U���egC�[x��[j�y%�n���KX���N��Y�Re]���]�����y�W%��UW��*#���n��u ����\�p��>�h�� �H��(_ �4a##����.�VIp� w4#���r7��y����,�#�l&X�j!s{H {xv��rt���{=WX�I+2�4�*b#�\���q�7�������Z�fMww7"2 �B� �t5�;��{�N��f�R 3FZ����k�.�S�$l�`�VB�7UE3��c�
HJ���L��,�U,PC�H���OT=6������j*]{E �g�[�s���6�
�OU�/���/��Vi�up'�a���:���.��� ���o>���V��F�UW �=d� HW4�K��<�nqz��r�F�u��b%&��W+F��X�� ��8E$�Hw4����"������������vI�]5��zwL�R��foi?�
�n� t9)B�r�yx*�Z@S��d�w��<A'���M�2V�E;{Y�����_��C=�����G ���� �� �.��R������#��pC?�QR�L#B%��D8�� ��N� Nbm��ml6���vj �h�~4�!m&����n�&���
��Si^������gh�����-J�A;����;���� ~��/����KJJ
����� � �� �+��K�\j�5��D~n��dw 5�(�*����1��>��}�������'D�5�)��In���H��.�����_N^m�X�%��ns�&��c�!y����PJ*�O#�7�+����O���i�d���4]^^�����G ����!��* ���� �tEc.5���r����g8��L�K�i�%/�E�B8�*�PmSG����~���b�Vv5���
��3Q���mK��gOn�� =���;���������\$���� EQ�� �hU- ���T����� �F�B[������(<
�4(��o G� D����M�G����1a2�����3�~��}����Y�o���"�RW�~I��5�T�K#$r�.-jr���.��my�.��� �__�;��n�/�)A�;]=�vYBo�4m������(�����'�<u�TeeeDD�O(��
�y�� ��)�*�@��Hw���{��I��C��}����& �PA 4>N!t�7�����]�� ���nk&� w�SJ���n-u������L�"+?Z�l���a�.��y�kx��?x����P)B�Z%�a4"Gs �i��L���=�55����+&��� 0�����%<W� � ~?�S�ZM.-ER z���@����p����{ .���2�u� �������a��R�i�X F����N���$uu��V){o�X1� ;G8:����"F��;��� �j�:�]2���d�m%C��SE�c|4��x�e`M�Enu��hc ����3Q��cJ��Z��ce������111�}B ���� *e j'�. IDAT ����PCx�vC��'��&ij�
�����nEL)����!� $J�R�� �0[T�����U���� ��Bb�k��]�~�E�#�� �RaM[��%��
���_Z�T�u�y���|B � � ��?� ^]�.^����#1 �Qx�I�� 7 �U�Z���Q�����;E��j�������J���T���i��'o� U��U�����.�[ru����]� ��K���gb�� � � �~ ��pXo}�L���t��k ������ky��7�y��!B�=<�{I���7y��8�7/;�t����\*I�����u�r�� ��-�a��2��g���Z*�]��sCC����d)QE-�^+��X���;�w]E������ 0h@� �P���) Q� ���N��$W�<����UK[\��0;�x`����N�k���5I�(i��7y��|�?1Y`v��R�|���5}�� w��1B�����j�u�%>�W�����R���(DQ��.SF6+%.E�v�����L����f _�z��]����bccC�� p��P u(>_��*������`vr+X��� @`�o^35IEP��5I��k|��o� ��G�7 �e�P�~�#`0b� K]YwM�� _} �dt�x����|xw��[*�
vi�T�D������#��2{s�5k�|����� @0@�R W�]���� �@�)!�(���4�Z� + aP�e�'���� cJ�j���x�T&!�5�e�"����,�e7+1���%��%-[�s�0������H��4 �OL�P�k{}���\��!�}R��kGw=E�������c7����RWf�Ho��0T�[����YyQ9�Ii
���HS��&e- ��2�d2<B�[�!�����>�Hqq�����������P? �m�������,��q#���[� 8����3�6��M��v\����Cz+V�R06��d���Kv|�n/�
QI��A�n"���m�}r��E�n��38#�U�w����L\v��n���F��SD��+oQg�{��w����u����>��Xt.K����.��&�KB���m�:3��� +vK{���)y����1!�Tf]��o��� TWW'&&��� @ ����� ��)���?t�X��$9�dJ��";�#I����qj�_�����H�;�}Nj�ON!������?=Gt� �D8�������S7��g����3 -D�[���x��"�J|n2�>��IdB���}j�
e��j�NT����������3z�*���MVy������E�)����z �p ��� � D���}4{��7�
C��9��nr-KY>6Bd�rHo���N�������uU�L��8�9q�2�zY���yg�|�y;��������v���n�sUj�:��Dk�P��=.����G��������|j��N&���&���l;_Sp����� ���Z]]����oO �?�C� .���]����s��W�/��:�����c����Q]��_����G�z��!�����/6 +`w��� ��M[g1Sua"��<��02M*��hS��4'������.���:�������L1��� 8p 6e �p!���g�]E��v#�������wc��g��D�l�����������$�����C<VDr��?n���*f������b��kZ�P�7��Ms�����4����s����lz
��������oP# ���;KxV@ �xKQ;C�gO hku-�I�J�<������a�?Bq���& |p����$�uL{��I�SI��U�6� ��)�W�v����~87� ���N~O L������������e+��U}��q�FN�$~k ��� B���Z�y� M���E�� !3���lmm���D���G^r6��i������l��i��u���iii��� ,���{8>���� ��Lf ,oG�L���� ��$�*����C�j��������)�i\�n���^O��Iu�n_�hDm����<�W��K�+C����N����z�����Z�9�lE�b&n����0j�B�3��7� )�X+��JMN�9��i���9����{��W��������3 @�����~ ����/O��"<�eJ�j���x
~N?~�uF~�� �m�<z������p���v��I��� w��V�7>f�W?1� B��Tl0�!~�o�aG'~���X��H��x�-�We��V�������}��_��K�cG(����`s���VU3� BH+uNP~�|����)>:uhd���n�� ��2 @X�s2K -�UI �K1C^�&�����s�����0�����T)V�N/�*;��&5/kJ� q�]z ���b�����l7�}[���}MP����r~�� ���rJ�"�9���)M�Z(���������u)xpTeu5������b������pO���9G �/��\sc���DBN��b��h�T�� ��_ �|%������Z�> a�)�i,��� DW�E��x�K`�����q�~�Gh�`���'pc;I1o����v��{�T1��!��>�F������|��L ������"��.p�k#��& ��N ��i]��^x����Y�AH�)���H Bh�����P�Q�"$�b5n���v� �Y8�T
,��� ���MfQI���wn ��a� ���0p{���uU�]#f��K�x��`a4��)��
!�F���DF���b��m�i������]����OH '��* ����g��${D���]m.G"����=���Ren����%E�Z����
�� �����pi�"�"2�2�O
!�?�C� �|�O�r]z�s���f;��;I����� ?�����q \�b�����{��������5�Xk��T���pn�7���T�`U6������a�+'m ���.�� o�
7� Fv#���&W�j(����r^&�q���;�4�����D���+� l� �nj�*H���)�v�{H�P��Zz4�?yA�>���;�9�K������n�f^�&;M�x�s� :vY��I;�� UJ��]}������G��S���;H����/�v� �-7V6�=M]�������� ���>��������s].#cR4<��kf�53�_o�������������z�e VD��Z��# � �w��;�����+���+
+��6��I�������G�g��*��q�=&��RWn<Z����H����x��==�; 8I�� ��T�'&�c�x��f�������{!5�T�����j�:������nkzmmO%��W�2SfB���+���k8�W�+�0S+W0_j�����8�����g���o�
7���"��g$���Id ��g��;�* A� D� �'�^w��� [o�������?f*P�/
��!�o���j��������8�I����e��J���Lb�!D�8�r�c����������I�^5��4���s.G"3�]���V��V����o�&���]VWSt�Ro�L�Dn�e����*� !�n������&��p�� O=���}���� d� e �` ��?-�}�L���pw��p�M��p�`���q����J!D��PC� �wL �[���%!�d'I�����h�J5i��v�
�Hf�� (��6�"����cLx��%n���5�>�2�T����}D"'�\�����uK7G�� =�@� #��;��t��I;w�|��'����o���� X� ���!������c������%H-�� �h^��o?��,� ��cl�.�(��@��]�"�������b �.�j�m ���C�:����%��9�
��Y&��d_�h�Wg� ?6g���K'2��2�������w��g�}��c����g���>�) ���P �P
a�-�0�x�x�T���7^}��\V���>��@�H��9������ H
oQ�� � [��T����3�}�W���� � ��(B����p���%+/
Q���0Y�rd�/S~�����lNT���
���� ���|���v��=q�D � @0@( K���p� mp r�V�����#��@���� ��D!�|Jx���� �K3����"wq��e� ����7�w�M�1�
��V���O�����������YW�7h�n����{��]�t�?����o���� @� ���!�|�6<>��� k�!ZQ�F9a���j��C\O>�QW@�)l�.6G���H �A��P���&��������
��X_��������m�/�����)b�;���.����Ln����G wCsx*\���j$������!���"���_}����>���_�r�-�~0 ^�P �
��s��~H��@�����Z�;�.)`�� �[7���o��RT'
61 <��D��7�
H>%�VI`�����=�������O�e}�������}�`��� �91�k�4�3E8BmQ��g] ;"�� 1����g �H`oi�.N�A�<�H�TQ�3;�����d��/��b��I�~* ��P "��� �F��)]�NB?$�01b�nps�����X$B(���o��[>�mX�$��$�� %wq� 47����� �EN ,�����v5�ey����?4'P#��EH�)��}D"M�H�;���������s������ �-I? `�H�JJ0_�%�b'
f��������CAoL�������`�uEu[����)��e�"�5R����H����c �����?D1�TyMc��3��f���������8l��Fi�}�&�57y$�v��p��{��A l���Z K]��"�e��P�n�+�)���yQ�Ke�4B"����&�z�� !�K���o�*j "����x�����?}����>x���E����c���!x$ >�J �Z��J�H���Z�tw1�,� �W-H!�'
�E1>��y;#F���#����5�|���� �p��h�jR����[�$���|��D�j$p �����vm����d���c>�+r�w� � (4��<��O>�d��i!x ��J �15I�vNl����sb���=������d��k��� ��b��b�* f��OL�P���������J/�*;�������4n9��$2"�LLY�����/&�}�1�@�@M��Z ��I"?H e�E��&E�)$r""M�S�� 8�fF������� ��P � 7�m/ �b��y �����C�0 |�2 �5���$1�*Y��5�x�4�#��
����oc{|;���B�W���yc�Rm��1�<�����"Gw=v�wH
/�%��
��k4�����syyNNN�< ���4�o������Y�v���{ ��!�KX
�z���]�,|�����g�V����������e�V��-�~o����������f+#�4gW�z|$wo�zqw���8KA�; ���L�����xt9�F�!������*��-7V/wi8��� �}I"�������;�5�%g�gd{�dd�k��.��$�4�G��g �c��������a�@(�/���C �KXB^.b�� 1�k��� ����-f��`��R&���#��x�n��*��&M�||������."Kf�1� ��<�=N��Q��3������6����Eg����+�u'� ��d�}<�\�i���L����5G+��T����.���g ��&<x��% D��C������k?�6P=8�K�_������DG*�f���x�R<k������J�K�����2��(Psyy�9�ti�������������5{�l!d������1������������r�[S�B" �2 @(���E�w�������]���+�|��+^��QX������J^vJ���W��jq I�]yq�GF+�f���;���P���Id�C�*7�s����X� ���� �����z[ua"vc���$�t�s�����H�/l�b�����O�� �} �X$d��n��;���" �[�?�e0; $E'G���-�]*�tEL1�Hy�)�����h�RIZ��t�8�ch<�:Zr�/��}������escp7^s+\���e��9�Y�-`R�0���{`l�5 �t��H;E+��������[�Q5!��<�������d�����v;���z����r5���?�S�N�s�=����$2�e [�k�Q|�5���LT �P) �7i59�+
�)��v��!N����'�������+�6=y�T��,�4�T�h=�{���F e�E!�r��a-��
VD�n�riM���n{����\�e����lNT��{�%g�5F���S�f�������������<�� e �P��"BL�G�[��]�|Z��M�����; �R�� �'�&c������ m^��}�:#_`v�n|��XSn�C� C���Xb�����(��$!��Yx���35����7G7h�$Bh���aL� ?
���O����{�y�����_�C��'� �h_ Ba�H�JJ�����)��e�"�5R����H����#���N�[P,��G��'����E�����T)V�N/�B](���>�BQ���|=*�i,�q���lw�t*���g�.��4�Y����O\�"�Z�:3/jr�L�FH�2]Z��Ro+\BC�*����G�n�J���@�(_���2��D�����2[����3gf����?��� ���u�1 � ���E'� A���$��9�e������X�B?�����p�,�%G ��
� _o��K��aG*o e-����&���/$,�����g�K�+����rJ�"�9���)Mb7FO���+�W�G�&)�($YiC|~���{��g��
����D>� �( ���% D�EEO h�j���C45IUz�����?h@/��]J-~
��<nw�671MC�/�F��v@�8��r��Q�����~l�\~�R4}�����N=�U�l(�V?����3��_�pa`��&�.�3 L^EQ �� j Fj�=�i��s�\��,���mo��^�.��� �����N�?���������O�M��_ ���v@^z-��)?#��P��
f��V�x9�n�U�s�DYz�&����������1���^��X���Q �A� j��2�Y^��� �x�R|
����x���]O�vGw��h���6���Y�������; 77 _��Yz����������b�i�.���� 3&0������g$���Id ��g��;���={677��W_]�hQ` ,+?z����lc_�
R& � *e �P�����j_����]�A�C4���� w)������~���|������jf�B����O�Qg�!O�����B�*��}�P~F��lr� �f�Kg�����;W�^��#��� ��2 @�<C ^�� D��y�tW�f� LyM���L"C b�����
�q��9�s�kn*����$p���5XO?n���[�6S���4N��]������9sfQQQAAA�� @�A� j�]�$B����m�|�!�>��z��g� f�����5a����u Q��W{��{�4�����'d
J
V��_�����;6���r��s�F��J��?��sNN���+ ��� B B �� �@�I"����x�t���;������5���:$ ����x��kw����T)V�N/�*;�ww!��� �RWf�Ho��0T�[���/�6X��F"���� V��� �!�b5ns������������ �� �'�� ���i6��
QN�|:��a�LP�if
` �J#uf "(�9h�����L=f��S �5(��dD���� ] �(��k�<5XIm���9G �jhh���y��� ���< ��� � ��i6&CYTy �7�0���� �._�o]�eSOr�����6w5bK��/uF>�����Ndh�N%w� �p�"L`�0BH�Iut�c�/���O�[z-���SN���8�K�D��g�y��'��h �
�/ �Pv�p���a�����S�l�����N��I����� IDAT��h\21EL�P� ��$����-[��� Z���tc����h2hM����t�g��6X���,�����ulM�}G��������o��%�P�}M�UQK�oll���)((x��'��p � T� 8n�N�����-[����w�*�k�������{�����/����W{�BQ����5��;�C�xlA2 {���;��)��(k��%�OH�]j��������HL
�!�>T��"��C2
Q2m�.�D��P�w���rcua�����V]��"�����s5��}I}���[[[g���l���+W��! ��2 �������4P�����T���l��0s�V�N��*�wZ�#U%�G�OL��4 � YjK�_>�P��� 3��������f��@G��&#��$�n|��X�4���?G9���]#3�u���(a�W��� �a"��i��P�����D������,�8;?������z��w>��#����C�� ��2 /w�|����Sx� � �� ��b�&��(��� �K�d_����4Q�K�37�Do�����D�����x0���3���0MFb�����;�}!��P�����?�W�����?2�u' �T��6�P}W��o�={��
�������������� ��(S��n�/���� 5e R�|�������6r8����|!q�?��oLF�� ��&�d_����49����=������U�� �=��81MF"���3�;�������?����5��o#���xu�r�9����o��S������K'2%\)����7���8�LP�����e�Z?���NL�"D��s�F
A�� �� \0� � %<�wj�����m?O��m�0���?�^Eb��:H�`v���,=��|����CzQ�bcJ�j���x�431>glfb���d_����Y����T�]�� �%�j�%uV!�*r����� �3���_HXl����>#rL������N��*%�a�i��C��m�����n3
<��I��d��oqeZ�����
�����N:���sh��mk[^L������P�������aL�p�M.���m���p�q+�n����F��o�6�,V�� =< �p �2 )�4_nL��
��yH.�UF+yHo��X�w����A�������r����(&/;���������s��w)�NH�I"�$�z�r����G�c^v�O_��g|��U��� �]
���j��*G�����-vK����~���>y�����^2Tx�@�++/
Q���K}E7 � (S[��r��U<L! ���08 �����#�R5���.���������J�_'�������������!��5��? @8�P � ������ �^�}Fb ��wP�o� 7�����kP�5*�\�E����eS��� ���
�K�e#���F ��#�D�3���W��
$b��:#�ch� =C�:�@��_ k8vB(��=�K��J r�����,��%�#x���uNt! /������i�}G+�U�P�d�K�RGG�]w����+W��N�5��4.P @��% ����Z���M�] ��T6;������O���$�!1 Fj �A���XOP�����?��p�:{ v'IG'�j���&<|������D�FO������)CE���o��L���I�J�[�:� �_�����D�F ���tn�t?}����'�}�\����_�j���Z�B���o�Qb/9���t"3}���_]���3B��Z��' � � 0H1�=d"^#]6.�����x�N��!�$�"�������`��\�����ZRvB�^R�X�;���������3e� �s_~��������6��Y�T[��5�� �w[�kk�z���K^��)Gw=�c�r~����3� �| ��b�� ����m>O���s*�0S+Wp������������u��� H�[��F�q�����M{��7Bru�&��,� dY�&�����|
� �� ^���C��gD"�_i`����!_� �4 nBA�����.����f���v�l�i�����jw��DJ��f�:'������0k��"�m���Z��f��J�v�E�K��i������V���O��(i�r�����); =��
���\�CY�6S���4.2b) ���� w���[o1��53`
6 �R �F�{ �G��VvtB ��/�JJ, �� ]��~
���
#�U�w�TF�i���m\vB���KTA��[���psCK]�k�/��g �`��V){o����) ��5�wI"�F(f����y�}�X�������G�^�e9�
���&��;?�����Xn��*��Z�x� \����z�G�������0�����#���m�w]��o� ��Q��7�@����s�W�����s�����v����H �����LKx�� (�t�B��u�Q�V������\8v
3!^�m��]��g~tMR������'��I����;���q�k8�i�Xf�R�Jf0����
��������'b�0� !���;g��w8R%q�{�%/� # ]����"3��� ���Dn_��f���FD�<pT�b��������4�b���?�p���dvJW���A��+��}��� ��L�{��g������.Apg@ ���� �� ���t'-���
� ��F����/Z����bl^� ��_��O��z���U�����5�Z�]�
`��vt������n�6 ����4�x�r��H�4�Q{��L�;8F$1�0YyQ��M�}�
Sl&�J&�<����~4��q��Qq���k���~t����zsg�&�d�lw���9sF�� � ���#�����+���+
+��6y��lK� ��'?
��Fi�}���U�^�5J�|�X��#������+�w� v3y�Fg �F�Z���?mao�Vg,��s@�e ?��&$�����HS��&e- �@w��N:)���ZGt^X��m�� ^����^h6�����#F@" A(�Uk����g 0P��~}Xo}�L���t����|�L�!}(r��( ,����N)�7.-F#�J�b4���-����){�
axo(��H���lZV^TNiRD�B"��8��yQ�Ke�4B"����&������/!���Nu�����������rPR%q�+���-���f/IQ v�{iB�o�b�����M�6I$�}8 ���k� ���Y��cK���mi "��s�:�����)4sg�>��m��f�����������o[M#��]�� ;G���+tU����� (%R�^'��H�
��(Hxg/:�i2�����������Y)a (#$
�4#�$�(U��r� �y?�w ����s�u���>$2 z���;�� BJ y����'�{����L� �����
-��lOKU��dl����r- /;��K'>D 0Tx�w+���Kxm�,m� k�/���uq�aw7p6d��;���TM�K�g\���NdB$i4w�����c�
Q��Z �
=, e ���&�������N7$�N (����UK�����
MIVQ��$���A�!�w��P�]�� �P�����k<Z�(�����+@����u��_�_��<5z�j��B��Q_8���D]���;�X��t������V]��"+��0�� 3�U���
e����P�3J���U��!�� \���zg` �
( �����)e���e�2�A����U��C��a�u��S��!;l# o�\3x�]�e��;�D�%Y�o�j^�&�� ���m���J���/�!��W�������B�svG��:���jD����\����@O+!�V��^��w������a-q��Y��H�\)�� +<x�J �N?����?bk`uR=N�{v��b% ����L~n���k����[��JlJoD�%���KLwOV^�
=�w'qW+�q��GYyQ��\ R��%Z��c ��z[��(�7DR���A�8f� !�SE,�C� b� \|��?�!���#�A��;���B<v�M`#��*�5
��+����.w�#��%�3�y��������h�i,=J>G]��]�?�Oru���~j�S����#��r���+J ����W}���J(�W��&�r�����t�C9Z� 5!W{Q �Zc� .�?� � �-���k�H���b��M��m����m ����x!�� ��k3D-e?��&/��������k��u�k�u�� y� ���x�,ue�����
CE��������[�� ���D _�*�ZFmY�3���� �]<k�V���������T<L���~X��Lz���r��ON��?~��1��!������z?���*�,-��F�� �%�L"K�D?c��/S\�B��j��O `��J ���?��p�"�\F�8��URb��:R�A ������}/��(�BYl���4!N�� [��k��H��}X��[bc�+�]�g�����T����v_��A��������Uz����}�d����9�
v�C�it���e��������vjav��{�O��x����&�
!��e�����n_��u������zv���T*U���-�oaw���? �
B ���?��p���pZ6.��`��������>^P�M��n>��=��F<�Z���=Eeqvj����u�����29����.jqI�J^�o ;���/������V3��w�O_�����&9�����]$R���g/)Fj86!�K���EQvS��5�B��&�Kj�v�e����mF����+P|n}��efS*:Q�7w&k"K&�~��q�-��������J%v>!���z�#q>�; ��B ���?��p�Sp"D,��D�!j)�P��M�I7��;���B�9�����5
�f%��ago�17�����^�4_���{��*� ��c�����j ����]b��r���pl��#r1���;�hL��� yo'��.�s(���V1�������h)B���B %\�������
�-�/��������� BH��= jU���>. �p� ������X�!��P�)'��D�R��Gj)������$�;�x ����/W���X-K�*e���3H��K�L�smN
�x����4��c�!vq� ��[�pE����N�N�}��#��%������v+��9bc� �z?��b������_�3�yO�[�� �k4�~�H�����7�AW��l_R��g 0�@( `���g� nx\�$���MKU��d� ��M��$���C���e���S��.jq I�]yq�G]����6 �J���PvS�
��i������E9$u��,�Z�KwC�{�w4�P[�������co�A�� �� �a����@�2�u' ��N���n����� ���n�4V�f��^��z�p�J%�7N��k� � � <T�Up�)'��DS�Th ~Z��\3-H"�v oz��yee���T"��CW6?����sj��Y-l>l�f '+._�I���]�
!f���f��]�wz�����L(���#���R�/�7\eu5K���N��B�@� ���t ���� �T������[�6���z�r@ �` � "XT1a�����$t�B����LHp0��6Y� H�k�I$��!��K�9/��5;�b�S��I�,N����6��r� $\����V��+���;���UD75���1��i
�� �t�r�9�K>��`�;E"���`Lv����$u0]K� @<e ��������$�Vl�N{%do4��v oz����4�nV �������j��t9LdL��U��g��P���;���U�� L>��`V�����4�;���>0� �7 � �H�����l��W ���7
o�B�!�]����g�KD�j���V��l��v�9����L
�F*�kOdW�
�7�7+/
Q��%�Jo�����O8���5��K�&no�{Z�&�r�� ��`p/ �@( ;������*Pi�[)
w�����^ ��k�Be'��~�_��F��!��C��-�T��f�@RE\����!����������Os��r�C�����]��G�9����?sy������@�������=�NY��JU9B���
c( �iS�� di��� �\�r������g�G"M��a��
� !�� � �����t�NI���/Z��&����;�)p�;��y�^-��:�d]Q��_iXQ�v����&�oT��.����p��4�8c0�k�2=�����N)�7.-F#�J�b4���-���p/y ���h!i5P�BE�fD�M4�����]O�vz����6���#c��m����\j��'L* &��}cI7�OL�P�k{}����q���=5�� ����,��$�U.�b�v ��OH+%We4k����}0��4]�\"M����>a�e�����4t- ���J���^�io��� lX�����T��52N�b_��b��pZ�Zg`m��Ni������� ��q�4� @�%���}[������=zX�����_���~��o� ������x����4���~v�y�����D���y���h5n�����t�b���(������|6����8�Z �Q����GKKK��y����< � ����0�&X|�=A@`N �X���SZ4*��98tP�nF|<������4��L�0��W��K";��6�w����`}��w� ���s��q��#c�u.�]E��^�\^�Z�|V^w�7����H�*�j �<!?5�{|��U� ��TLL� � !*������������w� 0����-&�\�@}�1�J�������Z8-<8f m��������S_�;��O�{+&M����i�f"�:��]���� ���K�����x��+}'�"��-r��X�Zf������PF�v'���E �R���� e�=�3 ��*����9Q�(sg���B� ���U�������Hd �MT(�a������~? 0�q�}���N^�q+M���T�����O��X�' ��1�����}wO�Z���:|X,��8���B�Z���o�O��&�����l�����S.�J�"!ATo!��]w��_d��������O���S:��m\lM�x��^mw�s�])
��poo.�%K7��uu������~� RbrJc��(�L� !��b���!�:���^��woUU��!��� ��
eF� �s �k��gj�
��9����Md|[�$L �� �T(��'&_��qOlA��h��L��j��Q���������y �L;�nPz��&Vc�����\�u q���R"�2]�J�v'�#c��n�x��U�}�C�T���D���G����{G>�0iBW�#��we���R�{�nHd �� Id�����!/�Ny�O�(��� �B!#f7����[�+�]�92<��9�}�4���s.Gt���� ���!�A��"�"m��t�p�� ��Z�������?�[�]D��-��Q�0��#���^u��`�J�j�b���2dGk���$k"K&����R���Uv[�?�y���*666x� `��P ����$���!/>���`O������B!�1�b o# |$�k��SHZ�\f�8HJ��"�i���)���/6 +`:��f"nr��p� (�w� ����}�d�h��Y�Ao������!n� W:"� *��y7�9�uK���yic?8{����+f�2��)�Y ���SN������}������� r�� ��z��`������<}����['M������� Ax�{ �pXo���#�|lD�b���aC^�i����?�/��dw��-���f
������b:�F��:��H B"!�� �r-Tq��Q��x ���-TB1_����_�SI���X�H~������w��"F��;���F?�hD���K�5B��lV`������#����o��+�d����:�)�8����f�T��7 �Bi�PR������ �Rx���]������g��w�y����{��'JJJ�l�b6��.]�� ��p!L�;��x���sCoG�^3��OL��������-c���q"j���3W1�*�����7='{��:#�i>����.g�+�w�� IDAT����7��x��������}�`���0�r {w�k����M�>`���L����+&�A���}�� ^��\���(k�f�����=>a�BR���.�� ��|Q��������?;;�����G �>}�;���m�� = �/�����2������ ��� �#�)��e�"�5R����H�������g�H�JJ0_zLy�=@�]�,|I*w-����{������rm���H�L�#f�eL��4/����O�|��� �ln�B�$m ��5��ZCmY����w�mM��-��� ]*�W��M�\�u' �T�w��Ig}w{�����v��I��Z�;�fu�����y�#����Q�m3�$>%#����r��s�F����� h�U���������C����?!�T*�R� �� � �� ��0��%9-X$1�o�9�6@6����x�������VS���F���w���B�eL��#����&�i�,��9 �Z��u��L����uQa���n^n�.l��dw��� �Ad�E�?' ��1�k�bu1r�������7�o ND��nKb� �21U0v�~�Ff�t�#�zE-��tZ���,V������ �;��2������������mGzzz�
�I��I{,� x�Q�y��x{~0�x������S������ �����.�5���O��&�(eRwC�y�1����=W8Q�K�Z� M,u����L�wr����[�<!�0����b�>'���F�Aie
&�)�����F�H�����c���C��+DY$�� C�?H'�QE9
~>$ @��}i��� ���KJJ���$���� ��[�B��`�uEu[~�aEu��&�����&� F���|�+�5J���7*�����K� ���P��!!����� �i���5�����/�#�ZE�l��BGw=E� �����BYd�pU o ���m��>��{+�$1a<��� ��B�c�1�/d��r���&��WlNv|�6f������a�����u*�ck(KO�.f�/}�{!� L ��J��7X,�M�6!��{�� &x� �q�I��6�
o�8n�S��,�����)�s�a4�u�NPa �E.���O�Vb�m��(e ������<DP�6��Sk�q �HDH��xl\Bn��)��������U��R��s.���a�a�$���7���D��m�=J���D�� �2l�|'���$�!�Y?,x�C D��t�P��`� ����R&))i��]UUU���������{��{� �� ����r� �.��q��lc��B��e�'=����(��sy7?< !���S�%Ue'���p'� �����kE����*�[�* �������_Pg�s�� B*���L�&��DS��
2�������F�):^!I<'�NvWq3�x�L+a]�3;!��I��u�AOW�-y����&�E� �x�����<�z��&�
����)D ���)�}~�����Eh ���#B:T��D�@2�,A��\}��� �yW)���t��v���� �7X����z���c��}��7d��j��w���r�V�a�y.<Sft����C�q�f��I����a"O�[���?J�� �� �] ���0�9 o�w��:V����I���A�
�-U�%&��DJ��'#�sp���N�Z�{+`��_�TF����_��arJ�5��%�i�WC(B��Z*W� k� �w�2���})1 �� a�^���}��W#o��@ �#�����>��J�rwzI�����:`�kS�xL2���<����J���o�I0������,m�������lY���X����jQa$����} &�|��wC5 s��ia�����u���3S�3�8uf�w%�NDN��F9�4��
��4���~����)�~�z����DS��b ��J� x@���G��a �q�S�H�2=�}��:��} ��El �SvB� ��?���?y�j�Ji��h�R��V<da_�|B�3]��!{� ���r����>��������2��_ ���ZZ] ��<���9
G�|� ���/�z �X�������nE�yO K[�n�E����s�c�����Q+u"���m�4P��������OPE-1�o`u0X����<�d� e#���-C�� A)�2�b(� D�B���{�n�I�EEE�]w�}�� ]YYy���O>�$� �%&@����D�QI ��C0>��|?���p�L�x�I1jYK�K;v���{$n)������2������=�����Tw�����1y\�����a�-3@�"���: ���b��l�� ���~.����R'� ��_UC.Y� ���'����u�����g?Z�t09���?[���;<c5n��V\k�����o�Sf�.���s5e��L9/�8��� �>&�q8ek���w��_~���.�F'�� Q�Lqq�L&�1s��3z����� NJJz��W���?� �D(��q�r���a��$
v(CyX��SJ�aq|����� ��B�� �Y��F�"��&�9"��I7��x���`��A���)��F�v�q�#��%����[~�:��� $ +����
�8���&v�PV~����u��������&/�x}��b�6�A�?VtI�^U����d��������*�r�#cRT�
�\������0R�
tFc��������;����b���h��&�\��� A"*������_�9sf��5���N����o�� f�5 �`q�@�N���@��N�|\��k����
2 �����&b2}���R�f{ �����%.fS��Gw|W[��D�E9>��2AP��!Y��HB�k����W/#UJ����{\~��%�4l�6;�q����a�&�i�R�%B�>~�����A�w�K�� ^U��x���0���`�dx����~��3������n@�efS*:Q�7w&k"K&�� �?��/uwwsvvv�a �#W��B��'�����.6���D��h�����R�z�RWF�1�����p74!N��[�� fS��;��D��#'w���*�z������D!��&^�}A��2%����]���Fx)+/�� ��[���`���QE-���� ��_R�jQ�9j�2�����x������7f��`~Fv~��U @`y�}i��!g��e 9w�\DDD@ p� �������2����c�Cz �)����������*�5J���G#�h�����p���v���;�v[
>=�&�b���rKi���h���6��7 %Qv�RW&����lJ���Mp�h��:����u�y�)�����h�RIZ��t�����1� ��^�$�BH ���
����:wgK<3J�@h~��X�V�Kv����SO}��W7�x#�8{�{� $��2���zaa���'�/O�<�d��u�������k���� �5�;r��Cr'��vs�!��9!�W=5I�vNl����sb�R��$;��*&`���&���MS9�:I���k���_O�[����X#����4�O������PL����)������|�(���� E��G�4�Y��y��K(���M�4HX�LZ)�&���M��Z������;w�|���w��5v�X�q� ��b�������,��k_JMM��}��/��v����������������
�r�� ���#W��o���� �������$;���
���4�=J�oZ<k��G#��0� ��t�ZM9S5k�#(�BD�����>z0�[���O�Qg�y�p�`6%�{<f[n[���z�]�����n���G������\��T�T����l :��-7r7m�R���2fb�:s���2��s��Ux$$#(�rP�s�F
A�����={�<���{��������e�4 ��Z�v-S<���w� B(..���s0 �� ��� 6��>O��&>$B��f����� ��@K���Q�������Qly�)E�a��f[d��
�� � f�M$�������5)�����F}C)���� �N.�����\� ��u�u��_���GD�(
9���G E� ��4�Y�t|u+��lK����>M��rt�;�o��mS��("G��^��j�B�2�������w��5q�D��A 2
��:��:}��M7���} ^����|����(X�C �$B��������v�@)M��Y�,������~��->j��&��=z� ������������F��� �`��%��\v�I��- �P&�UB��r��B&�`��� f��a������=�'�]t�Ns����mM)�^'��~��+���� �x_������{�.]������o�=��!� ���e.]�t����� �HII������> �����i�Ipc��c#����oE!����~�B_�[���� �����x\� >�M����Z�� \LHQ^����� ����L�=��E�'�t�Z�DE����a���W���F��5E Wx{glo�1�]���[�NH��������~�����r����SK ���P����~�����������6�?J �aR@l��F ����{g�.����x����oUN��D<.����!F8���q�-���!����/lIJ#!�3i��Hx��G�� 2M�������xx��d� �Y�����G*�j� @��^�e���_|���I���U� �w������k����H��Q��� \�OEx�Q�E !��,�)��*'xE"��-����� 5�E-�q A1�DX9L���p�_�xu�] ���������7�-��4C~f d����n|��X�@O��mM��^�����n�o��<S�z�6y�o�BId �~�e���w��q���z��WC� �y�h�Z,�AM�:5p� �/�S@��c�#�o���V� QK�U��HD�������G-;�_��S��$��s[�Q ��N!qW��!X� ��@�5��4 ���� ��>��h�Ti�2�����cO�H��&�w�>������Ho�L�D�L����oJB�-�2wlD�EF��j���&�&��-�?~EE��i�|�P @�y�DFFZ�V�������nv7 �~>�I8F�����������;���_�Dh��4�q�����u q��{�k�r�@M��������n�"��C��/�4 ��u���p&� ���nu���cs��<�W��K��� �\O� "� ��,��3�9u?�UO�!T��^x���P^��g���(�?���N���������'�(�`�TZ��������PeAqW\e��H� x-
� �Q�]���R�z��#ms'3�?�L'3���M/x>%����o������6�2�yK�����������$���)���]. :u���[��{��~zA� �g|e^{��^x����Z�fg����_��`� HK�T��fe�?�f� ��6A���2k����z�7Z���+�:���\�r����u��GM���]09m����J�>�:�0%0�KK}�cF)D�4�D'�� ��/�
����1��i������:����Xv�{V�a d�NG�������R�*J�tAHU�������g��w_[� AAZ�o�����CBB�� �-�9~���O� ���NX@��hVF�6����t� ��6ri���H8���ip��)#�w��Pn��e�L
|���1�����{%���uraJ`��.3ul�U�(#�=
�U��,
���6��m%B�� ���E�Z�����f� �m�D�G5T�M=����R�m�&M���9rd[ AAZ ��� &��� 8p`7��� H�ax�v� c�^�$�p�b� �04���~= x��F����W����% +��}��bo��o�2��1h����J-��5��5k����JA����� ���r�Z�]�=�|��r{�� [���:zM;����G�I8�w��������WTXcq�)��kK^�.)-hH�2 � U����!Y<���i�H��vpf�������@�<h��p5����=��d���]�)!��������Z���J� ���I>����0��X}�Ul11VD��M���&W_��4�����I�&egg�5��CA����i�+��8 n7��A��=DA:?3vU8)� �S�w�q: [��y
;���&���+���|������d�/�,����r�q�V��^/�s7y�^^mN���H� ��2 ��]`��3X�n��/������|^ LP�z�����<������� g U �S�Q~�i�<�����c�: ��-�Z�]��EJ��r��r�gT��}� �J�a��������_�P�y�Hd����(����Zq>��9yk�����������o�yA�������gjhhp��AA���{c�9?WA!tQ��'�6�/��O�x~=�;����V7*-�Dz��{�S���\���/z�T ��W����.��W<� ��b=i{3f����y������h�(I��H�[L��{
bzOo�W��O����8Vh� i%�0��+�t�S���q����-#����#�3}I �Q(U�Z�M����b�����L�J�R�]��/��a�� ZsA���s~y��S rss_}����@�$kjj�.]:l����!� ]����"�$����U�_O�����/�4������������G7;�9�Z����[w����c�]�������� D�+ �Q�9�]�����=D� ��f�6�n���Z��S �����}*b�NB�Mk�?������%�w�X��UZ ��n����n������"� �t!|e���V�\�e���� ����9s����SRR�]� �t9D]T:D��sos��*:a����7�^�l���S�I�; ��C��ob����e�����kV�#�QU&� n����D�kb�m�����|� ����� Y��W/��45�b@�
Y�G�Z3H ��7=���2X� �U���"���i�]�������5�M�'�pkg �������YYY=�P+O� � ��oF���� ~�!�� @pp� |��kr �A��E�����t�I����J�UYa��y^E��}o��:X�&QJ,��{���P���a�Z��W6H/��+��<��=�I+����.��to�n��� �I��F@�"�����0}@���}��P���F�������W���ta����+���3���4�f|�A��n�� ��.B����E���2��M�S/'� '�j3m�X{������u����w�k�1A����R�n�w���h��{w����� ����<�P��S����$����W�y����v~�i)�4Mp-i���� Uj�w����Eg�,�y��l,���Wg���y�*K���l
Be�k
���(�1������L��'���L
^�&�=�K��P���O� �U���p�p�]bz�����.��S-n�� s�X�)&F��hw����O� 7n��o������ � ����)� ��<t o'��&qa���Ut�f����_ok,i FS{�� ����Mf(��xU���0]�$�UB � ����� ��T~�����Ns��e��4��!oI���r��-cpf�H�W��;=�k��C�j �:�4���j��Pt]��o�=���k��y����xAi7|e������?�3� ��o_rr��O� �t
Z�����&o'�&lD"@I�9����uc�Au���W� xj���`�+���H/Z�F��5����n`b�� ��]
W�#�"��=��]\����� �� ������.��.�[r|W��D� ��*10N1N�>K�i��0B�
�#���s����z�����������c�}�Pt����c�K��u �)A AAZ�o���+&M�TZZ:q�D ���o6n����_���A����h���J��Or��FQ�iz����
�{uT3W��y��d2��G����`5��4�9���=�>wa��j���'m���<�U2�t�Z�4_�b\�E������ �9c��*�=� ���N��/��K��Y�^�"JN���#�k
j2�l'hHKJ���M���I����c�=�d�J7�w[F�y������ �i�g�}h+��7� � ��u5�r3�&tIi-|�� "�D����w�����s�������������� ��C���l=o�u. m�`��{u: �Mt IDATO��|f�>2;Ud�z�Q�[M��� b��� S�F� &��/����V�������M� �YG����8(n�{r)��b A2
�h���t��4M� Vm
tF*�_��<-�J�� e� ��m�i�f]?Y�$�,J�,���-�MKz�+� �x;��r�ywp��y#���c��Y�r��3���cF�i���L}exnAl��O7AA���SF�V?���O?�t[�A�h���F$ �����=��$�0�9y�;~a�s k,;N;+�<P�X19�����hR�����0���8J;���4����h2h�Z�4�B�l!�=�oN���~3��� ��'l� `$Q��n��l���}� �u�]@�cy���
?\�W�(2��<J��������3������?��n~���7�7���k� � mG��~)���� �W�#� 73���"��[�����^����}�GX���;����v����%7D|Qs�Wg���b��+��t��g�G���2"�3=�����f�Z�m$��v���/�_A��AA�9�*��c��c��cv���%���� F������
�SW�����p�]�]��X�����MA�'���px��7�����3�0O�V��I��^��CA�Km �W���4�h��K���LK6hD;l��{����g�8]Xcq�)��eK�0�����% +��}��bo��o�X�aFlT���#�.(rvwR��K������&��4%�l���'����}Y7� �p�a��44���� %��'?�_���c
*5�]G�mq�` x$�:^���bU?s&�X]�����c����n��A�)W9M�(W���m�u�6h����N�X��;2 U���~^� �z|e�����[o1O
��%K��Y�CA��Kl �����]]o� PC*I"\��;�H��5 �W ,�Cn1�c�s��
v���'j��nB��,������|�����5�
5oV��.� `v�_��d�������.��{���������M!H�2 >hH�t{T�!���(������h�&"y�|jz ����7�+�'X ���iZRJ����ZC����U84�����_��� ����,��}[�Z� p���Q�F=������"j ��3��$%����Y�
�,����!Y��x��� �� �������o��� ��Tx��������>� �����:)S�t��%��m�� ����)����s�q����,a�o�R�zL2�����v�A�\>�@Th�����p)q�5�\,���
KVu��Q�#oB���%��O���W�NI9.���ux JV���^r�S��V�z�H�G�]� ���\U��-W� %�[qM99�g�e��q��V�Q �bP����W�\9r���>�`�����&3�1*����`s�%�c 6� ���&��P(x#Z-��� r�#a��naL�QJ7"���b�B|���H0�#-%�� A���!oI���%{wr�����+O� �"&���D�� �D5��K}!������k���F>�W�.{� `��4��h6}��e���� � � �
n���qQQ��Q��y����z
�b � � ��E����2� ���2��Z_�"Ag�nZ��GAK�� �� ����Y�c������>(�7�D�H,�����n53�'m/t�&h�Z��-��>^��G���He�����Cp� ��+�'���h�h�h����&� �i <� ~�����$��^m��!6�i[y���P��AUo0��%���T z��K��+�w����#G��?��� ����Y��5M L-��AA�N�o�2����u+wd���)))~=� �����
� ����4L%����3����aM��t#B���� ��Fz��Df��c�4�I��Q�6�O �f�Rb������*���X �����i�i��`�e��_��:ie�����_�� �� p5>����;�
f���"��E����T�3J� ��f�������. z�����IG���z�$ �>�7�_�C��/������ ����fw�o�f7���Q�� (//��������h�"v�_�cA����(�� /���k��� ����_~��-[�,Y���� �t$|p���V���W��#GB�9
G�R�������)�C�* ���2`zJ�|�`!�S����������;9� 3�t�0�y7�P���O BH(4�N
��o��*��ds}��{�D���(H������M ���g6%��g�J�Q�d��v�Cl �"K- Po�}���W�j'E0qK_u{-���W��5j����/�-T��#���#�[�#� � �o�K*�j���G� 9z���n 5j��+�� A1 \�#I�"TJ������a���W���K��l�p�S0��} ����{4��
����/�Sb^�sN�SI����.����� �ql������p ����������P��0������y�H�*�Q���%r��d��v����7/d���oW����� �C����P��9��~�\��*��
��0������nOz �����d�B ���_�CiI)4s~��IS@7�1�0#����W����v�!�A;w���cF� 9c��_|�/g@A���o� ���C� ��� � 7\o�z;�UO0z�q1��Z�G��^%��ST��� ���p��P����9�����Z�G��F�xm\ �L��"LM] ��f�p��qH�:���sg��`o7�V_ �Q��T�Hv�_�NPb����#����'��+���fWP P����kA���;��r6�=_�Y�(2 �8*���\�����]��:r}L�����������������'�J�d]��0�w�����+
���G�5}���^z���@A��FKDAi��u�� �pH ���^Ef��sR �w��:&h����>�,7s���=��N�-X�����/ x)��)S��� ���ws� 8~+wd���g���'3���B�y�5j�� ���w��svNP�]E};���6�� ��uI�JH�q/�o�i����OQ�i��V���� ��v>r�}�>p��T
����Z����n�C_���!Fu�b��>����b�<� r3��� ;v��� ���7�����..���� ��T���d��� fD�a��Mso����_��J>����1�@���+��&}P��%� ��_Z2Z$H�:�F3�i��#�c�Q����07�i�Td�0���}����� �r����K�he@|��,]�tQ a3j�������&�������z�;[%$��W�7�P��g�����O ?��d���_��N�����S�r�v���W{_0?2����I 2�l/l�qR�0}����_���B~����S�.]��g@A�.�o�����W_}���?fE� |��w�m��!� ]��u[l�+4����ZMvY��||Yn���8A���".>��2��^��#�d���i�����i�"}PL����'-��\1%�0��pR���=t���~H< �����2��%w?��[2�� R�.d�����+ ���H��Cy�zE�,�*!���B����f �������~=��) p���_���{=s�Z��Jp�3����� ���f[� `|X�@b)iyt����^��A����(�r��
6����#����t���O� �ty��.�l2�{��kG�e��� vO�� ��}Yn��'�X%�%L�������L�W����� �B�� >�R�LG��K��0�����������v� �Tb���.dm b ��i��xG"T������v�*!]RZ��,e@<A�=��������JC�/�J9���7P��s� �Hd��{�T�9y��3���-� @
5��Z�
�-M��S�j��+����J"(�a�}�i���A�+���LCCCTTw� ����A�&�'o]?�i�����{u��H�,7s��|_���-�/j��<�%�A{z 7 <$�0*����������G�C�[�x��r�=�^1�\�#���o-7�i������zC��� ���y���0*7�I�ox���2N��U���"��� :��~E�_����4
�Z)E��?=�N1�/I.��8 �\ )�j3mT�F�z A�r� �- �HlAD��u�e���-���cmeT$��%����������O �0VI���-��KJ���\v�Z��R�-��xb�K�C��$&^K�#���G, U��T� �N��]�K��aU���jo�R�]}��a]��$�7,�x9���q&6(��G���_=�
@�l��zw�#��Uf���5�i1���E��A�-�M� �|�r��=����2� ������-������>^����I6J}�0aZ�n����/<h4�u p� ���{L��A���X�E�����?H��E�3#��/��p[���MW���y���w �z�#- � o��]n�b����� lt�����V�~it���" ��[���+�tp����AT� ��7Q��W_�����G �����s���k�����9� ���vP@�-�l��O����F����a
;A��H��y�"��o��'�2��D6gM~�t�;�&B�H�u�_�H �� 9��,��Rb�c���S ��D����_o�M���s��� �6�R��[,C �!��K� �6h�����vHGA M�S*�9��+��r����={�y���/�jo�_@ ���A�18\b[���4��qx�M�c�Yz��{�SP��j�Z��i��LW�������=�������)�O����P9� ��p���|�f9V5,���y�MAI�ZX)�%��W��aj��p!9'� ��
K2����LMKJ��}���*fpZt���\a;�\� }A� ��q .�Mi�"������c|{y��}�^��BC<a���'AA�������J HNN���i�� � H��)�a�DK`D��Q��F�D��h �������?;�[�z_�T � E�`����T$�c#�$��a��x�� ����O� 4���+-0]f��W���e'�OKJ�&(T�/���8F� RA�K�7%��A�LH*�:D���,~A�-�Y�AA���%����TZ� ��'h�-�fc# �D�B���{�xH��0�5�uV�J}[b����+_�RV��~y���?�R���B�l�q�MR��D,��7ah���� �Z_M�Y�� C�� ���������w�i�F���n����g��A�x L �$IEqn�A�K �U�AD�U����TVV&&&�:uj��M�~��G��|���Y� ���S��n<}���g�u�A����x��t���� aP�#��p�5�x�W7�����9��y\����3v09y��87W+�7c�9-�{v����3��2�"�U�7/n�h"q�v&?�����0r}� �P�����/,I�� S_�r1o~�v��qb��OC 1������i�5TmL�6F~����8G�Mk��Qe2 ��� r�9���v�������n��u={��8q��+6n�8b��9s:�o N��\A:?\f`���b[�1g������=*6�v�M;Tb�ec��o ��K���/M;)���%i{c����/z��.rG��^�S�@��_�H��xu��>Og���0�������E�
�n��kU�r����l��o���>�K=����1Y��yjP�7 �6=q \7 >v�5I �)�PA��p7��:}��l��!� H[�9����������� 4������������������ ��EA��������'�7g �)x%0�:�pZ;�o���M4U3��@�jN�������'� `���ikANq��C�)u�s�
C�[��\� ����?��o]5 ���]���+M���mkA�)7�)������&o�#�[j�&(� �^1(�Qd =1%=1�����5�����q���K�-� ����9M� fQ#� 7+��2��h4��C��z�- �j���M�� ����,r�t�9��d���d���v��n�Jn"� |��`�g�LV���W���q}x�9Y��"g�E� ���@k������h� ��h�*��/p�|[�=���s�_�? ����O_��a�u��#/���H��\}hi�w����Rn������8�zD�q� 4�@������%��n�
�UW�JI4�Gm�H%
O _� �b�S*��b5� r�B6?�CMM
����66������?
A��Z���Li7��h�0��J��+����t���pU��������!\�-�>^��b�z�� +�f �4j � k,N7�x�n�������w�
i���wX/nivU���KKF��������!z����gM �j^�� ���Bi�V> 8 �.s���9z������g"��O��-~� ��]�)!������z�"B�������%�v��� ��V0*��{�9�����sY� pD��44�R��C�� ��%&w � �>��z�������x���u�" �KB�X���� *2��J A���&�$'' <x��� :���PEEEE���A��0I�E�2���'Z�vdXvj���arja��>8]_iq�(� l:T��t9bO�0�]����ss����S�"��.u�+���d1Q�J}����VG���P�8U�����{h�2 � U��-~�LQI}��r���� � f����nv�����VE ku#|�����G�����42+��&U�1^=2+Z"���7AD�� @���k�s����������^N�MGT��7�V��R�������M�*�1���
fQ#� 7 ��/��������Z��������g�}v���ms6A��eZ�a��:^����.j��k��p[ QD�^��P�(���MZP����K���fi���O��>UG�s��09� �5����.1]�E%|�W�L���m�� � 1��\� ��6�_������_.�� �#�tt>7�IN\w������zk}b�� ��^�\�q��U+�Gm+��5����2�B�� �u~^�vS��_�k��Wd2fQ#� 7-��2���;w�d�?���:����BA:)�
� �=W����������E������ETp����`�*c��fEi� �R�� �O
�����q}���B�G�n�� �{Dw��h�59��r�lX�&�^Q��a ���T�����EG JRy���%�A{��L�v���M� 8v~���nW��Z�(/=t�5��h���4�� ��*-������m@m+e#j�+�z�M{��+}A�K�[�����������o��iA���;t����� ����tZ�
Y k�#J� �Hr�9��A��7�F @�i� �#S�`E�TTbZ����fM �;#�T�������� ��JN�
+���
�8�~��������{�����J�j�vB��(�������^����z�Y�d!\����Z���Z���R��N]�|�[ cPP/&��O�m^��]h��q;�k�M���� ��������3/�~��o������ ��<�V)� ��`�T������.�W���M H��&�IDb�����$o���ZQ�!Z�"9�fi�"�T��5��1����f.I�\e /�����6{p�b����~3�v���@w�@���a�����P��� �6��j�9������xiJ��Pr��r{��W���[��#����g ���YU��-W8�
A$�Wf���� �2 �4t��y����j���RFV���!� �M� D��+W���K��� �� .�}p]G`*_$D�.���,��j� ���p��Nh�#\��4j <��KKF������� �/���p�F�H=$���z���"�nE�%
�OV,��S�#Y���h|�p UA����� W<��K�e����7��"��
1���n�����+�
��o�v�Z���k9�qZ�Yj���%lu
���(i���Lz��Pt����c�n��Y�������_����r��J9x:� � �~e� 9��� � ����/a:E��C�ie�N[T�4�
���L�GG��*�� �bQ�����uG� ��N����i�+1���^��c�#�[�"�����:�u ���]E��jZ'Zu"� ���n����T�c3} ��7H��G� IDAT��O�s�q�5���~*2��)��v��=��������j�*��bADY����&I� ��>
� ���Z����6�X��
1zauRN���.Y��_DS�.�X��L|��[_�m-#���b�}�uf�����*���}� �!��$5_��#�#�ENR��
� .��q\�
R�� � ���n���=��'6)2E��3�����"���BA��Y�LJJ�����z����^x�����>� �gDkR|�|�o��h����)<��>uv����:��3����-��1mM������/~qu
_�\��t�X �=��$���q��7B�d9���]�O��!� '����ne���m�A�^���.�� �i��'�,++���o0�AAd"K��1cF�~� 777++�w���c�?� �?�������� :�: �l�����>� �Y� �W�Q�T{�h����pS��MD�����V+h��y���:�=S `-�1�f058rr�Ei�#���Nz`O��� 4�]U�p� u��Q������z ��G[����k�^Z��;��i������m=A�t�E��_�����)2�!� �M�,Q��^` ���n��w500���BA��7����������Kw�>\$
M��n��)orXgSjx���o ����f��j���h0h���O2����pr)��s�Y�DNU <��pN��+9c�Q�L
����V�x���o3���v�W�a�"^=N��/m �U��+��b��b}A�K���4N� %!�o���L��QO=���3gv�� z$��A���7���'��9�4����e��/_��gAA�;��iyu: ���V��E O�z�|�3����,�r��f:���{��v�� ]4-�X.U�m_�{�����G�nlr��� f��M�"�4�O/.s1�z��OE4�Gr~�i��������[W���
��[��/����$������)-*��V������6�F�i����p��s����n+��1�@T���� .<}���"�m;eA��f������ }
)|e�|�I���e��t��
-��DA�N��S��S���T��k{D��)"�p��%�[t _C���! I�������<� ��N�
��>� ��qMo�+������Xa���E����{-]D�
^�4kUd@����Z��B�J��c��Q��.�H,���c���x#��*[��q��WL����! PjS���?}7� �e��]F����hw��s"� HkX�|9[9�9���������j�MA��cZ�A���hej� �'Z+�s��� ��;��W(IB�������������*�-�A �&'�8c�����M1�,�|s��
�
�CHZJl���!z����gM��\����y�T��
i��T�X/n�vgkA�s!��d�F � pG&�j����u�(���X�=�|��r{�� ["s ~�j���d��v���X�t3ul��F���m�Oo B.���,�����*R R�� ��S I�/<m�*�qnG(��9'� ����A�AA:9\y"\��;�8�C�e�e �:�=���#��S��?����v��D�h1H�!KE��e� �Z`�>(�������/-�>(F��H�bj\� &jw��~/NU�"]=�T���P����!Y��x�T)���d��NJ} o��ZD~v���@����n�W]tt�����Df���.+:� w��Pj��#���[�,��Y : �#���� :�c����|_�?���L*# BI�A�_ m6�F_����sNA����}I��+W���K��� �vt�w�4�����l��VA����H��� |�E��|
��/"�z�S�F���*�,�0�5��/����l����
o��|Y�����+�����C ���+|�y� o�Nwd���gE
�]��mO����($T=n�����/
L�pe�����1O+��Q�$3�: Ne��!��.�Y�#D�}�����p�����J>�G����h22�F6�A� ��%�����q�f{�T�����5��K�Qr�� � 76~e� 9��� � 7!�Kl?�XE� `D��-�x����V%��cC�@����Q��ps�_6L�������+���\��E�OY(-� [z#��&����,"D�C�uIi@���,V+l���|����P��]eE��A�%���e]uSHA$���x�sL�Z�F����`pf���%l/��E�1G#]9-���q �N����dP< ��-�l��M�8�����k�O�`9�H\YAA?�2� �2x��4
�+�����[�t�� <:�pIh���w�Rb J~��V&�������� #w�c>
�2Qs�Wg��B#���#6OV��j-��e�%���� n�� ��B�;��gT�P]��H��^u�=Du0��RK�;-hZ4�� ����������(�Yy��^�
��U+�Gm+��T�@�1����%���Y�d��R��sa\/��^$Ai)�D���^����g�tA���/����WV�� ���;mW��-�Z�����&�=G
M�� -T���|�#���)�J�1�5E���J( ���>,� j��-< ��B�ToQ�,����O,��\��� ��^ �#��m��M=Y " XL��g��t��� 0=JmW���� ������lT� �C�X��� Zb3�������.]�s���{�t'� AAZ�,Q&777++���c����GAQ|��� .�}p�/���Q��!Q�]�k9H�����>Vo�[�&CY ��[y�I�D��Wx[0~��� C~v���!���$T������ &�X� � 2*d�J���)K�u3�>�$eV&�Ptt<W���+���X���`�������1 ~��Qor�A� ���[�A���U� ����5��z1��I�f�����x$l.[���o���woXX �"!� � �%����n��R ��� �H���E����l�l����Q�7<�"�7����
E�;yV3�f%�ir�zE��1��(�&O�����D��� ZPzfC
�� ��~�B����)��+4L�����M� ��J�����{e�<��{�1X���c����d�ts}a�6�W@(��WY9�����z{���l+SXe��&�c���� L-��7o��m�����a�5� ��
Y������� � �M�OV/<�F� �����ox�E!LYM�)��sh��U�^��{���y�5 <�%'�X���iM��2�zy�b�}R �g���:������e�[W�<�]���U�^_{]�a.4�&B{`}���Y�\�G"#\�>�d1tW�y#� C�l�^���"i��IkS�� ��d ���4��rS����k�������� � �H K��?���e����0� �M��� �YI��F� ���Q�7<�"nY
�E����>����b��{�Q���$lZ��f�z6����>������G���W���k����Y��!��Wyc�n"l�e O��`qU'�b��Qf���Y�#N�� ���<��D%)RefPP��/� ��r7}�*�{��}�����Ai
d�S�����j~� ��9Xj[��*}W���UK�~�n7��>8]_iq�(�iV2��?�Bu�i���h"@�"�$�W� `l�t���8A������00�5����� ������������ �K�J���#� #����?�3� �]I};�Fb ��/�U��V7 �#�&M�m��}�d�wr��>M������ U� $b�$x�JO���? ��|�r�7z.p%�Ud�56�}�7HB��dAAZ� DA��#T@�t�.#lV ���� �����6��@S�� ���a���w�E BU��������j(�)'�_k��E8M( �������44�b������KK ��U5F��RE�j���9������N}��RC�U������� J=�lwyZJl���!z����gM�U�tIiAC����R�
�m��fc����P7M ��&>+ u�]+�\�����< �P���h%� � �GV��4+W�|���Z A���k���"a���Ie� �3�9Xj�[d����9<���K 4-<j���ei�"��a�RU���� �g����'i��wZ���%��!Fu���<�^��8ZUY���F�5�x��%,������*��"����MzWZA�"�������y���T�K�tH:K"VI����%�w�X� �^�XL���@A���T�.�h�1x(R4�T�Z��>��i����� �� �����d�� ;w���i�"���A�c�����
*6�vg�
����H:P�3��0(��/�;r�������$@I�P�y��]4;'�x��6�~j�f�:d/��C��[N�|a�^��I�?{�Th� )���U�#y[q����,T�?=���� ��syWC�.�%H���D��5L�
@=S�w�'�?��p��X9A��!����#��� 81�l���(�� �h��R���=�����e�*�Z&} �A��N���.�Ln��$I�)�e:��� ��
O�h ��P����=�?�#��UW�\~$�=5�H�� ��NW����=�Mzu��qt��,�<Z����3;������w��������20)|��'n�P����O_��quF�+4RQ3��3q�0�����c���Y��k�.�h(v�n��_H
�T8j�4�P1���-���U��z��JIx��@��BG4���� 8�����OHHh�s"� H;�9���j_JII=z�[o����� ��={���BA��i����-���&9i��y�Z� �J29�3�S��|��H�5�� � �n�^�i�=Cu�t!�5]�|k�'l��P���Mj�>& eE��udrd�����<0�����lL�� ���2S?�(*;d���h�������g���o����J��A.�xP� )�4� �4��`
��#��>�Y�)[������v�x������Aid�23f���� ���fee�� ;v���BA�iD�}��� �W�u �DZ6�0[A�����f�x�N�&Gl{�S�� �$���f/��s���Sq+0
����!k�C���-%�*����m� �j(4��x���J}l�3
�N���B!�
�[WM����R ���^*2���%u�_���/�7f'�9���?���8G� Lw�G }����o�����6=$� � �D�^x�y`4o��6����@?
Aiw$�� LHACz� |��7��4H�t-�����/����sN���D\���d�-/��}����z�� 7
�f�4��^31�9�_[�E/�W�sh��r1�L@B��V�2]p4? ����e���V��|�� s s��n����cw_|����(�S�����*�Q���?��Qd������� ����=q��6:� �i��l�fQ�0}L���`�m����]� �W ,mU'�����p�BI�z����3���"���,9y�;N�X�n�iG��W �����]Xcq �c��2����M���
�)�N�.s����H&�[��dWnO(�����`-��^�ff�
����<�A�� ����>�������f�����K���/!N}xI�������H����k��+��s�HZIx| f��[����_~����w�"� � �� |n�����u�]~9MW�sz!��&}�������45������rS�uy�VT����W��[�K���$V71h����#�H$"q/���<g ������������b��%�IkA�)w^�a��t�zn�
c��������w����7�Rt��x��x��Ob��-���K�:�� A�p���v �����{w����;�$��$�f� s�i����d��H�K=f� �i���1�Qx��[�>���?��������v� �E��_��p�+W�����/���t��A�-�S-������x���{.R�dE���@ �FI^���U��"�� �O����l��I��AN��[LG��
�H&kA6cF�����^��� �1�
�r��iN{����w���a��+���% ^63X�1q� ���[��4��� wP�$���x&��D�����R���h'���� 4����t ��m��y�TdA�������3���sG-Z���?��T]����"��rb�%M������t�<l���o"���b�M�p���b�MV7l��F��(jm�df��5h���~�k�P�at�����?O �!h�]�t�=�����O���{ E��FtZ~n��
5��&�D�y!��/I���D0t�� ����Y�c�7��
��*_��S"E���q~] (��2 @*b�������o��������3�� r�9���2�eY�d��S� LM�c�����A�����j D����$����a �����LQ�� pZJ,MB��k��p�/� ��^b����.7�(3���Qa��������n��1����K^��K�t���� �g�x��� @������w�'�(�q�~(�A�
N���{��n���wZ����o�+
Mc
����u���Y�d��5F�t�6��&s�)RC�����ea��W J������O�"� � �o��?�p��Q�"� ��T4S-�0�� �{.�p����-��~�l��W,G��� +R�NOi,EI #���L��+^�+���hAf6�/ �Z�W����������O��$ 9�?yf`�@a����eitla����=��2��j��;^�-����C��n����N�<iV�� �v01\:�j�ss��J�������I �� ��M(�iw��
UNe��M�$<>���I� j����v�����k"� �n�&�$&& ?~��;��n��y��~=� r�K�&������H������-��4p� i�"�� K������U���c���f�����-]�d�qF�a0;�������[L~�iF)c�{~�8G�;e�?I���� �� tIi��+K ��V�Z[������ �3J� �����r�31� AIM�M��M4�|OO� �C@2E[4���oJ�0�����p�4�\� �V����_/6��h*|��������A�/��2�����&L���h���;w�(� �Qt��!a�S[���=V H��>��'x��x�b`!<���Z��<������Q�����c��z i�KlJ�( �{��N��&��L�+0[sl���$�� �-~x����G�������qG$�Q�6���y}F�3#��L�0lo�M4M,�~��f�WL8 z�o���e��.����"o\I�@@��SS5SfW�8 ��<��S��� � �o>7&L�>}���Cy L���~>W��sz!r�����|� �O-&r��W�= h���_����<�^��/f_4y"@��v+�)��n,����7c�N
n���4�@���M�Nxi}��#�(^S~i�<��7�"��TG�.���R��_��[���O��1I���o��Q]���q���#j3����L����%�����0� o��]%�:�&=1���U�����M�������I9$���d �0�E�GBA��N������F���O?�� ����"r� L" ��-�,:��q��O����=�df 5�0
�k�#�_���H���]+���/j�{���rWh��zq���!���E �j(���< &Z��~���]�j�?M=i[ ����J -�'�����O� m�e����-&^��
1��{m_
����o}RvZIN��yG>7;w4����L���mv�kZt���\1(��} �/%X���[�4�.�0t����mtZA�lt�/�d�S8<��ck���
��;��AA|@�4DTZ� ��?T��wf.�Kl �����]��;�)[��YE ln���f��]�����)�C�* ���2`zJ����fz�N������W����49������|��c�>���"������7�+�'X ����r3\
�4�t5>P��������=�w��3 ����L|3��8U��p��*V�|�����/tPN�1j��b� �����Xoy���g�j�a��42+��&U�1^=2+ZB�}�\����JC��R2�n1K�v�� ���e'�����%!����bU;)�E��x���� IDAT�UE:f��9l�~��(�^E*�B��NEEAA: �<eV�^��G��{�r�e� ;��S!� �&
10��O�$��G�)E��A��"���)�HQ�\���`�u�.q�����K��� Sns��Ph�� ��%� ����,�j��%���@O�^Ssr������^D����[9�"+k8��DQ��`��D�o]��A@� ��������K���Hd����(��1���R97��.�v
�V ��< �U3�N4�� ~HOO?�gY|L.��F(�i����3 � �_|ef���|�r�������� � ��K���f�/���z��n�u���i���y��g��.i&c��:�~Y;b�Ak�#0J]�y�nv�#�=7~������BI �Q��3u+����&��85����pr�.)
t����tf���3��-��J$���' | �t��9��oo� eG�1�0#���^E���C�X����%����x�>/
!(zy����Pt�� ��j���{�,���rY�v�[j���J��>8AA�Egl��Zt��4An ���/�D��&�>:P�S=|������/�z~����;-m���H!�����s�}�)V� �#� @N^�7������� ��5�] ��z~2Q��c� ��S��F�o����?+T�D��4d�.q:Oa9���t��V���`�� ��y��2�c��%�o� 6���>�7�����n��!����BwK��`w�x �~1� �2e���[���r�r���c�&��AA�:���{g<S��s~���l�^�h�rJ`�����?|Tvx���T X�t'�!�$��nx�B����W�k���]5u��_��6 9�O^:�4�}�d�g'����� �PR:���P)k9w�20)|���f-x%%j���MzWZa���g�*,����R�H-�K|�}�o!M�Ba���i/����"�D�b�)5�a�K�zP����qU$��=��'� ����'O���g�� 9��$�oy`�.��XA��t�/������]�p�B�9����� �q$A�g��hi���0 5 �; .�}p]�al� �%0~w~��[o��
��7`��_��+���X�RXc��q� h��l� �� ���#�.`R�� �2��������\��n�;�����)2 PlV
��*@�����Nk������.3U�;�����N��8�Ns�)R���
�Uq�6����R�' N���a���
^Vk�8�v��&�> �yi������^�n������ �E����q���djd� 92y������#G ���%��~�� ���,��� ��{��� &����O��JtN�
A��K`$@��V��%�o�Ko�?v I�k�^^rv�Q�S*�[ ����g�*b���G�<1�����h�xR7= �����MA��m:��znM�B��3/����_�h�����u�3��BA(��D���V�������A��
!�:��
��� �Sa�u�Se"�*���^�u`��*������ ~x��-�G7~
N�^K����� �>�Y�n�_,� �tZ:��wY�2�������g�9g����yA�����M ����k�������B��W�� ar�����
����" �V��:�����E�O14p�ix��S�u#��X/n�i ���zq�:|8�\#�[����! #��%�ngF�������V��2+##T�����;
������p��
0YK�JC �^��$T�bJ�BT�`�0� H �i��S����w�|� �� �J7ZO�l{��8A�pd E���7��o���>O��s�m�ti�(��5K`�p�W�{x����3�����-�J^���d�
�u��d^t��� �C���W�);�;8^31��L��p;I��T�6��� a����+|�yo�6�[��{�34�RFE,D2��KXu����EB0���#F���8#v��\v��8�����[D+��y��>�0a�������.� ���� � ]����]V����� �t2�[���m��^�'Z�G���7���z�H0�/x��@�r����;���Zl����;��&���?3�;=���m�DY-.��+� ��C.����7�Z�!��+pU�T�CW�s�K��(�J�MJ��m�d�������$M���������;�����l3�<�����{��
i}6*�}=9l����@P��;%O�U��N�n�oA���z�T�� 8�:Ns��+�4�S�IHoaMH�t
_2���G�G�'�!Z�"]��ZI2&^�.��T�2k���Vd,{��7�&L�FA���O� � �i|z���>�.����W�j����
0�0q&\My�/��m��q���S �It �����!8���U��f
��Z�'���e <�pV�q��_�Xnbbd5�G��;�?�,8������?���}�M,
%�}��@�xi�&QP�#�z���2_RI��^�V)*0��.�r Z l��a�jk�fR U<)24C�D��I�Yya����s��L���O `,V�G(� � H� �wz��
A���� � ��&E������� ��"nt��}[�dca���dF�x�������9\
B�
���o$i5^�r~��B����m�E
x���1��%�#�F��e3A :� hF��Q��arR'v_{��Ru��Rz�� ���3?���b#� d���;V� �t/"��'��������E��T������:�Wf�n}[�D\N2#����r��� � A��_P�q����=U�p���a Ij\��`�P�
����s/�'������1f��������4I����E�p>�f�x�
.��R��~UnS ,i�W� ��q/�%�RW�G0AA�eA����t!�7�wi�.��x%B�D�j?Y��;.;����0{�O>8��"�S��Xk1}�h��`g.0& ` �
��C��T ���������U\� 8M��EU�e| ����W���u2�l�m���
�\6=�Q�
�( N��2�Z�� ��T�q�����$��i���x��cn\��L��6��T��Ym�LAA�� @A���=(8RC�H���t���t2��kZ���M�������e{ ��,��\/9G�I�^o����3�z���?�R�_�9�r��zs��8�����h���W�_K�>���wXE������eE�R��wdVya_�X��>��l;5�pVVY 6��o=xxD�����V,MR��$!�VdBF�b4�'�@k:�b��2�N���d���8A�7���U�>�������p�A�J�O������ ��`� � H���C�����!/���H\���U�@u
���4Y�������"&Sd1�3m3����Q6�����v���M�*���
����#���>����B�|���[u�*�J0AV�_T��\�o��� ����X�������!�i�@� �Z���TDH�N�����~� w����YY^�h���xa����$F�nu?AA:eA�/=={� ����$�8O����������e�l�E;�2���kE3�llv���K Y) ���Q ����k<��e@���R��n"��w�v��x[jb��/��Xk�e,�*M��_���;�K��K+����%�fw�;���G�J�A� ���h���+V������|T�A�L�} A�[sK�j�������FF\iD �.�H
5{P��� F����x�� ��)���S�"�^�p�y9��V�^9 ��R/�,��#����v8K��5 ����c ��v�Q���G]v
��eS��m�@D�T����O��:2/.8Y����K��_��|�\��s)�-�/.� :W~� B��$�*�5~J���~0��M�u�����=;z�������9����� AA�� +eA��A��~�` ����[%Q�:w�� I�y�j�|��H �i�el���$?���\��"j8���E�aT���k ����J�ekm��m�U�<q�Z7�G�I�O4b����/U�o*�I���^��X��aX�W�
�f]�f��C �x�/IA�����M; �4�e �|�m�4D�DBP��k���w�F� �#��2�* 6�;S�$� �������.\x�}����v��!� ��(� ����w�������V� �o������R�tl?����S\!�����z�`).�-���t<��;kB���3g�O=�,"��k8��,�*�M������m_�~"Kq�����F 8K�8��<��'���N ��l�����H�@/��YX�
���� v9S�F�
�����"8E��� �� ��������/��;��5�o}U� �t&(� � ��b42��� 4����M�qw�do��P@23���qF��3��g}r�AK��kH��I���S���t�� �� ��x���"�/����Qck�.Q:� �t�m_O.t+�q�$E">_M:acZ�)�����'�����2�07��*Oi��n^
������r $��o_[^{����#G>���?��_ @A�{�� � �.��'�v�GJ���}�S�i���� !!qB�N%���lNxu�
,Y����~�����!��HHU/��5x��}��d �C����5|�� b����j�N]=5�P��g��l�")�8Me���8 PE6��/H%Y�}��#~��[+l�egc7U����^���Z2�H���R
��Z�A���X�����?��3�<��_ @A�� Q�n�+�� A��
OpzG7�O�4nq�vGHH���j�[���,��G����%���Q�i<�Pr���XL�G�c���?��� .��/� ��C�It6�s�j� ����+��M���jZ����lx3Ni�$�}���+����"���s�*���egc�xs�%/��NG��O>����z_0� � =� �/��� �\�� O�pz���)���4��j����`q2Y;��������&VRBj�yQA<e!u&���#�����_�eu���Q�Kq~��������)��K� ����������6
{y����;�`��#� ��Da�sP�\rPr����%���-K�
c���/��r���q�,�(��>����<`,�F���I������L����v
�_{ia��-�,L8A�HY I� Vb���7��jA�Y�T)3i�$/[� 9�� � �/�<��E����4M\��9��W1�Mp�m4�W�t��$�
��= �sh��
+���;K�~�c�����~ ��-�����?�v�a@��F@Q�q�#�c�s�����/� Q�8��-Kf� ����;���xM����g�����9�M�Ap��g�����C �P�*��������=Im�L0��?#W��K{��&�+k��q��tY6xsgFA����(c�X�o� 4M������������t;v�8x����~���DA�t�i��^�ii���[@N�ZN� ���q���zM��eo��_S�k$���:BB�2�jR���oE��X08����
L�`���O,�6�{�������<���`I�j(�G&� �����U���zS +5#+5C<�e��^c* N���Y~��&E �������� d`�V[�����(BC�?�1����1FKe��W'���6�l��EA���(���+���c��=z4;~�=����-_�|��% �FA��N6��n�/<����
�i��*b:BB�L+eT��2X��d!�_ po ��e;�� v�i�V8�'1 |%&yr�y�� J�Mr������7=Vz�%�H�����}X���&����������� +=2�S�x�u�%��w��G�I����s�0���"biR
�)��!Xp =o��
� �:�&��C���#Oe^�v# PB`AAz<>�2C�e_�:uj��E�M��r��U��.A�3�`�����������x��i������������������j1NsA�` ��Xb8���l�[�����tY�_���d��FL6g��o�%|�*sr� �������� �h���� �I������rU9��a�1[��� h"���R
"-;;p�� /J������kO���g t6�O'���;Y+����b��~��{��X� o���M ��m���ZAA�!��/566���� �DA�����:����J B4������T�KELGHH��R�*\������,� |��g}AX�G}�=B0��I��;" ���`��
���������@XK���\?+/~�4^4���%�{Hn����w�h``Se�������l���\z�Ss~�V�x���^�"��=����?�s�� �tC�e�����=������;w.8XX�� �t(�����zk��e�8 �{��+��Hp����_�2���3U�4]��l`��� ����x���8yu�=�mDqF0�� ��. ���g��mg��/G���1�� ��������=k��
�����6����� �5 �T��2���a�
/|��
�6V��p�������#��">�x������AA��������z������7�p �8qb��y������!� �LK��:Y��t``���$!\E���Nh��h���JP���h 0 ���3}LDG;��b�^y&�`A���t�-���g�����F���Q�U<�.-�{�3��J�6�2��Xu�E�|�6�����/�3��� S��V���0z�b���M�������oh�������j�&v/���)SGv��A�L�e7m��t����744$&&$%%u��AIF���=U�r�B���f�=G�J�^j����m,��(�����UdX�"���D$�h'6c[�It6��X��<e�39�@�T��tE��� "?�[w �����e��� �,���.G -QJ��I��� �$~�f�uS�I)-�V~����|�[���N�� ������u� ,U��?THR�W,eH�h�Gw��� �u]�� �N���#!5���P�Bg|y� �t�e �W�^o��VG,A���vd6��s�{�PGDG�� �n#.l�S��*VU$�Hr�������9�G+6c;��\���-�A �2m"���4]�KB E$)����������S�~ ���������I�es6K8�p�KK�^���P�({��x�8G�k�����/[��� x2���
�����/��u�a �� �3C�����6�0��6|��:a[ h)��~� 4���FQ���_#���}_g����T/U�,���@�
AA�n�0�8� illt�\���� ��{� � ���d�u��}������]I��Ro���(p�g������}%s� ���Ti+�
[����9��Q�k���|U��kC�����4M+.����^.�V� ���> ��� ��]N�}�+>V��^�R��k����`�{��_N�^���� �[���RH����6�-2��p�LK�\/PI��C�*f�M�w����"��0����~voKn�"�w|u�Y9f������ AA<�= ����9r���%KBBBH����[�p����;be� ���P�K���N+]h�+�g�����7���D��t>�Y^�y6+y��d��u��u����O.�;���!;��|�K�JA��� �l�AO�O^*q��Yb��AO�6,E���O �f�Bq�Rc��r�4��U D���R��s|��;�'�o69� 07�\ HP9�#-�Y� � =�D�����^zi���aaa `0������gddt��A����6�=G]b�+�g��df$0 ����E�Y��M�0)t�����W �o�RPX�����QH� ��p���S�.3x�'��7y�s���F-�r��8���3�?�u���F���'��DP��a/��>A�r�9�J�RI����m����Va�cJ�F����j��������AA�X�~����}���YE t:��u��.m��� �����UQ�������:o_M���y�j�+��x�=��J]�n� Y��/��RP�?�FW���\����F�V��k �i��^��ev)�����d���%�\� :,O���\��l������x����p�%u���
?�z��Q�5d�� ���>����n)=2���|>�p��]��4$7ZaB�� IDAT���("VI�eZr���6��}���|��������S���
luLIc _��2�/�!xor�o�$.����� �\=�W)c������#QQQv��w\� H�������Xw������q���#LsY|7'�V�S��WZ�0� �S������ %�v��c_�dG�������{e&�B0nr�k����=��� �������$�$���|��X�!�_��
s4����A�-)T�*I�d.2i�����9�O�#�~i��&F�#����7\�R�xS�Z���}O;/�h� ���K���3kF�
�7M�&d���Y�X�� � W3~{� �\���� 1�4���/�s<40��.�����k�,�.Z����L�����pM::� �`sH�=������>����`s��q��$��3A���NX������L&�t-I�# &��� ��4 ���W��_z w������|�I#���g��8jaF����<� ���}W���4��z�*���<������$��dC�bT��6U���c���h� V����w����{��4�e �L0��uA�j�O��������G� �����7---��B�TZ����<nv�X����_�����D�7����U �m
���4i%���Gk��,m�3���[�4�gu��� �O�i,��#�l��Q���q�I�F!����e���]0�=�m�J�����W=mu+�ifk�� �� � ��P�'w\!�����4
���E �mbo`]��:��m� @(��A�B �\�}h��� �lT��G��@�����oXE����/:�-�2� r�����e�^~������F����� ���K�)� HOG2�� ��/���_� 6d��� �[��j�6��4��W�=�}��- ^�Y� k.cuB����y� ZjZX�����|��o��e�Mg�M���$e��"��4yS�hV�X���X���xy��, ���2hNSY��Zr�V b7��_��e�[z�lrz��: `d^�d �����%��D
%v����DE�>_P[J:���R"!�?" ���� X*Y&�X� � �ar�H<����.3 ������@A����( ���_ ���g��=���v������*�A�?=+�G�����%���.�inOW�|Gl��Q�yS� �o�- �mw?7��t��RG����:��?
$st@��� rF�{�O.�<e��) ��
�a|��,����S*�+�7�X���d��gb
�V��P$� ��������eA����r{/����]��
�z/
�'�F�Ltl�P�
,�7�����y�����x�a��b �"��Tx1��]�K�k�������M+f7� ��h���Q(����?~G�A�KlDt�3-M��d��g��K���8Pj���t��w��� \�s!��]�b?G�L�^��1=Q�����=9�O�>�t�`*���"��|C����fB�� ����x$B�'3���)��@��d�+�[rk���5���N�����w�W�\���� f�yj\�TX�3C���@��� �L$��,�7��� ����6eS�������X���&_v6�����iCU�^�SU��X\�Z�`���{���G6s LZ���A����JI����� A� ���������x� ���B�6\�-q�U##��G��������#5��$"5��A���U�c��L��~Z���|�����V,���lO�q=H���;"���h �b'�:�b��/,���:�MV�aa ��/^(�7����K�]��~�S��Q��&����5��.��A���<),^�a��D����������8E����y�.&��r�IR��������X�����f*��o�� ;S+o���+/�}2�n�� &'��������H����AA�.��J��p��I���3���K������
A�S xpO����������� ;�.bI������/��d� ��0g*,��I���3�� R���)XP�w3��k�!��Fo �%yjV�P� u eH������c��5> �#�������sn,���� fHn���zN��� @�� �j �W�`���$%)�C��b!�xf�L&#)��{�W^�R30AA�8 ��� ������7n\PP7�f��_����3 ��!� H������]|�s]p�O_�����9�8j�R�f�uLV��'%���=������4��l�R�v�u4C<��/W�v`��F�r�@� \r6��������d��Eo���4r�9�8�����]6f����/�`������C�*�,�G �O���U��E��e�������bMh ��s������K�'� ��k,:1�����������!� ������5�~��[�l ����eggza=���sEi���T����h��n����t�"������zQ�T�m�9� b�2�����C��OO��-? �����? �$�� !��[�����qAA����8t{��\���.�pq��; 9A)7�dM|����G�� �u����I���X��S����W�o��f �J�@Rq�1 so�� ��2X � � J�|x�oM������o:n5=���sEi3w\v����I�0>����QxA{A9�u��� �" ���v�������H$kI ' A��d%KQ�a�,=�p;�:Z�h��&:q���w��
�l��:����# �������������?� ����r�������m�0�j�������[��5���+y�a@��\}; X����_@��*��W`�V��+� ���uA�(�����2J��l6k4���bQ��]� ����8*�V;�$�Z��d).0 �f�I�<���� ��i1��<�
K�u;h���h��A���8��������[��5����k�J�L(;6H�T������/��~�������"/���4%O�g���K�����+�a�mm��q]!�^��Yr�h�� ���y4 0 ��:C3��>��V�)0 �V:L �H_�] � ��6�OF'�e!� ��������/?��������{��7�KB�b�+���W���z�����% ��z�qT�T��t��=���;�Pmv9i���Zw���^x:A�RG$+5�\�5� �0������0r��Sdr�&o��RRB�W��ds.���x��I94����s�n��*u����� �bT�2RN'+_�/z���'J=T�C��k J� �� �Z�J-E��k���@�����7D����lx�vV2��vV�
o:,{�f ��|
���=?�����}��p��%����{�* �n�Vh��,q J�V�j(�����DA����J�A�������\�r� w���|�u���Y� HWpHo]��q�>� � �W>��
��n�T��da��������vO:��~z�����|v�$8?b������$OGo|��i�j � ���Z�r��{��SF@�T)�@�����-=: dZ �r mc���C���-�t�� � u~6������������'�M���y�,�H��xIS���#q%��
`5~ LK 80��#���w��6<C[D��<4�D -��O��2� ��kon���V�B $hB�=/�[y/&BA�G��(������_{�:e����A����a��l��_��-q*�tsw_�[�8h����6�������fQ#����1 ��L��l8� ��P�>(g���u���4�Z��D���;Z� K,�#*��#���>�le������ -E�fZ����S�=LO � ��X�c7�� h����W�2�2W��QJ�D[\���X����t^& ���V������ �O� Sz�u���n���S�������
��K3�}��A�:�I�y����l��aC�� ����0� �����H�Mh�a��Bg��nb�#('!�����f����Z�����%m,,dfK6a�� km4�gtl�eA�tP� ��iA�s�# �����H��e���5���\���J� �.��C�[�J+(>q�K��q����
����_{]�����R����I�U�[a��1y7M��z��Da
������ � ����)����e+��"r5!zt�*�[I
�= �&� �-Z8<�t���������syA�'3tX�,(� �����ayl(����{�?J
W� ��,��X�=e������ yS%�i�;�l��t�����e�Mg�m���eS |bd^ �0=$7�mbb�i�!K��
�b�8��m�r6�0�C��g<��s�a��\�� �d�h���J.[��
��� o����,/�1�Z���E~r�Z�������|\ (�1^> q� rk]���/���? I?��+�u� AA����������� �~� �����]�\�]�0 PW����P%i��HV�|Wn]�kc��)LHP����V`�sO?��v4���D��"�� J5�_d��S~5����UPX6�� �] ����*�io���N�&�g������������v.t��L7�:
i}6*_||Ja 8m���Dn_�e%�})� �(_��8��J�/.,n�9�F�LNZE��Qa�/;���v� �fi��� �J�{�� ujV�����7;����D?N�F�F9� �t9��t���s��������n���+W� ��z �3� A��X'�6���������-��<��
4�����}������z����
���#���5��s�h�����M��_��qo�Y3�:��i���'C�� e��:�.I���9��v=H�]��
����WM�+��^��Y� ���>�ar�~���2��V�t�6���d����
YJ9cW��g�B�*f�M�~
)(>1��g&�]r+3�u h���v��|HJ���.�
}������ ���g��(B�w�� ������U��\�R ��1����
��A���m���6���W���Q�#Cd���k+�~+~�O7��~�N�~���Q /��d�: V���/�A�iC ��;>����*/��i1K���.��6�RX��5��2�����*>�t>���N���w���5P��xF cy���O���lj#b\W�[m4�Eu���R�8���-�gQ���d\z��^�&#� �%��}� ���� � |����fM��A|�� p� �����V,]e��������"Z}�����L�NE �!<�;� nI0���&W$ �:��?
$39���:����J�LH �=��\J����XT`��]��4�����g���t�.8kp<��X�[I����7 ��_��~��a���d�g~K ���XN3n^3�� >6s�U�����- L�Z:+��@A���d������V<A��qK�j�������FFtm���x��A��JF���9�}|�KMy<QPX���TI����Y������h>�"M�w�DPe�]�as�r���*e�A O� ����'������&�i���3 M�>������K�N(�h��:�v�s [X��)��D�y��w�^����R�_�9�r��zs��X������)�-����cjO�r�z��S��� S�*2� ����w�3U�����krQ �]3���3RoP� ��"��R=��+�\=J�{�������htO����� ��_[�N�8a���x��7�l���i����v�\+V���vw�g[� H�1s�e���/�/#���y����������8sz�������N�F���)7�I�� ��jv��+�����BV� %�Na_7V%}�|S�Rh ��s� `)�o,�q���&>(c�G�\�� /\�%G�������,��#s����C���dz?�����6OuV���!3�k�T�/t�C����;��-�E2X����g��_x&���6dN�z�� ��������^y����a���3g��L�<y2I������=�� ����� ���zn����*)e��� ^v�]%����:�$���������n� ��B|C��L���pz6���T|1�8��z���/�����XX/���}����#����S�z��a�k�~����hHB8� ����[M)z��������w�����tAA:��������s�= ?������;p9� HWpHo]�\�"������G���j�"���eq�_���������U:����� �aj<�(����tE ��!�!�6�F26y�R#H��7��r����xr!����3��w�� ��`b��e����I�w�4 �w4����-�@bE d`k5:��_��c��={P�AA�=�g��z��Z� ����X�O�LG8w&�c� ���o��!v�H�i���H������Bw.kN�Zw�Mg�5�M�~��@-���'�J��%�#���D �
Q:��{� �^kv�����}����VB����� yr ��8��������O�}@���k������
��e�4���sn#�(U�} ����u�����oGEAA����(����A�@A�qpR��(%� Q� 6q���4## �-,����p���:�����,���n]P���V@�#�9��a�e\�v;me��+^*q�7D�@��?�����- ��UL�yoc������#�4���� .��_����$q\�xbW�c��|e���5 ��o�bC��UB?%��S
zs}�&d����U���� .M�� ����������={"""Z=� � �w�n�:v��K/���������,X����SW1��-
A�/��� ~��x���g/��7����[,_�oX`L�U��L�.�������U���
Q��{[O�~OuZl�����Rd9�����%6���X�����U����Q��=Vzdx����R[��q<�Z��%0�1��Yf�6 Z��_{M��[�i�96�T�������� T�x��O?�t�����M��AA��F�|x����� \�d� |�����w��w���; �0A�����UQ-O��0�������7ePr�FN��a��)����Z��@��g�������Oc���h��~-C�I�x��)�7�O)Z�8M M` �[��6=����Y��9�� M��� \�64W�f%
{�$���O��9�B�}���X� A�������&���8�����,
.�PJ(2���UR�����������D
<�0� �e���&��d'��4�tX���rAA�W�y�����]��� ;��� �^� ����W� ��d$��f
��Pi����rF�_�p!g�v��vh��-RW�\hnK�WoN����iJE#� �>����������ti(���qH}$)��I �F�W����2`���k�����h����LKY*�x|G��
��+`�O�f��8+CR���2 �������a?��P .N�? ����b5~���#� ��?Q���16���� ���O�� H7��8�����#W����~1��ii�B��O�^�%�\���L���Q7ACZ������&h�����P�����)����#9:,O�L�rYPr��<� ��� H�D(���$�������:
e�6�:��}!��I� ��� �Dp�bd^ ���� �h�mM F^�S����� �6�x�_��WZ��j5��0��#�uE0��C0� � � ��~�r�rb�����"L&�� ;�l���o��K%���e��U��8p�o��P(|�� �H�]�� ���t��� �Z�[�q�r\5� �����Lol��j����Do|���d>��X%�x��5�S}*em,�-��!���j���o�7O���-�O~�W����'Xk��sLz���l/��>�y }�k�.
4]��2� ����*i���"������mp���yyy������&�H���zqFA�_�LHH����#�����e� �a ��K�.=��
� ������o���;b�������A������TC�����j�k�����������,{���:��E�������X(��p�� �q����/���x������ �����/$�i�C���q_[� �@���� �����X��V����(����u� ��O� ����X;�`Zea�7&�} X��Q��:��$�.���!�`{�q� IDAT `zl��[N� �2��������S��o���������KRQ���Kz?j��Z:+�W� � �U��2K�,y������ �������V�Z����w���� �u����3g$'�\���w�a�}������������_�"� �|z�du�<�[]�ggM#�vKI/^�I#�U \I���/���V�J����� Y��%Ko��)Ns��q_��o�^�����1i��7�Z
L+�E���9�Mz�6^>t������\$]����d � h��MV�[a�V (I�u��!e+��& �[���RO�K.�W&���~as��~����zk��}��������-2�Z:K��]�0n�KR�� � ��O�IKK{������K�.M�4��o���/U� ���f����|� =ztaa��w���VAD�Z�PS��\}�����8��e).0 ������@�^Lye�Dg�9�m��q_H� �9�s���.Z ��� ���OU ��n�����5�\31=������B�K����*�y��7|��W��j���� @�� � ������W����o_rr2 ���$��ftb��,oq�� �\���� iii�N����/-Z�R�.]��+ 86�M�����ucy�� ��/��A�/jJ0.��H� ��H�xA�m [�n������l�����u�@�q}��=��S���?�d\�)VI<��'�,Iz��� ��h�W5J]��R��
.{���-��l����R~�M�iM��u�q�Pp�+����f��&'�$�=R)tP�c��8�w�^V�a�kFG���� ��o�����j5n AA�X�x��'��^�4�U� @IIIee%d�� �4���p8��J���l�l��H�"�����]�s�����{���w�x��� �o���%�x}A�m��p���G��
J�Q�[N �X���z���g
A0� �>�@0\[PP�26VI0�H�`9���������3$>�4]�k<:�]Hb�T�����J$�J��5�M�y]-�������O���xMhBJ���Gw�
X�� e�� � (/^��<�{�2��2999?����!C�3f��@��C�����j�rB���� ��.�?X���O;���/�eo�Y��paC������-}I������Y�"�b�9� ���9��=k��Y��,Kq~ca������ 6 �� ;�zk_���~�?v=?���" �`3��luM#��S���T�r�����9}��K�'!���;�a�s�h� �8Q�#�=IA�"�4 �qW�)`!� ���O���k���G����*J���p��SVV�����VA��_��!Rq�������G��mU���R�|������ ���r�0G�����=k<cs�����J��N}����/V�U��D����� _XQ�f�|����2(�� u�[�, �$���b��$� t
u���+�a�K 4��#H������"��B� � ����7P���>#� ���?O������� 7l���t,2�����{�����p�
>nEi3��[�����Q=o_�w����P��d� ��,�������-��Tk�\���u���ay��d�����C�.<rJ��������lh����G����,����F F,U.�����h�����LK�� J�[j����#[����$v�{������-�yc@i� '
cO��q���� �"��+z%���==1����f� W� ����� ���htOt\�R��AA�?�U������
�8q�R�������3gza �SO=5��O>�����[���������A�m �[�5[���E�B-6�
`�QG$^w~�U��HB:�$��m���L�P������ �<k��27G���SC
��+T��l�_����V%�y6��aj)���V�2$7�h���4c1�l��hZs���~�o�jx�!��k�&
������}R:�aga�7�y9'�"Ae'�p�<�t� W ��I��(U���4�/� p�w����}y��'���kWh�5�Z�'��+O�AA�?��2�f�z���o��f~�3�<�U���g��Y__���x���s��� /�p��7�n������1c�P��o�w�y������ m�#D
q
�� � <��#T�V�� �
��{GZ�� j��� ������64f�2g��w�h���� S�I�rp�U@�r�J����t{� a���[��;�R�_26����L���YE@M����������������;�agH�����Jf]Q. `ju��r�(�#��v�w�T�uS��Go�(�pQ<�gAA�h�'����gffv[O�$7n��}�����O����� m �� D��J��A�h�� �E|�v*> �R�J�A�rU*��;\}���?���� ��� e�����)5��x���0���Fo���/C2.3��(��2��J��H��(wZhK�[>����c�v^:4���s���`�m�c� "fZ��y�w�?��� d��F����:����b�rIE�
�b�}� ^a����8+��7���L��j�� �5���\=*PkCA���2���V�\)�={v��� r�!�0�_�2-M��y��(��du����D���=( e#>W;��*U��(� O$���O�h�O��~Ze��Z���fy��Kq~`�Z�:K��J*���|��xr!������q�+ �8L�'A�� ��RZN[}����:�w!}��b��&@@�E�����|�h�������~����m�#�X�*wDRJ����r#���#������NV��� `�N���N('���J�����2��6b�(P CAi�[x&999)))88�_,s�����/w��z��=D���A�ums� �(b�u���]��f���*���-��m q.y|����EEs� �����"���S����} �� �}g�����j���09l? rm��5���6�����zJO�,����� � @iZ�f���2�=.�V�_����`�����W��; .�b���G/D�����2����Hj,u0t�/�$!���b�N��0 ����)�pQ�� ��x��T��b���(YYq8=[p�����>���s�GD�(B�w�U� �t7�����K<������K�, �rA�:F������@��zO�l�T`���i�*G��1�M[�-b,��j��M�X�u(�ei�� � ^�5� ����hW����M�R5����F�" ������:Zy�d��M�bmM6�-2�oq{q�K����'��E�q+�0������N�c��/
i}6*_|��Cz��vE�\�p���}a� � Gw�z�SlC���%ZK���J�����T��:���
��:=�������}���� [�nk�?b?�����|.���# � #�O��WcKk�$ WSQ���.�pq��;�n�xp %o�a!(%!a�~
��:��y�+6i�`�r3�u9��81rm<��(����/,7�1TSU!��[�N�������-��Ku�Q�����v��wN�5aOu\6� � ]H�|x��k�Yt��+� Wm� ������M�����9�B��
QR���Z�f��Y�<�QHK����&o�M^��X.[��X(%�A��[��74@ca�����F���X�� ����|}���.����\�z���&�
iE�\$>c_���r+���%9�~J��qyLM��Y:A���t(���9�w���s�� �*5����� y����E3 ��a6}I��� �FA������qM=���sE�������]^��x���X��6�M���+��w�D����W��zUf�-�m�^���hc@O��4g�9 '�B�4�8��������'/�'��M��R�&�c�+w���[���FN9 E��\$��������� ������o�[���5���������sfr4]����7��$��DRq���Y��j��������C� ���"� ��t��w�<eA�N�����`�����6��`���%��mG�a�����hpMY���_C3�z�rF�Ch2�=N�;A������"y���XL��������)eS����>����OJ�O d<_�u�PJ�fp��� ��Lv�K�����0%�������������[ +=2!q��k�����L�:F���=� �c ��:��@Q�������'����bjJ�' �N.+2@(T���������v����{��� TdA�&�(� ���������x�m�7>Z�eoi�0*�3go9E ������� ��L�'�Fp���#C��jg��S�;"��� �H���8m���d�4�8/J{�0M��F��S7+������w��
���������F�RT`<����xA��nlzo�� ��h��8lG4��hz�3f-�H����/X?����kC���7n,�P7���K�:/d������ �ds:�������Uk -��{�����-[��t�M�Z*� � H;���;= ~:��E���T� <���J��5��x��D2s�e�{�J��u��c�#�#~��� x�*ek�^��o������h5g#�O/��t!���Pu; v,�7J���.'�|�k��:Zf�v1�� �����g��#�%v��F�8t{��\����W�=L�r6�C����g"����*'Bh ��snKj3��p�V,1�,��4�w, �k�F��c��!Go�7(5��'6m�����% �DA�G�x����\�m7@��^�� � *2���!�u���j��I3l��A}S/�d
��4��jUqO���� �5%� .������|�D�B���;oQd ����ds.����C]A����(�,'��z���n@�K���H��2(9LC�A 0N�a v6�����}2C���J� �M���y�������,E\��������Y
�)�%����"9M��(2 �h(��]�Pb� LC��n�)e�G�� �4���B� ��
�OH*2 ����-e3�������e���y�_�� � � ���s�]�iP�A�T����J#�U�Gj(IDj���Z���.��A�i���X H� ���: ����Pq�+������k'�v���C�H���m,,� !gTr��)>���)��(lOY�'�����bmS��RF��H��[��~������iA ���r��A ��D��&J&�����8u�#�t�}���-2M�`_� �Z�7�q�h��NV�r"8Y12/hf}J�j���)EE��V.�+��n��h���� �?P ��%'�q��J�_i�.cr���� ��j-�� � ���(� ��x�/<����Z52"|����8�x�H����JGT��� �O�l1A�*eK�����u>���4����|�5!�]�����b��}-V.�Z���*�3;\�_�3�m����� .[5m�&��������� �!K���F�������+cG���u���F�A���a��k�is���u��tf]��[o>����n�Ov}���G����$�����]^����}��5�fWL�z���Q��G����{� �'�ov04 �cjO�r�z��S����3~t���� �n��_R.Y��dQ<y:qsE�aP��]R� � HG�F�� �J��x�p���4� "�m5,^N������j����H`���4##�3���dJes6_��� _� j�`��%�Y�E �� ����$�T����;�X�oa���� n2����I��u���5W�p�#aMJ
�p~.C�E��6DI<'=K��b8�a~��"z��o�-(��u��R�'w\!��������:�
c2F�(� ( �Jq����4)�-�� \���� L�& �$���k�B�k���g�w�N�tvb7� � H������~�������D�g�9� ���z�@��{]K�J�2��:E������W)�B����l5}�U�V�(/}zi��c l,,{��S����Q4M;�-x�2���[��d�s��*p -i�&R�,�������V�]�IB�BQ��M��A� a�l�-�O" �������6������v (%��u`�
��OLPY�#�fe��k��Sc��^[B����T|p�z �x�p����|���+��<mF�
m[� � W�����J���/ ?~�fk�(�l�� �1��B�j�b���y�E@jX����x��w��6�+T���dl���m>r�{�[k�
+F}
����t�k�U
n��n)O��2M������$�������/���PT`��]�V���Dl�_2��]�.��V��8��#�W�z�;qJ����_�SA�|�qU�/2�d\�X���� L ���-< AA�!��2�v�Z�t��������A���eA� AgzJ����N�H: �yL�A�*5Iw�V^����n��is� (�R&����1���oSb �� �j<���T�o���i�M���xr����������Z �x�0�k�)``����g�����7���N�^�>���,,��WD������x��������T-�� mE@<@`a�\\�� J�R�M����3�� �N'3�4i������������V�<y��IM��>�� �.���U�f���.O� p��0;I<�J���1N�8�2��m�WO�<��AA�;��2��-��}{`` w�_�~��#��U���������N4������z��^�kO74���%�A���������b�CO��[T�Jkog������*8 `o.��a�ae �#�8
7 ��;��!��lS
������%gl�'6��HZ�U��I�B �9��� !# F� a�����L?5��"Z�piN�5�� �� �D����
�D(�f���''L������ � � =�D�F�Sd `��Q��A�:�����\6� ����q�����F����L�~k���}���Q]�nh T���J�1������ O��&U��D�q���|�M�d�T��s0��x���S +-�n.�7���Wu����
Z*�I2�P� ~��
u ��Gfcr�W��f�:)�MF M� �6
����o&W�_?>9czt��������y6/.f��G> ��������o
AA�O�N� 4��J�� �����i&A� �0C��|���8����
L� %��� +�7�.#���f?���_���������0iD�B b qX" ��� �� ��*;L��W���9���U���TC�HS��������K ��D�>�
%���^#q�������Gb�`��*��e��uf��@;]�� ��k���}���3� x`\��� � �'��o��t��W^y�g"������$A����sb4C-G�9Ta^{�Ig��Zg�V�h������11�^_���=�����-���1��&^Z<���RXT���ti��F9J���[Oy!� �!d ����bV��u2 �
Q�M4S�~-U��V,����X%3�b-�^���.URz��<RN A�j �����������2B�W>:/f���RM�/}����S�:phN���� �����D���E��`5����5�6y`KT��V����� � �_�]����CBB���>n�����}]� H�2#E��d#O��(5B ����^6.�� �Z�O����� �����[2�S�
UR:4���P|�S�D�~%�>��>]�����_���������E~15d?���!i�H
4P]��R3��������� e��� $z��ua��{
hYI�����"��_|�) ! ��
�����6�~����#� �t��2�&M���x�3f��]=� �����s
�=�M��RB<��a7�� f���8u�T�j���*����������F�q��0sZ���^�i��Qp$�X����������_�[k�9 �2[���g 9#��1���'LBDd�� ����%h ��� `s2�I�1���������n����?��n�������4FA�^� ��[�V�\.�I5=���k� �?�F�xa��"#a���n� �`�yMk����7P;��Ht����s�RB� ��A 4�����Iyg//�����pj&��T� >��O
R��0����'G�fO�� IDAT�0����p���
;U_(��8�E��~���������� j�[v������BY���R�{N�:��Gs�:��fi:���e��3�r���cKj����������v]�� �����{kjnn�(*(?w���+� �����||�%���jF�#�'�Z���� ���������5��R�F�Qw��I
���:�-��>0=-�+�L�%�����=E���&���~�'��vv$^
�$P!i0�=h��vX�N�0
)Y���?2�k�4l5����� `"U��;�w5���}s���F����~������w�g�~)��9h��* W8y _2*��` �?~��1Y�&���� }�<CY��W�(6
AA���|���� 9r����
$I���~��%#F����A����J���g��-� r��S��$.1���H (3�irq��# ��A 4B_�������-BIi���#���YfOzZ /�iQX�����{�9�+�b�tv��n���A� LY����V��`e��\�( �i8��Y��6�.0�vO��%5^eZ30
K��1V �{����4�� KN����wC�x� a�� �*[Z`.\�p��w/^���L�� ���4h�P�T{<���-AA���������e��l�������/�|��w��������Jvv���A��bT�r�=a� WK�$����nV�����;��������V���C����� D|a�<��!�����(nT�c����)1x�9-,��qvsl��~E�� L��0<�n,c ���I���BS���Y�4�dV6�Z 6:8z���wU��u[L%����Wx�i
�%'2o)m��9������[
.����_R���m��E��_����z���=$� �v" ._�|�}���������X���o���k�Av��uj���[� ��dgg�o��w-�x'������� a� ^�v�����-{cC��� �����Q� �+|������%��&7����R����;;}:O|0��� V�J�=�E&sZ�� e���A��D�v�-�KY#e�X�������i����e����gsv ����d�6���!�� U�
cW����U���$_�%a������
�2��5��?�������
����o"��SW�\����_~����zJt���,��33mS _AA����l�
���E ���,KD���eDD��*�� �C��K��\��7�w�iD�I_��o���������}�7���#���iJ�
�#��|�}7I����gx���� �:�g+�H6l����(L���JhbfZ���nH(�M�J
�G2i�A������=�������:���"-q��L�mb���)U�
/\�=\�K =z�� ����fgjF��4�����$<AA���kOA�� �=#E��98�U��{�fT��?o�]x�0����L��2�O�v R ������R��hu��U��W"���#);:�tSN���U�W��V'Ml.z]�1&���d��(Ckx'�� ��2��L ���z��i&�[ %�� !� @��^>��&���dhm�1�� P�\?=�~��e�4� �2R�}��/���3�<��vQ ��� � ��xg>�� /L�<��{�aW~������n��]P[��?
��S�y�
t^��V �p?/(�����NT�?k�������^������?/+��� %�[�qj�~i`R�� ��b=z��k#�h
� �C���l� ��- �..�3
,��ub@���=)��lL.v�3�x�F '�*��R���$wf���-�\l�e�O�<�K��Rk�R4D8[��@Z��#�z}��y .��`^��F!]3}�L�f�� ������w�:erss |�����)S� �w�}�~��m��uMm� ]�� O��% �>�H� �|��tA�� $W����I<lr��3J `���� ���W��/������?�z�H���i61��D�Y��i��3VL���+~�f~��9�Q�M ��]q�q���LZJ��Yo���qC ������*���
4cw�Kn�8H��P�N6(�����4?k��� *2�����cAA�
�D�V��?|��gs����#G���C�PtMm� ]�'�<���<u!����'bP���3J `�9 p?�����t�p]UR���� �`^�4�f�R3�� ���\� <��2���!"�=�+�0��T�J��o��������J[L������1� ��# �'T4�-*H�U��� ��*�,�|��
���>�h�"�j�� � ��^{������������jA�O t��
WAHn�<M q�hT��������v���J���C\���� 3��uZa�I��j���Hw�6��/���I<��.B��;ti$�����e�$�O B� ��<�r��#[ 6+�&#��2���m NrU����@��P�|���D �47���k�$L��-S�#�
AA�?A�_Anp�[�xb�+nx�
Br��iJ� o��n������T�E��[[ FJ���[O���a�TR�p�I�, `o.m8�$��0������MjF+K ��B%�:*�� ���M0�C�fw��t9g��a��3��g��[_��p��%)U�
�D������f�MOJ��E:��(�z��A����q�����WZ����4� #�d3�dlX� h{��a�� d�.wBA�>��Y�l������jz"��� �'>�����se$@�� t�I������uvK]=}�+��+��� �����.�E��:��k�]n��h^
�� zs�����Mr���k?��MQ��H����s5o[�;{=?-�� �?)�� ����
HL�����_� �j��(9����^j������sof�����������|i3�d����NxX dx� �i����[�@�O�f<v���P ��� �g�T<�� qP��R������BA��?����S�����? � HW��uK�,B�\��:��;�%���z ,����D�^���GK�O�R�����K��a �����i0Zy���5��vW���)pn6�<[�*2 @�� �����e��(�{��Iqr3��� ���hW� ���yx A�A]��_��g�WqK�fY8 �R������e+��6-++���i��`�; �AA|�G� EQ$I��VA�W�-B���N�]g����#�E���5�5X� �;�������������HAo��&k_�����-k{?/b #�E�w�����R���C;�:���%�d � ��pS��J��2+�/����YC���>��}^/����8�a����G k1FI���i"�o���`�Y�9���j��% E��L]�f�: ��c�)�h��qBE�� CH�i��'B��}!� �� �2iiic�����_~�e��g���uU� ��Y��
o�S���� TU��4�4J�*�[u�;x)�`aY���rN5i~%r#�l 0���d2����R��L� R�0���"�'���7<�tl?O*�d���x���Kq� ��� m2RS����u4g����h� R���=� ���9���-
; T�~��� ���tn�F�����NQ4��*lD�c ��!}���x�B��>���m0��.� � �#�H��5k�� ���#yyy�g� ;���A|�'>��@��6�o�� �(��BU��%uX����49M�D� �� ��wkfU�C��uw ����,��\�����L� �'�o�P� �j���"����k������|��s�}y�� %�O���[v�I!��"����L V�Y�9�U��(slI��s�3� ������.�=l�M���p% x(�nqJe�Uu���� �H{�����C� @��"�G 1��9��&��?���8�/�� #� ��<e^y���V����~�{600��E!��� a�r��������H�=�D�0���o��;3$D��23M\h��*@mL����%y~�v���Ol:�5�]~�����i�ySC����;_�|�H) ����gP�5S�����n,cb��g�����)�W1�!Ht�L��9j�H'�(����\�2��Kp�7?D,�$��i��0�k� *���R+�6��^��h� �� �S���2�� �U����c�������R����e�1�������mxu�����lAAz.��N�2���@�"����Wx�F��f��� [\!�a��L6�����U�$/�m%��N�����q#�`�)��3���T �� Ub�b*��mI���<���akhs�iE�G� �&�)5�bzy~�Cs"�' �?��m�1�=�mkZ��p�#9� ���=�C��
nQ��� }uh���8���*F�a�H n��W�cM���^�T��� dA���N�y�������A�x�F��f �y�@Ui���V���CbQ}�Y���I��6`B� ���,�t�6�����3��tfE�G�+������8ibe}'h]_�u� ����x�� ��{���ah>�D"k��`{�8�������gX� �&2������]re��FU1 ���?d��+V�I����E��AA���A$������� ���x�F��f ��
OJ��?�{D��%�����wy������ c��7�\�(2<�V��Q�\���f�'��;TG_��/��+�����%��5 P���1���?��� 6)�c�IV�im��r�,�E��8�J4�`0L�0���� ����i�CQ�f��� � �'4-��D�� �5D�S �0����
��p1���� D�;{/:+�a������k}���g�(9��F!]3}���8hy'?��`�&h����O �Bk�(� i�]�
Ak�|s�9�R��.o0[�N�-r��|0�gw�i� ���V��d|@��z)�A? R�d��G @�!�Y�� #/,N��UZI� N���3�d�2h�
��8qbrr�����+2���:��x��"�&�O;��K � ����7��W�+**N�<��� �N�z���6m��5�!�� *�������{v_�������bd�r� m�Z"%�p�d� w"�W��;��+r���H������N����TR8���w�W����{�\�rT���.^S m34��@��wu6�}�!=-.o���!j������6hfZ\��~E[S*3�/�� l���L�o��C�7lH(�T~fCBqq~����7�M���c�{��qh�������y1���Z>����*!� p�@;�fs�9���%� �1i�������<�*2 `7��W�e��� � H��N(������ >���>}�L�2%77w���w�y������ol�Gfeeegg_�Z�=�*���;S���r�u�W��7 �4" �(���t5����^aG��.��q�@Q��R�������s�!�
_�*��E��SPT���S�VO\������ ���"���j�)�opY�L<9m+���[�u���4m�����;F���������1cf�fS�#��{BR� z���=9��eF6�Q[�>}��]K��}D� ��)�� �i`R�� ��#� rC�������~���2��2�&M*,,�j� ��Z��3g��l�<y��={��D?�?;������q�p��H>���� ��:=�&�Dgfr �M[���]8#H�w� �& �f��&}�s����}n� u
Z*�I2�P���i (Y>�|jN(~��2zu����1�X�)c��[�O�/�K�?PT f$I ��S��6>#1
l� �M� �j� �� Aj��My�5��Oz����
E L �G�9M�y��!� r��o��3�5�Z� <��� �R�T(� � �S�*I��o$��Hro��i���- +�/�$��"7N
�� � A
��p�����e�x�NR\���Yy�As���S�e^���Ab�� ��jDE��`g��9��#��O'7V�R�����n��~j��?)X��:���un e�b���.��aP%����*h�� � �wx'����3��Z��MHH�qQ� ~�W�H�t^�������/�c� �z�� y����$)sD,��$=5�_� lURz�8L5�srU����J���|x!���J� ���/8i+����9�����tY���l������[F+>S"� H ��tVY��m��������K�%��W� Ji� � ]�w�P��� 8p���$&&�Yq8 ���]P� �_0#E���}����� ��K �x+��PD=n;v�vM[M%��- �������j�C�5����A��x
����i ��������>g�����n\��K��$�J����� �j��K#�=�s���+�ghN�T��� SIjF���R��x�R�*��[��b�Y�N�SJ���:A W��8�W�+B�e<%-7����H��},� � rC���D+V��7o��dZ�z5 ���/X�`��!]S� �Gt����J��b���IBU���iFxy�)^����N��D�WH��|�V
�n�mdI�XM ��RUR���/� �X��������l����� J ���[O-t)�|��E��h����F���������}��<e\�>��A@�N�������C��
fy��mB���1-~��3�JeAAA������ � =�N��477���S�T����}XS��?����A�F�&��2�
z;)���M�d"�m�M[��J,�N�t��c����r�R�sn>��&�I��=O+��x�� �� �g|�����#|%m�l�k��xqJU��Ve����>nH�^�j��D|KH��|mi��_~)��3���E1� � �����w��g���W�y�#+H��������%�p�7y��-�TbWh� ���0A�Rn@�h�����\�C)���������l� ����h�h�q$N��+�ZI����>�u��1b��H��H��$�_ ���>?���Z�jRy��F M���e�������*2 ���\*AAnN���;Nt#�����-��� N;���8����b���v�s%��L[�Y
L ��4�$:K5{�)��l>���8�VR����'a�K�p��J��sS/�����bW��l^R7����->�����H\������^#q,J��T\XTP4<]�8�*Vi ��+,rI�+��*�"�E��(FA��x'�TTT�t�������S�Nm��a���?�p� �.�t$���11���K>�O���@��r��Q�p��Q&^�^��0��M>T��~�����x(X>���
������+����2��7M��}��@]�>SXr"��������4� �8�
����\���
4� hH��<��-0�P��������os�AA������]���y������g�yf����v�Z�n]� �.�t�`9�_���)��]� 7pxS���N��Q��zc���E��%.�<& � e��J ��+r���H����)���JO���6�o�ZB���?� ��o��(��.]��P�����n (.���� ��8/B�i�qsH'YR��
Z�U��*�m�����N�<S3����g �� ��"(�|J]���� #��/�Tw{X�hT�AA���n�j��I���Z����?V��s��1���'O��gO������X��Tp}pG��]ei����UJ�y�#��A�O8Xa��7u�v�������N[�.�~X{Qt[S�yL�I��%.�wmz�'���3_��5�:
�<CYZ^��a;� ��(L"pa%�x�N\1D"�
� �S��HC�^���r���C���\ �I�^�P-�AE#�&�-�=4���
�<�|�� >��
k��}jke��������*�^��b�� � >�?��{7�d4�Z- <x��? �R�P(��4A��d��'.� �5��k���##c��0����+9-�}��" {Q����D��L%��? T[��_#�i�_�3JSvt"0��O�!������ ��b��
��"�/4O!$/�FS {��{fyl� �M�*���4�r�GU �NI��[�(H��
!��������n ���er�}�7�t@����`bA��N����g 444����������A:���H����@�����B� / IDAT�U���
X:#��tACw�
ex��.���������DF��O.����[t/�� ���&XW����������Pbd�_���m�a��78��}��.�� �7lH(n.�1.- ]4�c3�dlX�I�*��$A� �����t���]�5��if�J��RAAn$���III9p�@IIILL ��p8�����0A��OO�C����tF�����`�Gr@�"���� 4�hs�����B��>�R��u �?8[�%�T�/��n(���x��E h�D�>^.U��p )#�}���bRg������I��w���x � �X�>���uP2sCl����O��YI�x�6m� ���Tj��4� AJ�
p_y0���6���8^K�z#�JA����: ;�l�Qu� � 7,�u��X�b��y&�i��� P^^�`��!C�tMm� ��'yI<�3��� X�x#�1�a�������D�j���<�x(�#F��9�pa�3FG����X��u����yd���r��q��#^D��� ?����V��
��#��LdlbT�A���'a�2�+���}��� �Y�S]]���+v�U�Y^u�?�m)K IX��� hL� � ����(�}�v����� `s2� ~��
����%�����(�]�JO�;|�~���t�� }�J=c���i����������He &�� ������9��������Y���(Y�[wjg����03��>^~����M�XrU� Le ��$�;EU��I��_���,���\�u�!�������V|�]a��+� 8Pg�}|9��[��v�������2hv��� � %� � �1�ex 6�W� �t����u�'f1�]�� �]�W����.�eaQy���VFa���/m^ve\��W��x{� �v�~�j���=!c@�P��i��n5����q�L!Mbo�1dV6� �t� M A�Zu�|�c7�e�� e�l~�R*�l\����}�hT6��W4*���������F��H. �����W,X���`��������)��=���
|�� � �t�eA����K��+� �!�/` z�^�k�5Y�� ;� �}�>K��!f�rdn.� a��W��J+^�u��6r�L!Mn}p%7 ���OfAs[uh������}7�{ ����� ���=�b��sI���+��D/�!w��=Wd 4 A M�H�WZ�5����\���o���A!��>� � ]�wF� p��� x 5��C��������!� 7$ z���U��
WK�$����u�������g�� ��n���b�+G�� �rt���Q%���h����w�P��QA�) l�AM:!� ���w����6�'
��|E�)��Cs"��_�����vc����
��U�[�u�p�� e��� ���Po��O�����X�������(�,H(p=h�
���:l4�m�;V����� � �ux'���?�y�����[��2&L���O��0A�s���A���aL�������K ��!k�9��P��3��A���i��r~�-������X� �uS��f�����y1��rRF2 L�Q��RM�T �?�4����Z`�L%��6�(��1���G/�r[K�-��? ��-���d�v���K�6�-_�(�0+� K�~Q����K�
KNd ��`���� �IS�<�j[�o_ AA���w�-[�l��5!!!����c/]����Az ��0��������fQ��iq�� ;p�3��F�����X��
��q�J����UM�v� t���{��`|�M��)R3� ����u�=��I5�����,m���L0xi�� �U�] �6�����9?>^�����������JE ��Q�Q���R�\osP:�Ag1� Tis��<�s���� �w�������t%���AA�^x�)��� �]!�l�I?�E�&Gh��0���j��� �0��<��c�^� U]�f���G�iq4M��KY����_�<�-�ZK���?��e��������� �Q�����]XR�����.~��F�9}i& A��K����E�R��_<�)8�i�� ���6�vv����"%��BjF���-K�vl"�� @��IU
A��{��S�T������� Ld���A!�_ AA���w��L&�gAt ����}�������k-�t <��!�����}7�5��C���S�9c����_��W��. ��}6��X��M%Vr�W��% ���L���b���������U��j�U^^����� ��b-�M%��#s�IL�x����8lt�����w x�e�Us]��m�� ���n�}�\ ,!�����H5���1�����m&/E�-N��V��,����_U����nl!��7:BB����Bys��o���qPAA?!;;;''�zW��D������/����]��?�/s���tE�������Z�\O��� ���!��j�����������kF ����\ �*1���3���$���7�
��JD> lk�sa� eV�v���cg��3u���KxIL�x&��x��U D�c9L��:�"�� ����D�� ��.Q�p���r/yN�:�B�5 � U�|@�F� ��*��A�W�,z��\E:�|�a��EPL!�8qR�\��������{/m�� �t�fAA������vN�gC�w��[o���O�� � ���o��r��o���kjC�T�����YE����RI�
N��
r� E����aLu��s�e �$h#E� @��`�J�&4@ i|0�g x�j��j_b�5�r�W��UR����7p�l�~��X&<J ^�����M��(L�<���h��e��A�W�{[�+r������,v��U �N��A :0��=���!� �q�����_m ����a�� d�1�� AA���;������>���W_�r��<p������^��1��"�x��J���j��������%y��;#E��x�*%�c�@G9Ta�|�S��������;r�� �p�Y(����"^!z /<�a=" M zJ�$d�Z�yO��6Vo��>&���%� %����Or��3��Ta�p"a��o�r����eJ ��,���,�r�{k���d�~ ���@���� )AA�&�;Q RRR
O�>����gee)�]��,� ����<(�C�������
�J !�K��3H��HI"\-�3H��6�M�
� r*,*��z���h� ��OA����P��E�B(�� ������G
�P�$MLB�p�\G;l��/L����G�������$&�x���K�J���o��s�����YZx�b4 �Smq��;���>��k�'iJ��"AAn6�e�����A�'��<���U�.7�� ��W�
.�b�+G�� �rt�NNy20� �q$7��1���� C����z������; �QH�� �I3�5#����IW\}a����y���)��
��J��JJ.
�2:K�i���^���` x�$�@���_g�n�
���!�6�� I�q��gAAWx'���5�����JA�+���p.����4���'� 33��S��������<�b S�v )=-.o���!j������6h&���T����P�A���`*�?��� ����M���!�����DX���{1�� �Euv�� V[���!��S����% N�J��~)�Qk��K�y�pQ��$�zrt^���������y1�7�e�/)J��v��� �\����Z���������p�&\� ��0�!T����W����P�zAAq�wF��e�����7�x�����]I� ~����At.��M+ �*�k���>W�eF�fM�fW�1�JM�V^�f����8R��s� ����Xe
�i�N��'�����v��NH^���)���E���@0���Ec����������p��X��ma��YZS�}������c��ACd����lx<�VB�M�\8,�� r�6��@��Uw���� M��:(��)�Rjg�Tw�+2�5M�����5B�{AAq�w�L\\���go��V���S��Az(<����\�P�q���Q�W���X%M 7}i��y(W�I9c����_��v���Db�B{i�q[��g3��|�����g�����G�5��G
N88�7
��?��������>�.;:��2(�����_ _ �co��RF ����;O~I���;+��������&���Q �[e]5�LB8�A/ A�����&��������������U�b�J �@Tp�����1AA�;�_z���_x�����.�A�`Ly%��L��K�*�jZq�� 7 :��S��
���D�RA�Dt��7�$�����.���&��vj����Y/�*#��u��#JzZ���������
�O~����6�}|�e�}d��x�R���\:@a����[J��m����v�~����G���0 S�EL�m� �j2+�6�d�s���l�esIiAHIi�:�/�� � �C���9{��=��3n����~�5V���G{"�t*��&�N���2*F�w��'���$�7���W 7<D���Q�����w� �RB���
�5��7-��uly���gY��:��t���T�z��'������m��!����'d���kT���i���/wb�Y�PW�M4����2[@�L�I��yFRgXR��`k+���k�����~26�`��i{��~U���NL�� 3}^-� � �v������f��y�/_.m�h4�$� �Oa�����^9X��#l���LwT���~�%Yovl:gp���
����]!���I�6����Ki���\#��m`=_D������M����u�N�M�J�~�R��&��x��'���I������'
�A�5-���_�(2 j����AA'��� y�Y~D��]�|W� �Gx���<7��������a�����1�]����Fsl�2w|��i�;v^�m6����"�U<g����\�C���������
w��������Um��uo���\�^��m��_n�O@Z��k� �������q��� �i'�_?��a(,9A������TE+lU��k��
�F" �D(���k�U�T����F�AA����i~�����BCCy�������}ZUO� �{
� �v^�9�����q|DW\��9�����8O Ei$ ����W���� �ky�R����g��B ��������^e�������8�&�� �5��J��\ ������ iW4�SI���\V`"d��a���oL.�_t�T
J��:����|K��w�\���+�
cq�6���L��D��� � o]4k ���}Q����AA��|��]��P���Y�A�z�E�+���QzbX��T��9-x��:�� .���e�\j1 <B� �(��W3XE8!P�C<O����P��k������S�v��
��F���rn_��t(�v��m��'9�k���2�J0�L����(� � �� �D����_~�E������� ��� x>U��K���H'�:�� ��3Q4�+�M8x�h������k�m +�3z�K(7 [�BYHU$)QUZ�x;Y�`7�I��
� �{=���r^ ���% ��~���_����OQhp�h6 ;5�_���1�)���#T���������z�n�G���_�"�xhJ�>�47���k�$ '�AA:�wF�4Mggg��;�����_~�:u���+��2A70y��j��$���9��#��Y�jo`_�bw�iF�(g�jd$QZu���.5��v�.�7fn=]PT����� v��9�0 �a�����_�V�\ �$j�gn=]Zo�Qna�b�����G��$_V3`�G���7.~� �,�Ig���L<vj�R^O���� ����.,91��/u����ye���)_>#$-+LB��^M�v������f���{@A���N�Y�f
|��w�=���E������o�vMm� ��\9�W�����{y���I�Pg��N�E�5����$To���$�J��of^��]*�Sm�8\�E��Qe���� .�H���5Z$��!�
������Y��0��Kv�3X�@���\qv�|��b��,��_?��L)�+H�M ^��XN�� �# ���\�N�t��}�{�M
�7�d��?r� �b���� 6�_K��Pq�l���%41�M�� AA���;Qf��-K�,!���O%����������t�9 ��:QW��b�?�i�X�f?���|��+���/'X����f�m���7�UN9�;��,������k��������h]�P�@��,�.�A������Y�M���U���Da���U�%r"(I���8Yq~����O�g6$�7�[���=!��(a�����6�]�^ 6 ���������j��[}Y�� )�"))�������M� � �Wx�)C��D��e ��x+� �Iwz_G�;�
a^�B�lrJY��.9c����_�c���-
����#�<d7�5��� W����� ���3���������}�&� ��Y���j~���+q��9���U0�3 �����\�v]fx�5�w��/)z�����-��������+nq�^��4�j
����� ~�?~��=]fm�> ����5�!$��sB;�� � �x�w�22��ht������e��[���A|M���t���g��f��=���@e��n\����yY$�I9
;OvIO���6�o�Z&!�����
����J��j�x�RM���Ub�?��3���[ ��^����4
����cJM���+��"�j�F3}�h�!��C�F���2B�W>:/�lg���.3���x��=�%'2oa���!/���b�� ������?����V� ���n���O<������:l�F��
������su�2�1W�M� � ��x��k��-[��Z�J.�@SS��9sf��=n�8 �4i������R�?���K��:��
�j�0���#;dI�����X�H���^p� Z_����������L �G��m;�L4|���QNM72���������c��3?r�qVO ����/�6ym
)i~w"x�g�3���D��v�=�#Q��� ��c'����� �dUk�N��g7�Zy��(���<��'�y��->�������m|V���M��! pp.X4�l����l �� ������o�]x���47'��%AAz"������rssw��Q__?n�8���}�����{���y��/�8w�#��=���"�1��j�������(����zu�T�R3
���J��'�2�q ����z0p7����5��H���a����a ����7��������+r��R���&���1��OT�Wy�M%��#s�^�a���M�7������$�u!�����h������}p��j����v� "5� R����S���2���g�r�.)�7���C=�AA��?��{W��a�V�X����^z���C���'���WA:��U�e���6�w�26��g yH������g/+��� %�[�qA�sV����� SPT���Y3} o�JTI����y1�����t�w)�o�^��*�
wQ�M��h��"�G��I��(��c%���?]Y�R��UYd����T�r �PT w���P�$I�^�� y�M�.���������9g�|j� ��GW#� ��x���w��~���?l�0���+Wz��
O?����B�v���~k���Pk� ��!T%������3�V�a),*�d��R @�-ba�� I�� ���5NO��i�i��
T���?3�?`��������WF�g�Zu\���R(R�R�M aLy'��;���e��"I����a�n HM�f��4�2W����
������4�\���2������|@M4
����!��C���#-�b����� �g������[��"� � �x-���VW;���p������U�$�SlC���_���U�tQ���
��&�y�#��I� ;X�� ���qC�;{/:�#�K����@�8�j����8��U�[~��P���E�#fpn���p�+��1zp�k�/7&�/Z�����H�N�����~���'$oy�g��
������`J�
��x��zx �F�ts�������"G�� IDAT�W���|U3� � HO�?��{�)�x��S�N
:� ��)���>_W� �M��
��V%Bg���n"h��q!Q��*B������E��U�p����O�gnm��@�4���Ny=?�ZO���r���c�n�[���f {s��H&���lo� ��"���~n��W3XEZ���;e�a~�f>�8`�Ra6���N&8�������b�����J��" }T�a����u4�sSM�R;;#� �_����O��i�&TdAA��D�={� =z��� �� YYY������I���B���)n�
*����n:��e74
�iMZ�� ����C<Wvbd�KT����n���q�[�b�Of��g�*�}�����e�'#�}�TR�?�I� _���n�,��fZ�/��2{�I���0^1e
m��2�k���@��?Q�^�6ED���*qfj"x2��� lj;$#��b�:�g��A� �I���I#��E�������^mlX�&������C��M���/����v FA�a��������U���jwbb����y�7n�]==�Td�+��F� 22V9g�6\-��D�Z���3n/��C����tF�������mbD9G�`(�����#����5�v~5i~%� {s)��1}1��+�C�U�Tj��YY\��Tb*�� |���5���r~�-�FK�l������ l��T ��'F����k^34'R�!����] wm��-� ;9�������� ����+�?����P\���;3�P�Tj�V.� 9C�������;Lx�<p��! ��� p ��Mf�z������N�ZXX8z�h@A�����f��_�Z���S���? 4h��I����������5���!r��[w[��mW���0(%��)��Q�p�� ��eNs�
p�K�?r�o5�%����b/�����`p�=m3�����\�2+d78�e������8l� %9�
6P��O� ��e�q����� /G.�x��)�tl?�@Myy�hb� ��������p#�����bjzPzRM �ff� ��w ����`���>KR��6 @D��) 9r�� ,((����]�� � r��N��={� |p� wp'�^~�e_W� �Mo\����������n/��}kR��~��i�T��K��h)�4f��UvX�L3����U�4
�L��P��m���6����)��PRq�~��#����#��_���?n q���� �r����e��ML��3��F0 �E���U�jU�]�<n�O��2�$���������>f�OnAA�f�e����SA����\�~+����I<�Lr���>s�7���O���{e��*) Zd>mk� ����\��2V��"���seG'������J��:���t���X=��N��L,;2�Y O�_:���$����h�T~�u�a j��d@��" ��?�����5B�������w.F���� � E�}1� ��XBNS��UU�� x`��u'N���"� � ]�w�P�V�2�� ,�.��3g����.�����Z�x��
�jg�_�hi��g]���|���Mw����m�� ���p��4�����s��TR�p`�"�%d���kXG����y���=��MD"$s
����?-,��`�H����mz��������,N�PK�S����{6����� �: x�M���-��:3�f��X�!�-7h�$������d����i��TE+lU� ]��HC���7@����Q @H�k�&�9f����'O����DA���7�����o���{k�Zn���c��^���� ���� �������"�(%���Za&����$����������_s9����S����K���������}�lo�(\��
H�e�u���W�
qr�%��4��N0mk �^��J#%�-�����"d��m�9��X���H�DE���:@q�~����o��d�������N��������������O��Z�J��������;����
���?��L#i�x"T������5g���� ���w���w��AAn<����w5egg ��z��7�|��E�(���� �W �0r�����Q�Gw���_��r"*��
7J�H }���[T:@AQ9w. ��`�(�k���ex6 T}���[�&�O���~�"x���'&H[��Q�>��/qw9����|�v�H$ ���0--\#�).��4��r!��qxmDd����73j \����t�nZ d�����s&k \4�L�����(v����S��������L���;BA�Q��7� ����������� {����W�A�C����kk .��e$l �~�P��� W
,���:���% �pd��nk���Y!��Q���v���{�ze
-y��0i�C���H��H�D�DV��*ve�q��1--���ccr����� %�g]h;�p�,���E ��h��D���t��r��dm����2+��i�"�R������������>z���;p;� � 70����;�_ 8r��[o�H�d}}��%KF���!r3p����Uq����������X".��n�1F��}�5�X��k��+2n�uV[~����5����08Gt ;|����+EF�������34��7����[]�_�"��������m�J�)���,��$D8�$4��\� �4���3K6X��+a��&��S���+��/
��� .6]�/)bR��m}�65][3a��+V�"� � �S ��]TT�l�����M�6~������nQQQ� �
�����e��.,�A���B��W�������
�� �Bw5B�!k�9����+r���H����B�� 4<O�� e�������������
�E��Yvt ��1��q�M����g�������?3�K�)�o ��&��RDij�r@��|O�� Bl���)<I�O��<E�;�(R`���l����L����nvC6�|?�����S~��c��SG����.���4�p�U6lI��2 �D ���lz�1 �+ow/-6 ��n��S���~ ������ ��!I�������|�����F����-���:�?- �l@VB���f ��-Ee% �<�y��Y��o<��C� !�B���w�N��v�����4�������m����������B��������EX+� xc�$$($��A{����1F������9�1*���R�:��5��a_�0&V@��:�Z~����8j'w$���(�d 0l�{��Z����4QK���Z�H�ff91-f�;�X<��&�:���RI3Pzs~��9J�g/?>����S#����'h\�s��J�56���qA����p���e? ����f&M5 wh���
����!�B������)c��� ��������#��o���|_m}�@c����4~_�m� c�sG�D)Eb��R�r�'t��~����������w�� �n���4�\��k��� ��8H%��HU���&����#1��i+�3��R^��%�v�������2t��lD D2���?� �rj���&�� ���� H��d��$II ���6">�|��JF��{)avL����69�7���$;�%Tb{��o�6 I�W� F�_7� � ����:s �B�$
�aN��_�� w�B!��V�=eB��JS�RN<F =����xy@=�= �; � � ��B �i��Z�&��v�2�s����6��AU�r*���R^l8��t�q+
�s�&���o����#�xo)"k���x8��[�8�:�EzN{��g������}�%�sn���J�����l ������?�^7��'3������;@�K� �����V� ��(�)���z��r� !���,Q��{!�B(��4h���{�� �����?--��w��V�q`������m0������ ��?�A�ar^��Pc<�����R>K�A��l
�������XU�[[_kK�
�>$i��h ��uI�v%$�T����!Qv��b �M?8=� (��&��l!��eq�6���j����_�������;������7�|?w�B!��J`A����O�6���z��� ��W_�_��Zn(��|~F ���6���s3��6��7�2�A�j��*��l>�oL�J��<��E������h5������xB�R�\���<��R�c���J%����� 5��%H����(���~ �j���D9k�-�( B�w��� &,Z�#2!�B}W`A�����;w�^�z��� 0v�� ;v�d���Bu���I����f y�B�4R����rw�������xrE�_�1 Y`9����!V �����'zK�+�m��G
�Y��� ����C�_6�=yD��)���?7V��l�*�-}� %���y��}�S��S\~<��-&� VR��$�U� h��w���t���M������ �-V����������{��'�h?!�B(�c���%88#t��i��G������6���VbU��Tk���� �+M�1����������R�� G�l>�6�Z9�O����Xw�RVd82��`"$*�����Ii���<�9Gk��5�&��Q;2 ����PF�['��o������F����� ��h�� �Fy;k��)@�^ ������ �$�����_,X��-!�B]�������/�P�����q!t��s�4r�"�'�1y�B��/�w���L����1��
*<�O^�/j�]��^�E�������-�� ��� �zeEj�ft�X�D��:I�:�X�7�$��yulD �&���:��a�� �R�~��!�rI�
'>]hD���*w���f��EU6�s��,*��Y��i�����L� @?����uM�w��KD���g�y#2!�BW����V�\��G M�<y��U `�X�M��y���� "�P��4����;�#��<V!��/�[� ���x�Rt�T�ne�w�:G���@��5��M�n=�D���A�>�z,�����.�^�LPg,W�� �"%�ya)/6 ��q���� ���� ��;<�y�I��� Cc]�WJT�e�J���� 56��
�wG��� �����hx1�ZF
� �l�+�G?��S��!�B��2e�j�����~�m�K�J�x�b&@�B=J��r������@�JS�E ��A��T��:��z�`���y@��R^��%�v�T�y��� <� �"fJ�fm{��������0����J�r01KY� �����������������$))!<��hL��Bc�n�^5����.!��
���/2�}�����9?���u >6���_��NB!�P
,Sf���;v� �����}�����;D�k��@��R�s+���u�qP7
�K�y0M��5��>�N� ��c�$L Dl9?���V5u����d���j7 )c:��
������~(5�h�[>��N��8��I��Hoi��u����n�����1��p��v>���1c�<� ���Q��e�P��oKOQ#7/�<��vjts���)��v~�o� /���m �B��XP� �H�[��1�!��u�R�G ]&a��H���������c�/J"�M��+��g�V7��t8] P�l&��x�Ef�Rb������T�E������uF������8]t�{ )
�,u�6l���g<�2A���=N��P�D��
-qcU@���j���J���j ��l�3��+4F�.�y��)�Q��c���h$����A�V6#2!�BW���24Ms3eB( ��v-(��Iw��/��O��!��Y?��f�p{�3S��j ������0(H��.5�W�����'6� ��*^�bRzt��} 7��� �g�)9 0�iD�(v�s���d���s�� �6I���Y5c���L�� M�}LO;:���[� ��#(�m�1R
TS�G�(RB�j� ��})
�8lV��]� B!�
b�e�� ����?��C���-[222�{W�����n;u��+}��� ���_�u��
�����s�#[�8 0[�=MD�S
#����!�� ����7�r�TG/33%�r���"eB�uU�2����)! ��-��8��5lIf�}x�s���Q���2� �{1P�� ���W����7��Oh�~�]�����49+���� B!�
:���v8 s������<y2I�{�����������
�� u�PP�6��S�����g�J��;#����\qI�<�<�53�ggh�����hh`?���y�N��rp��;@$�{�j)+2 ���k�����7Kd���C����� ��.����}oL�����O_J8*�� .�� ���=ID$���D�����.�h9Z����S������v�A]J�I �3���"�����Ks�D��<���(>$��n�B!��5+8yl��D"Y�~���SKJJ :t�]wm���Z�� �:uHo]w����rRt��e�S�xz����j�S=���s������l �2�����"5[3�@�N"H�S1`W�['3��"��_�J����{c��R����+"��*�zd>!�� � ��`G;�C��x$?q���������g#2���[_��\ZZ����|z[�����\�Xw�!�B���2�[o���g�}������eB����8(PJH�d�1]j(�)^0h��� ���V.����-���l^h���v(^M^�O�p����o���c�w�����jHX�x��j�]��lg&g7lI��z�����O �u\YIZ��v�����x�P�G.c�F�{9�:���-��2t�!I�����,M�������^� ��e����=Z��Zkru���Q���O;jIK+MQ)�U��E���{w�B!��DW�2!�?a��z?3�pr������7�7� �{����-��?q�J�����@���:iO�GO}��l6;\�������4�r8����3J�w
�s-eE��������9:~��6���}=au��^B8%�o�0-���������K�
�KbiL�+�U�d�"%'jf�8d �vL�{k{��w�*�nNE:3������}c��X����`[w������J�`�:(��E��_�� 1���F����&��&1�������G��{Cb
%����%�B!<0(��Y��1��8�����(����/���Kv��`���p�DD&�+ f��������6b���fV,;��W���m�4J����+, ��b�������?�m�H�XD R�pch+<��u�<wz�Av������Zj�}�h����:cM�=�{�0�G=r�����P 7j����,�I������:.��)~������w8�L��O��=������ ��a��"f�4-N� ��b����p�C��������j��S����7�|�{w�B!��P`��B(P�?������")��.����c0���VE�ma�/[��?#�K��������f�:#~���x}4���9}�i�M�U �(�y��C��|nka���Zu���������Y��
m,Y�2�/�L�����/d<��6�q9�� Q�+���1Y3 �,��n-�����=��U{�/���3b�W
��D DJ�&��TJ����$ ��0a��1c�������
�B!�0(��YWjp����T ��n��&1L~�}FR����J���� ��"� �Zn_X��L�A0��X��1��u�)��a �G�� �3�� � ���<���}j��kN��)2�gk�]���p _����RK`?7�����6WGR���&"�P��Sk�����'N =z��U��w!�B(h�D��%8�j!��}���T����s�_QI�|�I����mb��l�6���UO�.UTX�� �t�E��N�-i��x�1��p��=�&U:�\zO?��[w�a�t�Y_{(-l9����w�$��G_J��6�
/p�2����nC�)Z ��c2�����A�`}�B������������,Y�t��+��������[�PB��\Ky!7����;�8~��~��� �,���� @i�������� 6}}[s����Q0�����<"��uX�a~����d��]zsk�2t�
�sR2��t��Ygmc�*�[���=���!��L�����f��� �Bu��K�����_a �2�+8�m��^qI�<N��L,
��Z��.hX�� ����� IDATVh�����+�����^ ��f� �������)�������wp��2�$x�*�2��m�����c6[���p`���=� ���f��=�F%��=3;���Lq��]�>|.E's��$��i��E��2���5��J����v����o�����k��!I���B!�S���w�� B�Iy{�p'(����� q]TR�?��y�Z[���o���M�$��kao`F��YxL��N� ������b6" &�s�/e�4���<��6�"������kp���!�ty%��� � �%���xdayI�����'o^QX^�, ��mpE�]B����C* ��%��%�(�$���|�7-3![g�m�����o���`D!�B������%8�m����� 6��� F�ZV�?����T���'�:IIZ�����>j�Yf%��}�<e� �J&^3s8������� ���n3��.��w��+��"V��8q/�� �~��+s���_��5{��� ��$�$��6 ���V�25'%�[B��g�Ir/P�0�G�� ��7����o�� ��L�G }��B�������B�OJ ��{��d (Y&o�n�3%�6�i��+���L�6��K��a^;MU����6��?����b�����J;���*�r�K� ��-�K?����\ZZ����0)3O��I�*����l63�CEw]����=0��`��E5�L4
�`5���MEe%�j���S�r[[]Nk����z ���VK� #2!�B����"������:?!�������4��jx�@�����o�����T2~��
l�� ���jG�X�e������H��K'�a�5�'�l7���R�Y�Jd_8,�[U 0��ost���C��i�;����w�� �6{�W��T���A��{��r.9�M����������&���H���4�LT�V
�����F�$#2!�B����.8{����� �
Bu�Cz���m
f�����u'�~�_Uq�� m��I�J���R��"@E���P�0�&^�� ����x"oZ����=�����q�� P\R��9�.
�����I=�TZfA|H���!I��������l�� ��:W�]`<�� @ia��z �`C�q<�����j ���5I� �@l���N5�?q��S�6��-n 54 ��������e1&�g+��a�D��a��E� y��) ��B!����T�}���~�-oq��E+V���]�%�Y��z�`c���]�U���3�J����.�������Z)�>��y�C�&�*tb�T=��I�������q�G&=���������j����q!���r��j�����L\���}� M���KS}������:����Ej�]`6,5���+����_�90O�����0�+sEqy�9�Z~�e� ��/�y���T����E�Z1t������p+�X�6-N���9 �x����)H����~���$���?�B!�M���w����h���~�m��������[����}_m�����J����n��<^�5Mo�Q�r5y�C�s&d�E����V������ ����I�o�K�(G[7�<a�� �sDoj�kV����721��c6����S
T���O��b����XEJ����]ac���Q�`-M9 ^<��&i���d�e od)/6 �� & p+
�s�&���DI�� G.���B�h1�������8mF��$���}��l�����M��k�H�0��r���������v��[�Q�D�!`�B!��n�eV�\����~����� �B�*vHo]�K��E SRD��O�� "�Z�L���'n$�3�4M<��)����J��$����CQy{��i2 `s���9���e _�if"2 3%��m�O��2_�4���-���+�q��j}v#>��i(5A� ��Z
q`���`b���&4�%�sn�*j������������4������w���� ���D �a����Ej������o��Qb)��m53���Ab��
��a:�$�frd�0�1S�\s��y��Z*7��jX�A�B�kP`A��~�m����&MZ�bA���j]���U#���'�|~��Dd��qt��4��K ����T�O��rnH8�&�Z(J���k���T�5?�S������ �v����a�kL���?�eM2yG�j��Rv�5�P�X5R��� � �^h�����L��� 3����k�y���/�1���N�n:� M�4g��� 8M�U����a�Q.�E�M�R���z���T9�W�W
�:G��(�)i���1���q5�� Y�B!���e^z���3g��?�������l�q
B���1��&�c��7K��&�y��c�R�����.��������0I.L*Jx����E���Y��n�b �l2]� P��u�k�����{��8�j����~x����~P��
��f�D�e1"n����0&���8����������xn�&=K4Sl�=�X��l;���� ������&Z�VK���}�GR����?�
�<�GBLx�A ��_��C!�BW���2���O=�oq����� �P #L~��7��E{�Q�������=��� z�z��������-i�,7�� �� r^���q4AL�~�M6� ~pz�A�%�r�1��-
N��e��[���)���j"��W�"���U��� /p ��^���AK ��x�A}CH%���8W�j423�Q�� �I��2��]
.�k��9�����^~��u����d�V�H-�d�� ��k ��!�B��XP���>.���v�fB}�0���Nq���(]���^k-��.To �/�uZ�jM�/�<8��58 �8(��_��"�'�%
�^L&��������=g:�m��&�&�)�0~��H��x���) 1��p ������"h��`"5�^k����b[�� G�~��(A� �P�%C�*�L��, 8n�c ��J���m�H����8 ~Kd_Q�G }������o���Uem�����G=��$�.Ff��J?�%�: ,'%�+#�B��C`A�����}���M�A�=��� ����/)�����
�z���*wk{�Je�9w�I���3� �6E�������1���f8���u�qz��G��c�k���aL/�oN;4��c�1x�C���=(C�������8��,&m�G]�!D�$�[l�o���& Y
s[�"��e^?���TZ�ko�S��"�^Fw{C(>.������+�\^e6@K �%�Sb���JvZ ��o!h�N��!�B��#��B}��4�\��< �?,������gFN�z?3��V<&���M���j�����s��' ���d�o{�"����%
�(q�*}i�}a��t�h-�9�V������*����_.^Y����R�*-2��Fz�&� >$IJJ��$ifA<7��a�ony�r����r���_��<�-V�z���c���Uf�������;��AF�[�D�� ��(v�g��j��}�\.�ps�N� h r�NC4{��a_r�?o�B!��e� =z������3_>���UU���k����!QJ��$����#B ���i�����l-�)a ��'�m"����aK������L�2����
[�]���udV� gy{��8�^LX�v!� ^ ���Y��}4
����N{ ��PzjCriia x��6�"�I��9��V2���N8"DQ�����������k�'��#c��F�������� �������_�}���
������@qbp����g��� !�B]� �����}���?��c6(s�=����{=�1�P��K~�i�\�r!<9�G����Oa0UKNc��tf�P����)��X9����o3&�+%"2)\Y0cD���c�(='��������&���]EH:�P.���mn��iM9hf��������
��Ns��������0�:� L.Q�5 ����:�0����N��^�������N���}�vX��`��S�N���S��x�� ��p;W���O!�B]��)��+��]�6<<�]�8q�?�����B�o ��GW���}2�&"q���X�D�L��e�����)s�S����V�,3������`4R6��d���)~�|���d�p���K���;�S��V;M��:<$�=�"�R���, ��' q���$��������onyGAX���
����D�����m���b��{�Z�f �� !����!� =�U���I�>,B!����)c4����0a�zH�G�`�rJ�1}��}��;���+<��xL/�����-?�� G���4� )Y]�5���Zn_X���-����~�E% �Z�����@Iv� PI@<b)Rr�fV���G��Hyx��� ������h^ �����������YV��X�.�r�L���oeKw��2�5��[��1#� ��6Bgn�w��usA�yK
�]�vq#2 `5|���?�$ O��:��=.I .!EI���[gf�\���"�B��I`�2����D@BB�w������^��-��� �����FM?��x?gls/��>�Lul��7�~8�y��et���������%}����T>f�����&� (�[���q�<���HG�� �� w(b
�Z��7�Tl�`& �Ud���'e�k�U
1���i����?� ��r�H��X!>��BBB���.�\
4����5&u���7������c��B!��
�
=��������eB�7q"���T�S0�q���
�4�[=2�pd>[j��j������v~+]��}���L?8=�`����f�������{��7�D����y]����m7v�G )&T �Q���s���� q���K�#M
��Y`j�����K����!i�2�yk��N{]�E*��3���#DQ�K?k��YCN��(����B!���-����_~��� ?�< �\�����X�b���=�5�������d[����h&r����R����O h����������������m��p� �H���.��R�i���3���M������,eE�k�2���?
������:����]����R����M b%�&1�F��2�� ��J�" �q �{L�=�@������e�H�&������_n;Uy������� ��j4EWX��1�U"Q�_�*q���QI[C#=��5��]�P�5�7!�B�D���&� �={���^:u���n�����^.�v�W
A�"��M{y�`c��m�u�J����������������!�m8t���{Xj?�����c��z��"!����m{K��F������21i}}����J�����%e�T��� �O�W������G'���K���^���MqI�<Nc����y�iu�N/�U�"3�$ ��e�d�I�5 .-l9����sP��� �U����O�,<7����R�����!4+�I%j?��]�[����8�2[�T��B�)���+�B�(w���w���� O�0���}L�.�~���$��>F!�B�.8y�=�-����W7�������������+ �c4��P o������]u{�� l�u�a�J ] Bq����k���{��51-c���8=� �9�8�H �5;PI��\����j_�W�������3n @3P
4�����g=���Gg��d���z��f�����J6�B�@�%��J�E]����m���c��w��~.E�DsV�'��D��|���{##��@!�B�,8y���G������������P��+�c���MF*D��E�B�����7Y���F�+�N-Ml���c����#����n�8��������)fl{���r�f.w�$]��v��L9��.�lN��.a�Xs��g�8���T&������a�]�V�Fd � ��Z�"��vEE�d�f��8�KonMP���0U~�s��}�A!�B]pP�������1���-�� BA� ����1P���� �� �[4'��&��-��"���&�i�9>c�����|�����{Z>ypV�_�&���� ���D�C��@wd�0Db)/��8�&�%��=?�Z+�>� ���� ��/�d�v�]'���� 9)9) ��K/m��#2!�B�r�9x�����'M��V��E�����Bu�.�;z_�i/B #��=x���`��$ �����xU<��6�Ejv��G��2 ��Z�H,�����L����p8hB���=���#s��
��}
]�� 7��x<W$
��f��4��@��]O�m��n������t����7 8p�_?�h*�B!��XP���_���� ���g����
�P��B������&�����=x�������n;���a�����b���j&�Vi?�[
������ �j��� w��# �O��M�v��'�p�2��-���Y����6���a&�����xF
l�_� HP'r�X��[�)��,�z�&������T��)����$=���,�����M��m��>rB?�)���6:|��a����Q�����T ��+W~��g �����SE!�B�����L�4i���=���(8{!��Ao��;���K]h��h@���u����
����X��?4?K�83�F�ab �����
��q�(I��v� �i'A���I��r�,��� }-��y���|*���N"Y���Q��b��QJ>R{:���77�
������A�EN��P����u��J�/r=��i��YL�M���p��������"~���"��{��w�|��u�� 8p ..��M"�B�+.8ylOS�N-..V*��E��r-'���+B\�3}��3�{�v]{����������C-n�������8d`��� �Z~�es���HF<a �]���B0�XMY�|�]9�O����G���
�"7�h���K���FMo_�3`����)W���h����gM�8.V2d!��#f�������>>��d���af�_��[
8 9{F!�B�/8y�|���^{�����[��)���o����7��6�%=�����I��v]{�.w��F +�������)Ns{J����\������w�#�i�|��]�����nPFX�� ��M�!���Ob�5V�;������
4k�v]��!IR��5���do �[;>��� �2�y��� �� B!��.~eF��� ����w�}�F�Q���� �B�O���5����^9]���]������Z��~�
�YbU{.�����������2�d �sD/�^ $=]�o���) 8MU�u6�P�f�~4�� h h�-e�H�ne7���OM+^���eg�_7��m�C��h�u��r9�� �\�7*:�����/�25�6��A!�B�����{��g������1c���[�uW}Ipf@!����y��5P����(?^o��*�^���@gkw� % �B�}�x��| �`����9)��)3z�"%���I69�j�zl�!���Q,�<�
3
|u_�{��$i���Kh���2�� %�s�������-�*-2�}�������;�X|w�M� ?R��s~���� ���c��������)�C�-�C�cFJe�,��g��3 P��zsk�2t������*�;�l�iin|��}>!�B(��/��?�^����7n��� ���� ��#B��"1ID)EsGt����5P������[gm0���������#��F��BS�rh� 4>w�u+ ���"�M�| AJ��$��6" �9a����h��r8(������dQI�e���� Q�7���.��� J��� �H��6��.{G[1�]�f~�����4����3}=���� ����2��w�Q�^O�L�j��S�I�)-2 �4IH�o2��s��"j�j�����^����,���M5M6s��E�G^����?����J�`�:(��.n���Q]��%�0"�B!���_�K���������!�u9�k�V2N~����$���>��f� ��Eo>k��SP+y�(,)m����g��p9y���&�?�}����/'�5����`�9��9�����09o��6T�H����+��G����3k��K�Q���Ko�~i�����@)��X���������U :����E������W��8G����, p���K��� 3������ ����q��������T;R&C-|{:3��=���
@��v��d�f @��� n�T� &"��] IDAT�����}}��d@!�B����Wh�����j��\�P��m%�%����Bw��^�<�tu��(����Sl'��U��-� ]�
��
'w]%/�w����f���x1V9 eS�&2 ]*�Y @ �g��x�?!Q�G.c��my�&js�L�B&V� ��u��c��%�2� 1��n���J
D����
0.��yM����7LP&;5�&`��]zs��E��~�������A!�B=���%��={x+2�������� ��E��B ��HABM7���e��l��� �������c����� w$/�WxLo)/�x��[~_;��/n9�ni�\�e�a������'L��D��3Fdeh�'��a f���"�D ��>� l)/4 _D�l!B���*[�Y �$%%���QJ�P$�L���!��!I�����,
��n��J����2�/���0����N=���H�eL��~�p�����3���sR2*f-*�+��~z�������B!�P�����I�v���]�iz��Y��H����P�2{W���pIB�����_�[� ���cz��m�G;Y��xY'��k�l�m��_L�� �L�`����n2S���(���=gx-~�S�}q �|*���!�,����3�Mvu���k2Db��`Ky���<���Z� �-��'SZd�M��\���g�?z� ��o� Ta�OK������m������������K_������O-Z�o������|�<N:G!�B�#8y���%� l6��CB�O�?�zl��&��3&v������T\R���=����%�i�"$RdA��&����>6" &�s}��.��'"���>����u������0Mv����)V }Ax��t^P�7A)'�������i��KY����Kn ������� "1� �����Kf���z����~Z�u���{S�W��-�] Q?y�c�x����I��sM�B!��]nP��(�^��q!����V���i�[�p]N�� ��� /����������T�'����d�U
[���b�� �����a���G�[��k�Mq�E���;���%7��D@� ��5>6��<��q�?f�$ �%�qJ����/�
!�%D[���4���-u4�?����b��K���S*��x�C���� �Tcnno!L;k���D1���
��B!�B��2�� ��l---G� a)���s���{fo�>����f����Z� j��Pu���\>a E'Xax ���g���(�]�cO��H �������=-�8�y�H����^-,n����L e�~���MM�����c���^Pc�7= �qz�A ��x��<NP�3(���g-q�$�x6������ >�UG��@5��%$ !s�� ���1�uc��g
:���U^ a�a��e:�X!�B�k��A�o���b�\�x��S� ��#GN�>�g��gD{��%K�.]zE����
�4�]�N�4�M�k�K��Dp&�t��� ��a���7&�k<��aUxw�-Y�d�� -M��NVF[O$
� �eoq�q������W$���8 5��]��� �9�V/ ��*?z��I����]<NP�������(Iay 3,)A:Z����*�:
��e��R t���L�t���-�iW��[��w�B!�� K�.�������%�>7S�L��cG���/
�^A])O l�s���{g���/��o�+*������9\1��W���7��� ���+! �Tl����"��j ;�������SO4O��^a�Ud��6�R��� ��a4�� �!
|u o�D�������@��������R�9���T��s���B�U?��Xk��+�~�lrt�?��f�y��N��h������e�����v��0)�����+x��g37 !�B�����=��� ~�a�!tuh��;z4
V���Q��dgh f�H
WJDdR��`��&���#�'R�:��"a�����NJB�?X��E��q "2����m�v��mi��D,����R��;@�TnRo�� ���07���U ?w�*��e1��{D��+��Jvs#2 ��6��,� ��H�V�Dmo��F �k B���B�y��+x��t.��B!��M�e�����0�������
��JS�E U%]k��rnH�X<�������H���z����pQL9RQI{�a�&+C������tQ��j o�j<6sO�D,n0E=2��t���� �V�wa��?w��#����,�I��"$I�Y���o�_��'H; �r�cUf <�t��S
�O�����M����\ ?�L.rEY�� �E �DB���D1N�R KbR � ��Dq��[�F��5�"v�=jf��F!�B���K� ~���CCCI�lnn���3fLO� !��JS��4��4����{��n��$K���m�����r� �����xk��N����X���h��6����g#>y�m��j�����=�ra�s����@m���u.��]��Qc��,O���� {B{�qmI �W�W
�������J\�V� ��l���p I� �=}����� �R��"�bN�'#��B!�B��XP�����W^)**
����9s�,]�4##�g���c�Y|�d
�Q�=�c7_
�����}v�A��wHD����B;M�G|aXn��90�CO ���J�:z�*R���Y�y��et��������Xq�3������$���������W���-�S�0v%L�����R0rS��iU��
�T�]��
�8��? ��7��I�wGV��� � �/j�f%8�d+3i��3bB!��S`}n�N��v����� ���������m����
��+��rHo]��>3ox���LW7��|�=�w�da��GoL���zv� b����1��MK� ���XEf�M`�%l`����M
!! nc`�j>����
0�dq{.��2��l��t��q����D�;��5��h6Yfey���Of�\����w�[�)vH��E�V1��w���F!�BA%8y,S�f�q#2 m�{N}G!o���Eo>k����� ���+Gb������ �R^�
��Lx�����E���snH= �;�Z�������v��t)���'Z-?�rO�� 4
�4��_�R��w�=��o��-e+�^L���������t�������� S@��d ���R��l��z<�l�o!�B�N�S!��H&���� ZL97����p�`c�{�L�Bd)/6 �e�(;���l�?��F��� �sC��@ O��\|��q�� ��q)&��
�0�M������Hfav�����QI��a��$�T v��8FF��W����JD�M8w��[�+��'��� r���B!t6}i��A{�������?--�[�������k�98�r� � ��DU<N�2��c"2 �a2�X�s�������;y���>Z����G��/��d�sC���J>y�����woV��LD��Q� ~%+�S"k ����cv���u���K�B!�X������M�V]]=u�T ������_-7�A�*� �?��d�_����2�
�kLU���f�?W+<������[� ��-�� ��M�' �O� ���%�y�����I�����'��K��:��t���� �e�H��E�GA��{�W���>���Z��d�<N���|��� �B!�� �������s���s����c����C&����B=�w�zn�u�B$����+�>����j).������������$A��~5yaK�x����s����� ���s����3�����p ��i�G� :e)����p8hB��]X^���.��5A����L���&M����@�C &v@�B6 �D�����i� N[�>�] Q�P���B!���`l>��gg�����F�C��z+/g���1��S���"���l!QiF�U�d�>k������;
��<���@���j�3|�� :�+QN���^� � �!Hb�F�p�����2; L|� U�[��8d���7��q�������H������ ���f��������r�,���o�z\���eh o!��k�t��!�B�����c�_��i=]����Q=�� ��@����Ke>�Dd ���?�Z'X��5���Tg���kRl r���21��aD �U�5M��:�[N�.�d7� �����7lP�hk�3�\k����[�r ��l�f��H������B!�Po�� B�� -����(a���D���i���0�9�Lm��a'm����)hm ��Z�5)#��"{��7*����di�U�\��%Vi����:s �:�� ������� H�{na�cr��" ��H�B!�zG`�+BW�C���}t4�QxL��|�t�����
��/�R��d w�L�l� ���m���� LKKY�� ����EkS @J��$����X?�I��9�>�n���~�r���`� �J=rY���V��n������&�=$C��zy�� �� Z��\�H_- ��cB!�PP�L��-�z��,��jF#1�6�� ���W(;CK�w�Yn[�8i[��������-?>.��2�m{J�"�}.����������G���/e��%� �ds�w�����2&qO�[x�Ml��t>���� �]�+E�w���*��1?���; ��!�B� � t
�XOt[BO��\M���7n���� se���\����\<��o1#Ky������{�6������K�M����_��\���>5 � ��}�`�`���
�
!��]n9D:�����s� ��4%���q7I�G�L.���� �p�
�H t�!�B� ,_B����D=Z��<�M�������SL�/�����H���b������;��}K����x��|�,�d���l�C���\r��r�#N��_�\�[��l6�D9>$�04~OHL���w��R��:�r�L������'3& ]�o0�B!�����2]Cz���o�F����G�&������%�����9y.�`�i�n� ����R~;jb's��$���mo��,>�kthE^ZM��!%�V��s�U��.uR�cY)��.x����'���;_�6�es ?u~���StM��!�B���2]Cz�����FbRH����h��f�G�&���� ��#l��]<�Q"4� ��������p ��i� ���~ �"=�CZ����X�7�H�T���r��l@� �_a_5LzN�������� rA%� �� RR� 1R1� =�������s��U� z���y���'�Z�y?�.<E��hA B!��+�| �k��TO���'����t��u~��������d<��{@�#�.Gv��`���p�DD&�+ f����_��s���)��a��lH.�PzjCr�wO� ��n��S��P��'X����1� ��Dy�2W�.�L(�@� �0�JD��\nr���0�v��jX��jg��?~��������aO�O�"o^@ ���kz� !�B��)��5�'������B^���� ��0Sr �9�V/ Q��8�#�� �:\�y����)9�U�T, w�O�i2�4~���9��/��|����#;��3�<�S 6�TG)��D};[wdQ�����UB���Dq*B�5W�����]���[�]
��KKK�� ��k�������aO H Mp�����FB!�B(�aP�+��#�[O�q������YY7���0��<���?� �sCI���tO�I�W��Snc������r���fJ@1�=l�z �������p#2����E�(��n��w��������D�x+g�� 7n�+��2{�l���Mp������FB!�B(�a�BW�!�u�����I�LD�} ���^ ��sj ��� !E%n X���>����Ur���+�r����5lI�� m��l)/��2+�s�q(�����2�@�R�Wh����hUia w���!B}_�$�+&�����]a"2��/�������.���S=Z�B!��
� t�\�^ ��s���En�����Y��c�\rr��+��B^��Wn3�����9���� L�bKY��L�� �jV���H �r��3��-G��l4!&�$I�����������mu9�^�K ��d �i��S����gW��??a��E�=��#]�S�������$))!����kr������w !�B�`��K]1WAD���9�����r���{�7J�D��2�������%']�^a�G��0��\6���R'����$�����9 �C;L�K�#�
G�3C��%
��x����^���*r��a�w���AP��4�M��pK���F��I��8,��-� m �Ykn^ ���:Si�n�����s��?����p�����@/����~��e�I��FB!�B�O�L���HA���E4��qN�����'�f�P��������rJN�U�S`��n&�es��[N���E����T����
�C�����0�f�"5[3�@�N�)����r���I� �=�=^�b 11xN� ����H�{c"����4��6��j������t��6��] ���E�(� �R .\����������'��}� ����B!�P_��2]1��Tk/u������qNW��S����� ������l J�bb��5 � A���,v�Z&�`��� ��/�-�����sb�2�� ����Z��
�y�:�A��AQ$=5�0�)�SDx�m_���"%G���Z~Jj� l�zLSK�+�Mt���Q�'�G
D2�e�h'
��>��%n�*=[��4����v�EU�&���E�� @�<����I ~���.b��C����\c�,?��6\%�.�����233�}�� t�+���9!����;���> �������}m�(B1X��HQ+�J���[�D����[m}� ���T���"�"*W�(J�@B � ��${����fvvfw�9� ����og~3;���O�!�� ��r$F� Q�] ��/����#�j^�������u�!����%��# ���3URV�H������8h�q[���B��0d��9 u���z�����I�c������� I��v:Z�)�C���5AFN|O�1����!���B�L���xa����Hz��� Ne���7Y�,6���&�7�^�q���+
� -���?Z}p& $1����)���9�g�s�d���=2l�kcg@qe���;��v�6�p������'��w�{��G���� �����X��B!t��_��=]Z��!�kB���R%���u-�����c�b��Vh&����b���9�
i?luq~�������9��� HE�V� ��q�k��*�F�o��3N��l��x�1��Cq��[�`�( ��h��d����WW�� c�r�{���uT�-��l� ���� �[555�&MZ�x��e���m����#�B� j`~y��%�� rQR�$�E7kS= ����?����� ��Zm�[������9���<��]�[��zK]�
z���� R�r���������UH���$:�O�%i �Hrcz��xQ�� [� � <Q*�Uw��e:��|{~�.e��:��E ���rD�rn�� �����o���� ��"2�3G!�BW��(�� �`B���f�:��E# �+�5ym�k7v��� �� Zg �~[����
Hz������vK-��d/��?v �8�^�I��$\���*qM�6�/�����) Z�����Y��H.�x�6���� |�'� ����;mBZ�" �)��Xg6/�|��#(�_��(?Zt8}�w9O��o�s����Vd��k�S��8���9���b���i�4~���a��) ,X�bE�
!�B!����>���20�]�~�Us����D ���� z���E�����O�i& ��B �Q)H ����j� 1��?��7_6�;�I F��h����"�i�f��r���6k��Q��F�z>�t�B�n��X�,�+��jaT�' |�������xq��7����R�p � tV&�R�v��S�/U��9>V�T����&���Dh�Q��������<yrnn��O? �'�����}G!�B�����} ���e`�^�-�y�����$�7#��^T�|���wU��t�0S�"�H��k����l�
�=���� aJ�t�_��8ie��$W)~W�7��l�� IDAT�+��qj��^S{d�� Lh�e�[*�����]B��yoHyD����
TiF���D��p�#���n�����!)��|y��)w�}���+{t�@f�,� \����0�B!�.���� {� ��
U��)�����| ����H���(%RZ0%o
#Q � ��fvTE�| ��FP^b��_�'�tT�����s�3�����G����n�,��?*��1�� \����3� l����=�o+{&��c!" �� �J��������� s~�MS�����)"��<�m�<��2!�BW�b� By��S����)��!�>�YRV���XU����U�����6���2
���|�i�J�v�P�&��0��<�#���J���^!�"8��A(#������
}A�
}�q\?�Z���9����>]�i���
9fmB����J#8���B��g������E�mu��6�Uj��r��j��������Z�� Pd��s�c��9�� ��,�yL�c��(w�B!�." � �� &�C#
qZJAqZ�����}���j�I� X����N���I)�32-JKSdZ��h�����0����&U
_>H�8��~TA��K����<�"�'�T��%n���{]qs��G��� 4�)I�0�)'%�h�O8a��g�E�{&[�
�2�����;?�����5r���L}�x�9������>��ae(�) ��Z�fM� �R!��I�B!��� ��BH��N` �l%o�)��$�#�
��� V��� ������P�k2s�/W��k�sE$)� &jhz��������w
]+$�N�)61#/��7J���3������V�d��uN����Y 7�}_���������k����;<;I��6�)����g���
$��$ S!�B�J�A���,��1�����o��^�1 Ap�9)��8� ��5X� 8��~�+�?��&����P8����s����<� �����U����%mq�}vO8a���/����� �o~��|�[�f��� �&��5��, �0
���^��@&��I�B!���` 6 �� ��!������i�:�b������1k)� ��6,)�����s�?lo�usG���;�f�t���2���2�i�.3�s�����'�1:Z���y,�u��I��i�9=wE��.B!��� �/��S!t9 �eLh�@�t�J��8)vsK��-N��==�+ �W7nI���l��n�,��� ��`o�(�Z����>�GdH��2��Pt����>9l"��M�����9����B!���| !t���eLh}/��� !�P �q3�p~u�#���p*���%���|d��Ye9� ��\�S��� �XcK���,��c '�7eg�V���<5i������m�&mD���3���-!�B!4�aP!�'{k ����:�(5y�P]�����->l���
��Z�+�>T\B��I�gW>�dCy��I\&%R���z:N���=aF[���%(�E�~��F�:��*6U))M���:��g�B��^���� :�h��"�.!�B]^bI��e`��!���ov�����2PS��� }�c}�/�}�T�����PMC@�V��_],����Y<�I|��-� �$>R�GKQ7���M�p�,��������dv�q�z�Sl��,O{������9��+��m:GI��xJ�tO�*�n�����k(&���R�����������p��-���,
p�G9]dd��Jm�� �B!���� 3eB���)�C���`�NY/tP�B_t������ ��v��� F�"XDF<�I>SI\B"v �9) G��S��;�9�����> ��f`���\=?�tvO��& (/������*Q�pN�.������M��
irX�
� ���� ���C�2-� � ��~S#��W�R���w0(�B!�� �A�Jq!J~Z�������~w�/*}l���-~%
z�v������b7�.0�s
�V��{Y��
�����'t `�R�=�
�h2�6� Q��;���ce�����
>�]�B��zOt"������}4>���� �Q���_ ��k �1A�� ������=,#�eI��
��!�>��nC]!�B���A��B� 4��
�:��]����~�h
��=/�E���VEw�����-~���J�:�b�o 0v3+ro�.0�5��Pg��J�o��Q��&���~�����t�7qo ���* �v�-��P&��I���� �E�"[��~�sgbK��O�k-�6�Ok���_r*.FP�� ������u�Q���\��B!�B�= ���!`�O���7D��|=Y�q�]����E�< �����v��.>l���f�J�b�B<�3R�/�#o�[RV���XU������T��Me�J������������)��[}G��us����M �d��)�iI+�i�qEBe ��q]��w���� ����WrT��J*����o��:�m��t=��u/�S���q �Sok{�m����%�y�} ��~�%����ge�g+Lh�,��J�� +{h��%�B!�x�A��p�J~&���4�i)I�i��F�_��2�������G>2�-�vL���+��HB�gpB�uC��j��"����M��V9�~��qK���di/,���uR��*��I�T�3� A���&��[ �O!��k2����I��7��������no �M/�o��r�����S����
�Q�E�:����"����E��)��?U$Z�'�2�?�c���n�8gW>v<u��[]N�M�B!�BW,_B��p�J~&&������n��z��� h]�9� �`�]��`Rs��4:#���������E �
���n�M�=h$�������]`�s�u�?���MC�� ^o>^��O������� f�{ GvP� �^�~$q���1RT�<�����a^R�J 8� �`me�������f� ������Ta��SI������`�S�����}!?��B�J[N�M��A!�B�+e�"��[�?F�'�3
G���� ��^���#M+�R#�<�d ����x�?}i~��n ����N�� 8�Q���A3����u���f~Zv���iJ��� Ma|��?;c�������
i7 ��Z4��"�ZG� 0H�
P�@ ���q�d���P0]a��o-��A�Tm��z���h�\C�Z�T
kF�])�2c
�,6 12�� �6���B!���aP�+�8��OH� �������S#���M���<8�S+�$i/*�9=���� 4<�xl�&3�|ky� �qr ���V>"�sRl�� Iv�u?�5��
?�w�A �S��Z�w � �d��f�b� �8� ��6����S ���Cy+�Tn��^]i��3��l��\#p�_�����!�B���A��}�3����F^t�u���3KJ��x��L��^ @�����[��\��MaB���3���04Q� �d�#,����� �!M� Z� ��X�� ��5������O�R
M����&m��$��Q �B�E��"q 6B!���2�����n������q�cz &��iy����>��Q$�Ma�$����fgd�������Xc�NH�jT7� �%8���!�8����* ������6JH�����9�0 s����|���|
��t�� _N�a�B!��b t�)
A�3DW�^��
a��!iv�x�O1������F[.����J��e8 R��>�o����kg>�r���V�T��sG��I)/��Y��3y�);7T�#k�s �����k��:�U�}���/����\ ���b�:N������{��������4=�X\W.��F=Fk���3B!�B�60���{�� ��+B�~�c� e�C�Z���o��R��s�����d��8���D�>���5���FgK�M�U7+��V����?: ��*�T�=�� Ne��\�����_�� �� � ���L������M�N�8 ����i �F@�!���'.\������X�� IR&C��w�B!�P�_��| !�}��+���n��%�L��3�w�2���s
��}���c��N�kl<7��[���=� �Y <
�=p�Y�J��& H�F>�}D�� ��b �wh���;������X������62�
&�j[�}�������G ��gI������ !�B�+ eB=������K�LIYM�Vo���V[��c���F�S����|>����� �����,�8��5i/����$ ���y��a��k�DML=����y � ���M `O~-_��Q���_ ��5m�.>��kO]A����]��u�?@'��������������G ��j��=
A��zp!�B]���}�KI�� oO�������p����xX���l8��� j����]'}S� �N��]'�<�^Y��%�~��qK����_,>lN/��\�#����W}f|C3��W��N����[��9`R�B�*>f{����W��� ���6���P6F�r�|�����"}�w����� ������
�V5�{{����
�� � �T%+g >��2�
�(C �(=�yLf
4��|{v���� ���@hD'k���l�B!���2����f���}/]X������+��"������G�z�l t��>���Y��n>��^����&�1>P�hu5��n��v�/�}d���� [u����b<�j����=����ce���V���E?���P��8�5���w���'��k$)i���-/k�k8�I~�K��o-���[j����
V�K�q��*l ��S&c'������6�t��Ik�j#���D�P��Dm�RZ3%��!�B��5k�_�/����K�` �
B��H��Lj� ���~�������H��&�xE �<�������,��l�����|�B�W���B���8->L�K�d�� �f�< ��4k<|yQ�A)�i����I�o[R��@�BK�(��0X)�`���@@�E ��������6:�4I>�c��[u��8�� ���(M��^o<����_w��qI�*�^/>��N�����#�B!t!�Y�FH��q � ���ovl���� L4�g_^�
Uo�������m7S0}��-� ��J�v��p�+Ofa�
5���CW�
A����
��:��GXJ�-)�oi����c���d����-��DvR|y��K=]_�Tx��T �7����������f�X p4z�=b�$�Y�7{��� ��� Q� �Y� bs]�k�TC����&�����+F.^�A!�BI`�B(�>�B���p���������)2-J[4g�����J:����������
}�q\�&���%XGMf�q\�B�F��B�����^��P*��/��mo�2ih_�I�&o� �����pwcQP,cQ��Yl���_��G�� �7�����}��� ���k���� g i��E��������E��4%I�4��{#K�jxCy|cz9�J#P$sl���iG�M<~gb �8h�������%��� G"�B!�����K�� u�P�X�����[�IxoF��h�t��~����D���^��/�Wlj�+ n��}�����L:�b���p�z����y��������ey�+�+MTzati��� ��k'�� ����~B �Leg��q����T���� �����- �n�����;�]�B�r�����9���K,{��fT/
���2�?k�o�m7��pBa���Xsj��� %������=�lC�B!�~b��;�/!��
]Lt�LL\ uA�qk2s���=�h/��R9~���'���L������9#��������F^*��@R|�5�b6�QR^t��
��}
�� eBD �n��#���2�,��q;X�gk����o����t��� �w� �` ����$ ��rP�P�J�\
{�� ��l����������A!�B�[X��
��YK{k K�4��l\��yomfW�������f]��h�3�/&o��������vs�ls�}
���|��` m��Rmd�ws'./�k��$(t����F0����EK ��mm��&1���$ga��}K��z�=�����#�~1��~7�?b������ �io���nq��_���{&;;/R~E�B!��eBAM0� i��R
���R�4�7�R?�i���.�4na6� 8 (q����s �r��u��n��
����l��n�Nb\Z�,��D�� ����"�=�-mH����x�-Q4 ����%E iE��M��z�k���|� V�� ��M?w`\��F�"�N�Q�n���/���C��B!����%�P(��� H� � 5�i����-�
�0���9%,�i��|�g�,�����������)�dL�Z/� ������������-q�>u�y ���~E�j6� 3�����wx�>����&;/2;/�|�E�4��Q������j�T�:'�Re��,i� �gRG7 @�SO1w�� E���!��_���� B!�B=�A�P/]�&/��������V-�.>l& $1�"@Fy� ���d��,�C���_l��p4���\���;H�DPB��^Y�Y&���vd���JW����������r� ��/��#�[ ���l���f[�IQ8zF^np\��' L���6���dCG�G��RQ P���N�k����F!�B�aP!�K�
p 6��y���0E� g �mRhS= ���(t���������K�b"� ���� ��v ]��N��5�(Ay m�yR���u���Yc
�K 8����y����� �
���?�aA�':Q��<��;�v�Sf ���y��u�U��aygK]���o���jo�������q'#`5Cge�8{`��7 L:���
B<I��?0x������;666��!�B�`�� B��.P��� 6��V.��^�������K�j�� �j����������������6����j�I�<l����/��^~�7��D����I���0��m�����u P*���_O�a��_�Q�b�\��W�� �s[��
]V�H�;��)����j �l �crL�l�q�;�;3H��In�����3�5;���b
����z���W�� �NS����W���;�<�U���������kFdB!����8���20G�#���[�7y����J�jm�^��T�usG�v5�����,�r�E�b��VL�� i2� X��w8=��x%���fA�vE��7{
���<��_�6�VDd��>
O|q��5���� ��}DZ��b���6~�����c�
on�����~S�o(/�r&�W�����>����^@i��+�=��kM�:��gl������9�V#^�8�v(�$�� H��who����={���� �B!���20��c�B��.D��0I�X�N��]'�Aq��Z�tQu� 4yB ���ye�n��)RC7t�2��M ���k�uX��X���y�!��� B�8������w�&� �������6~MG�������O���T����v�[�"v��3F��X����T����#�z[r� ��=���
�9r�����[_��ZG �i��:n��_��A!�B��0(��e��L`��� ����&k��M:~�Z���^���*W����H�M������wg�q @U�ME�*��xZ��D|1���%� ��8����c���#�$� �,�1 {�V;�(��|�;����Si�iov�� >�s_���?�.5����]+%J ��xk���=j�M��^o<���s���T4
~eM��#2 `v� �AmT�J����9���bi����o9����~���+w`D!�B���� B���f���;�a��6f���� 4�Z|��7�p�/�K�T�n��W����� K���FHW��k��_��X^|����q���f�*5���d���"�����w��]�s��8`����$~4R�7��n�)�*�q\�&c���@���������I��/�z����� �����&MF�>�>���3���1f�q���[^V�7��AH�J X���$ ������R���+�����={�����"�B!�z
�"t�y������ ��)k����Z?�Ey���D� �P�@�M^�6~�b���T-4E�E���� IDATi�����U�thU��q��,Nq���db��kf�{�C������X�%}�9Z��D����o�0����"�'��
�i�M|�Tv�qrQ�!MI��!M������g�u��=�����I)�32-J+�m�8��Y��n~T��b�������]�#��M�5S#� '�� �$��u��|�x����4}T�
s�g\+~��W_}��W?��s��$?!�B!� ����e`�
BW�;�����M�{3��ku{�~,����� c��d)],n�b �^�1_r��#J�qsc�� y`[�$a4���/;G c��! J�t�/v�
�H��,O{�xEa���s����t�y���3�c ��Q�b ������*Gp �9��+R���� �n����{n��=iii}�g�B!�.����} ���e`�^���w_5�[�J��������Z�/���X�U� j�X4�pA�� P�����_��u��yi��C ��vc��I!.- `�@��U�kE�HL��d� A��
���i�S�+�V �9�X��ag�]}|9 a����b������)�����g|��A�1�b&��!�B
X��;�/!t��7D���������������Z��=�y;��i�[��LWr-��?ei�@��7��P
��V]����v���<���\������x �".�R$%�W#�*����z[�+n�n��%�G�l��le��xq,G��Z��0�%�������g�����*�F�S�o���g��!�B���� B�� &�C#
qZJAqZ�����}HE�ovl8��hc<,�w��g��e�k��2*2�E[���G�l������k]���:&��}�'��i��Z�O#HZ�O�������+����oT6nI�W ��Q�
��%��� �w$��*���������y6F�� ,�g���w4���N�;}������{7Im��\�M����!�B!t%��K]�&&��k"u��q���d5��z����W����$�N�� g�.��%���C����I�&jGi���$*>���8���p��w�c��gV�n��%�o��w���:�&3��B-��s��K�o}��IC#d9�$�z�@���i�`]tU[S+�����^|.�B!���@,��� ��4��K�m��l������onLQ it�K�_����rp�_s����(FB $�Rf��~H�Z"�It���w�&��;o^V����yF�R��;B��+)�Y�?�z������mo �#���W-�]�r�`���y
������G�fV�i���H*����x��f�B ���)�q@��_H�T�����y�M��!����B!�����xO����{E?�~ '4 �K�`�#�<z���`54�a8���?����l&Q����Un�������1|��41���>�p[��������]�d�[���Z1�#$�P^T�L��w��L���q�G,�Z���� { ��cg���'i��>s�;aY���T&��
��o����>�~\Gy�sV���������n?B!�B������K��~�cCW��r���B�
G���
�Ir[z�6X���V;�0 �1L<ymT�x�5�[u��w���0U �Y}p&�" v �<���H5m�J�l%�E��R{��k�N��M�{� G�7 ���5.����!"�r��������� @� �$�d��$��Z��-�� �r��'�|"�:I��s�/U���Z��B!�B��A�����+E?��&�� ���� �w>"�_<W��}e����(�#2R��~����z:�Q+ta�������o%����H�i�L��H5 ��Z��g�EJ���B*��5�JQBw��_ ��uB �L�����J?���Q�qY�������<C_5T��e����� �x���;��v�6�p������@B!�B!`P�>����7T����! �����NVy ��w'�D� i�����*��.����h�1 {�=Vo�C�#��M�\I�`�J�v������Z~��*���{ �w�2 ����1Aw�?��%�j~#���R��
��4��������z����j�d���� �����~W�������$]��7 ������SW Gge*�9�����[2#s3!/#'/#'���B!�
e���l�P� �PQ�GD �[� �����a�[Y0��W�g5}�4��m'��1>5\=kb��g����[������<l��%������z��v���I�8B�wf~���;��� ��F }��|���g
'�w�#9<C�6 �m��u) � nV*#;�;O�-|�� w�GV,8 -���D��f-9�X\Y�R�������g*� K�������J���a'��8� <�
��'P#�� p����L � �B!��0(�P������0�BE �����-9����/-S�Yg����^�` ���"�� ?�nc
� ���5_=s��3@�g�����6I���MjR�~����R�@g�YX T�z�2��3I����o�,fGK���S�x0��R������g}7j�M����aXEz3t�,��m�����hv)V�0� �Bpp�\ ��Y��s���2 �+�����{o��0�B!�B�G0(�P������0�BE� G�G��_ \�K���`��x�R�e�,� ������=������A9>�r��������Uw�.e ��������R ���P]5�pK��}@p ����dk��H��1J������K�>������������<u��K�/r���)��>&B!�B��0(�P_�����0�BE ��d��w\%��b���������Zm�[��������-W�)3\5;�Y�"i�����9���(�OD����wVK�|�+���}N�&#��(���&�k�sM� m�����w �����K�+2�� ���P4X��C��Cx}�A�r�D�� �s�?�a ���ksn �1B!�B�O0(���i ���v` q��]'� X����N���+��x:�<'�
�;wW� ��ezQ�% �-p�$�G2�3k<�U��������\c�>��%QN��������!HG����"��~�2��2!���)Z�3��|`�Q��{��v?}����!���F4I�t4ZB!�B��� B�����!������w` ���!Y�ns��gQr��������K>Z��85'>yT>�Ir'���yt�&3���rbc�e�#9�L�S���K��� ��m���F ���1M=�,�6���)����[�q���!?���fG�:��SI[�����"O�]���B!�B��A�.m�0������� �:�4��
=l� ����M6�J�� ���zwt��ey�{��������i��pIFk p���:���5�3Z�6 ���'��I��V
�V5x�lu�L �;}�5���G�K a��./�|���qr �Q���A3D��8�1��Y�
fnlKI����;<C\ � |GF~�����[�no#�WB!�B�'�A�.a}s����|������� ��D� � �' ����j ��m�Z|����������&3����vr p]}XP���p U�=��@|��� �����+PR�R�k� T ����&i���$ �@I��g ����i-�z��g��~�����sfaS� Z*�F9J�uX�YUv�| ;��>����Or��gc�B!�B?
� t ��C��cVN���n��w�W�a��9\#Dd �
�������������2���Z����W��+6�"2 @)m��$��=���� Z��_���F�Z �C�y |R����9r�&��
����^�' q����q������ A�����[�����&�;��XL^FN���B!��G�A���0����g��h�z����n�+������.� p��� �� � �kW�N�������xv�� ����\�� ��S��L�5nK��@�%���z��E�A6P 87'�� ���M��7�I�v�\�> ak� ��~|�r@�STl ��a��� �,S���` �An&�eB!���`P�KXO� ���W���3o�n}�u�����+Z ��Rb"�c_���*���t���m �A��'jI
�
�<6�p�0��O�
� H�|�%;��T�}��u�>�8�F���g�O�9�k��z#~�C���W6���%� +C���H�9�0����5����"wV�k���0(�B!��O���7���yCtjQ:D�3��W�=K���^V�`�I��HC��R�D��zh�a|�����Z�tu�A�S�"�:�b����v�[�X��9og��=�{k�S��K,l�3|D ^:�gcT��|a��#� �:n���o�����-����1 ����r�V7�/��" ��[!�F����^��+s�(�M� � T���k �S���"� �"��8s����-{v�����Q4I�7>���1��
�������m�^!�B!�G�)��%l�I� .D
g��<�Roe��i]�$��a9�gv�(Yfb�Z~�����;O�� �u��a��l��g����r{�+:��/�g9B|�������M:��a��
����X���8��R�>h��kz�8�����U}��#�K��]��e�,1����� ����Q������;i� @ii�m�����oO�: ��1C�<��o���I�Q���0F!�B] �A��0��<�a�1��{�h�8C�� �
�`]lJ�j��~�7��j��o=F\nN
����&�M ��XZ! �A������q��� ��I�tc���'n��� ��/z��|JeO���S�����"�H:������%�������������O�u�`�U'$��}�����[2G��eeew�q�[o�5s�L�������o������W���
!�B!�/�| �+���I��i�IM��`��Wx����XG���� ��g���uR�������uR�����`Z���S]/�<�rj8�A� ����� >����!�DK�J������b�� '���41��r�����Kv�qrQ�!MI��!M9�(9�$l�i�/���K��z�=�����#�~1���Z-��]9r��w���| G� �9s����o��V����9E7��K���Q�����4]|@��*�B!�P�L��8B�S��g���z�WB&���I��T�Vz��� 4��u��y#W�I%�+�����S�p�}������%^hXP��NT�,O*�g:��!��:Z ��� �jv�L���xI�%;/��1��1f���n��}y7V7���q���� ��5����v���_���EEE��v[�k�e�c��{���WW!�B!��e�����}��� KO�' B$��F�O7��o�D�C�'t������f���� %g ����`>p�&3 R�~��X�/�2;�+a ��1u�_L[=� g� 1N��/��A����}�g�qaG���J:���� Y1�V ���FR��3f������~{8�
��
!�B!���| �+WO�7�t|� F��?���?AI�^Y��%�~���$y���&���V��a���O�!�������I���0������.>l����(�=�U�����tT�l�� ��eh[����������@�������4K�d���$:�����!��H
�h����W^���~�u���B!�B�#� ����A� �%,X���������, � �������rp��6�!h�q�:>������
� �4e������n|����z��S)�� 1�x0�,�������?Z}p& ������ol6A����W �Q����^����n?��U��lMT�����d��Rf>81t����
&
�~e����|�qS\{��]�� O%}P �����?��G����p�$B!�BW����} ���e`�^h�=���-��^!^Q����~T/�����������7�U9���Y���{:N����[s��6=�A��?��'t �:n��ySG�W�M��B�%M�J�jm�^hl����QS<������������ ���~
a�R����Z^J�� Kk���+~���Q���+�;�V263���G@!�B�50���{�� ��+BJ�raB�{W-�m�TI���yn��[
Ip�������.��8HI��w�����*�>��� �����C*79����~��?*t��u�^���'�d�jO����./n�yXgj p������ +C.�q��Q�u�%F�p�����$��mzB!�B����� {� tI�[�X��9og��=�{k ��_D��.U�����6���}[�6U��K�X0}(���e�r���G�
i���:����wV�M,Cs0����XE���4�c\����� Z���{1j����'�V�*9 �f! ��������YNQWe��q �<�J�� ��v ]���B!��W�A������p����xX���l8�����~q��uR(�Y�����
�����u�Qk�������j� ��N� �����*���/ ���t�� >T��������t��R��n"D��T�`)a���U
+���� @ �� �������� �x��� PIz4���#�=Vi8�c���G@!�B�# � t�y��Uh� ����5���"=�/4���qE
}A�
}Z����8{�$E�`ZEJ#mvO���iQZ�"����_�^w�/�Y��GO";��n������7��7���� ���nh����Y��nOg��`��b��n�QS�9g ,�N��� �
"h�%Ep�4.��i\^�@�����{p�4(# 1���*4n�N���p�"1A�^ �������Gh���B!�B�h �T]ZfY��-�y��_�B�����`�?����{�$T��-f�� ���;-a�`R�zf��tW xoH���K|�1SIn�.u�U������� H%�8��Zh1S^b!�#t�~�'���]�ul�z������Ym� ����B# �$� ��VH�dH����f�/��sG���)c�*t�� ���������sns&m�����������B!��%g`~y�L�.=1J���B��B������T����3+kB�L ��|AKQBr
oLA�B��K���c�&����iR�)���(��\�MCt)Dd@�b���u�����>(o���q�qi� \?o�q�Sok{���z�� ��/W8� �;������/����O
(�U�Kw��e+��!�B!$'�����
���O �k�.��������u]��
��� �9) G���n�����[���B�`�,� ��[���A����;#9��� @�a��-?:< � �A������ ��;2;� W�������������� �7���e�T�$;�cIg���.�b���moN�'��%�rX�N�l���\?�sT,�"o�)X��4�V%+mdr,sF�b!��M�B!�B��2]z&�� ����:�(5y�P��du��0��u�����U�� �� ��wHN���|xP����<���V��H��{��z���C<r���u�I�@ ��M�v *�G���
�i��y��y�.W��������Sh ���?~�H�}�hm����o? �"#r B�b��`l }?����l���f[�IQ8zF^�/EPq ��#�r��������>g�����n�!�B!�s�A��41Y 0hl�[��]B ��"����V)� �(L���n0��A���� ��swu> ��s� $�f�kv�S4O� k}�����$�����L�UH�Z��O 0� a�����������>�l���5��� ��}��x�c�*� t ��-AFr�CH��1�����c-I*w����)��� `6��#8�]�fjN�f
�T �u��][�������pn�7.�6.������Ch���%{�v��C�� ��: ���a��k�
��!�B!�z�2!�0�]B��P�V�(L�^6|�gX����*�=^<�H,`��B����u���:������DE����fG}5�����;S��
�9Z�JC- te��cLA��jXg�f` ���\m�����j�gO��]�� ��K[�+|(���( a[B��E_%Dm���vS��������V
�KV���V���:�>I��l\�>�Q��V8zF^�7���������#8�4����y��F!(Ck�j9N���p?��"�dLj����89��2D5 IDATw�V!�B��aP!��l�0{��q�� �G|_Ymw�t*����?nP���,.�
��� U`)]�W0mo�qY�#6V
���e�� ���.}mr��3� Q�C� Y-a!���� <6n��)�J(����io�1�+������Md�#�������:��S|(���M Q�[�7�G���u��6F ��#����n����~=k%��sjt�@�)��v���s�:�
!�B!�#8} !���Mg��?���|uh����-��4��b#�}%���a��Icf� �a9>����-k
6MI��k W���$�R�}|D�gc�/������S�\� ��E�|VK8�j`�� �h��O�a=� ��g*������o"C0?{b��[���r���}�>QG+�7�J�;�DC�>"������NJv2���Y&u�
,�B!�� �A�.g!&UK���Eh���TRV���X]��e9��y������1y�GL<�:D��&#/n���{]u�x�1��8����5�:�]z��xm���b��m���P ���?�{x� rny���q���<q{�L%�� �%A���( (����� +��L\�
s��Q4I���(��e��zW��,y�dDKV<DT�WG!�B]X���e�G�{�Y%ASD���u������q�W9�����%�R9"#uP�/� D^�iR��>�d��DP|W]����"�[��^�������X���z�l��O���� ���3�,, �KgJ��� 1mZ�^7������j��
����4�>d���;���#�n�r%�}dY������-/+�� �el�����B!��X0S����Z�� kX0}��SV�h���!o%�Y�F��L�4��p������|w���_,C����SS�0Q�&5����M�&��� ?e�E���u_8*-JKSdZ��h����BEb� ��XY �z�k�Xk �T���x]r0�"������������0��iZ3U��T$����A���\slg��-�*�������nO ��X�%}�VG��a&Z35"�qR�H
R�����/B!�Bh��L�.C|� #���lg������o����#��U�=�m= s����3�\];��N, ��@��/� }z�8Q6��:�(9&o�)o�)�O _/��7x�M � yWS��� u����8= �7�������� 6m���������9�@���!��H���v�>�YnfNkk����S�G9�y�.` ��)���
�L���E!�B
�A�2$i�+��P�F_�9�Z�`���[�*�BW ��j:�\4{ ��s��%�H�&3� ~�N8�K,��h~�7'���p�xg6)t����C��� �ce�y8���,
8mz�����dT�q� av�������5�������t�MI��|�8� ��<r�����*!��5�z�yB!�B�/!t���@WP���p*���I)�32�*�TYM��W�����GW�s��[������U
��U���#�S#��������4aHS���:k�C�:���v�!�8"y�!�X��2��Gd!��]9�������Ll96�x����& �3�%Ym����&M�tbX����mp����`n��������}R��1_u��MB!�B�3e� ��^( i`��>#&���`�����h)�����5�.n������E���>�� m%d�����<o5�u&zla|�qK��������LiM��BMF�$�R]:>�o�����xHXs�� �O�-��FM M��/��e��bi��v�&~|6���� �4�� �^�Y�a���_~���OJN��h*�,��� ����N��Uu����p�B!�P�"8.�+(0��g��}f����1 ����# �N�(~k|��LB�a���'����g���������[��{��,�@!(B �E�y ��G� .D T�Q�2��8����3�8nP �80Av"BIH�������~T�Ku�H��{�#}������<O �=��~&���� ��
�i��;��_� }}�X���`���������_��{@��d�=��p�G����M�,�`�Z���A���N1���7]) � a�>%xd�b]���F)���4e���m_Y;_��,��Zbi��^*� �X �wW��+�V�;^cQ]�� 8�}Iv��S�9yn^���e�9���������_�yZ� �B!t!�_���.,�������o���$�c;��e�3 �t�9e;����B�[�����~�{ K���or�`�&#�b}�������C������#������}�,@��G��}����L�� S�]���;�a
���8�����"0]G�������/r����/`�������L����V��'���������{���C�1�i�������_,!�B�Akp~y��K]� �V�a `D 4y@��t�," |@�b����� �*@����vy�L�.��yJGW�ibo�}U� �u�h��R/ ��e���y���=������z�%w�G�[e��D�eEe����U�� ����G
/4� j��2eZ�*��v^F6%p���( Po��:�iW� q��c}i}�F����Nk��2Bh^_��B!�Bg �"t!��`_��=��u����
��l(&��R?�+�S������ +?�84� R9���� ��� �� ������G�@�����Z����W�������:
]2 d�G�]���s����}Ddl_Y�^]M��DW�����]�di��hi���'������� u���M�� |U5�Q3���)�x^��g%� ~������ jba�B�����1W���,�z�S� ,����N�A!�B���� B��F��2S�Up���*�-3�1�F\���+� '
��+�L*��[0E����)��{����[&w*tLN����.s
y���{�2i&��o�p:��U��[{�;��1 @m���� �^����<W\VU��7��qK{��I�����a��(]�M��rj��(v��
��k���;��O�� �|"�Cq���Ok� ��q����T�o')����k !�B! _B���Q���L�]�*-S�]����W:�^�3V����qQ� ���S_�#��� W(��^z�) �w,4v����:����$�i�w�� zj����'��l�_�Q-
��}i�i�VZ�|� ��Jw��M�B�
�N��+fw�M���I 4;�R46���Q��, �����{�-?E%��)���w� � �u��k������hkd���O�// vZ��d�G��J�T o�����^�!�B�0(��#�(L������AW,43x�m�e��`{
����� �y@i���f�_��aC4)��t�����
o��ss�����Lq������l���K�~��������" �:X��|P~z6o[���8�[,����Y�R�5���� ���E�����.�����W�O����sr���^�!�B���/!t���(��<�4���VoW>�%m�����3�4 w;��Vk��=����/\�Sy��:�b����� ����w�E�vE�l������r��5�����'S�� P���V}�9 �i��� ^kS�"��)� N�g}��K�>�n��x���V�eba<������*�u�����������QZ�eR��kn ;��"�B!tn`� B����~4yN0y����cMk}i}�� ,<���gC������N���6r����.�nM��9��"����L�{�!O�h�R���_�yW�0 /ra �[�S4��M���7�5�W�����n��Q����p�����D��'�$8��+�3�N;��O� �!�B�� �2]0z+
�X���l�A �M ��������,��xD�� �����������
�!X�������K �z��� �����Sb���� �
����f�* ����o$�C� �p��5�U v��_5\�]����lUE����� �B!� � t! Z&F�6Y��0����T"%��H�0���s��=��Co*��,k�y����z��~�B��2 � Ijc�/�T�$hCh�~�h�V�-Ed$�x�J)(#�T8���-�������Bpw�&`�:���7����Z\��V\�
z�>��>�������j}��� ��m
(�d���7�B!����5e��
�^��K� 6��tZyA����+
LZqKT�����)��w4�j�"OM5�/�2��u����B�z�SV=��>B@J@�JR���&4�jr�-u}S���FX����^��}hx��C�{^�����]mba�B��� 4�d>���~����V��O�B!����L�. ��+
�^�R���.+���+U���D�3��?y|��T�\-c���/ ���� y��P��� S��+����� @i�
�`���#��o�
�l�����S��wcR�)�^�X��"0�,�����Xo�� %� ���sG�{F�P d�E(��I[�2���3���H�'��\y��e� �B!�Bh�aP�A���*��%�" ��\�P��� ��}����uZ[���V9�g�m���?6�6�� �M�p�
���);�eq�?J��e�����-��!�?y�n�-��o�������,8��VHyB�{����sF���9�}���|���Y��/g�"5�;ls(l"��� ����R���X��o&V�_���t��CT~M�:�������j��YB�, ��2f���Nv4I
T��$[ !�B!4�0(��`��h_{��R������m�$���+Q��2�@�*'�����\����:�Y�z{�z4�.o�Fd&�'��BVe��y� ���r���W���0��d��V�
��$�GvL5�
@Hf^o�f7�C�6J��Q
��+ ��u (�Ejd�[viErqc�gD�vy~.:~����DV~htC�� ]�b��l �zg�� �c z4I�9�������*��Rph��,2��B!�B
$ � 4��\� ��J��q%:����6u^�B�|) �w�� � %h��=�3R=��v��Us��������*�']�<���t��E
�~�6��^�O�#K>��� ��"~��9m�Ak�+@m!���� �N����Co.Wh:�����F��G�Zw�� ��Ks�O. ��`�7k�Jz� ��H������tg���n�?G��o�����M j(�yC�w�B!�8�Ah0
�r�k��+��>�������M�_��'tR�y�jn����L`;j�z4������ ���|O= p��g~`LK���H�u�������%��:�Ti�h��'�O�d.�G}��w�P�)We� � �{�����x�$
�}{� ����h����� ���!J��R����>���K��[<#�+� �B!� v_Bh0��=�2���zS8s�N����T���������\# �X�����.�nS�3��~���
����+t�d�l��E]�y�B��Q!h�'*�� � C5�UWW�F���Di^F������(�aS�Qkr��K /�� ������[M�F��gW�>[u��������!�_ �6�~\!���z����V����CB!�Bh��� B����5�@��Q0$N�>06l�Y*(����hG{~I�� ��� >�����5��M��r,��]s��y�� � W"t"��:�g$3/����z�c �$N�Q�3_� ��>������3���� � B�m�� +�&����/ @
�8�y�:����lQ���������_�������N����&#/b��>�0�B�1y�&}�l��G��7B!�Bh0 ������/��g����m��-{��7O� B�����7��Nd���_'��B����K��\WZ�����I�R�5��y�:���w�]��c�������������>{F�m� ��W�u_�m�� �;��/N�I�1�~� })�?���#GJJJt: U�.;Xb�����'��O���/5Z�!�B!�3
�/��qO���{E� Y�E��/ �������� �kd7���,� �p����n�~�!;��A���i��� xJ'*{�����o� ����<d �`Y��a���+�j�j�� ���3B)}��G~������^��
�K� ��F�9���ewB!�Bg`p~y �{�� ��+�Yt����@/a��%-�|)����
J\P���W�/�>���]�C�Vx:�o���J�������s4~���7���������
Om`ID�r��L���~xp�� rQ����%et��c���G�~��������J��E�����~������ ������/�<<V[,�!�B!t��w�)��E"�a��9[j2]�i�Qj2��� B��m�������Or ����k�L�b��Mw���oF-�Dd ����}�= �(�z?�������U��<�&��������-����d���_�:�r���N!�EI�M��pJqcT��R��x��RJ���>z���~��HBW8F!�B]�0(��E"0�"u�~jKy`�i�% xc�e{\0� ����-�]t�h�Z/��e\�� zT���Q�`�q�<5�8�S ���2����������i��"i���o���w$t,�hP3���BA}���<"�o���<6v�} ���.Yx�� ���p��L�n���������?o
��p�B!��()���������_�����F{S�C6SJ��jPS �t��z+��Zn]6�6R"����� �(� ��`c�)�����[�f��vy������|}�2� ���`5�Z�2���w)o ���{P��_�;�V'uQ� c ���oI�-��0���: �kDF��R�:���_��<��Rua^
#�P�h�����$"rj�sg��bB!/#��e=
g��U8^5sd>�B!���k0 ��� �ci��$�Ij� @(DkX�H��(5���2���B�W:��K���2���}��� 7����3=��z[9�d/������(M_yC�}����?�o��� s����+��Y;V�!KB�C�"�m�!��S��������� E��e �����+{`�%�B!������ 3e�xLKR3<��Z� �IO>X^���`,��1-I "���L^v2����G��vQ���kt��F ����@#?Q4���n�B �% �S8Y�^Q �' �j��e��A���7;�2_��G&�q�� P��XV�3V~�W � ���E�t�:kW��=�'0�B!��%k� tQ����9j$ {��4�Z�0D�VM �1� 7Z��`hX1c��:y(���{�I��e����I����Pc���G��d��J���O���� �o���� ��X�&O ��B�Hp/����/+ � ��&� ���]I�)a�B!�B�A�.2�������[r���n����/�~ �n�/���_��hG���s��gZ��o��Sy�Wt*��['DL^����S�S}����� 0z�������%��d/��>��#�C�yI8��� ���pA
Q�k��*Ynt�"1 ~��^��|h�f��a�
�^S8�F ��������o���!�B�K _B���a�,~q�p�d��"��$�^�}����e&Ba�������2��I��m �� M�ZZ� Ml��^��d�������3� Q�{����-�x����������{� m������I�vY�y��v����Nu���2�H��D���j��� �����C�L�����6Wd��+9��-)6&& �2�)�eK�� �6|��Y�����'�B!��$ �:7��Y+]����fI /�� pylg{��&A��z]�?�VUd.]������(�:��E��+U���wX�U�a���S����0\��N���E�?�_���e�����d���4XZ�]���
��t��F-�4�:�_��=���s��"0� �$�����<��)2C�]�+(�|�^��;r�G�|�CB!�B
���� 3e��x]����� qj�?[�����^���<�����;
�n������2���0A� ]���i�!��c�,k?7�FDa����
m��U�-��3�h�#���� �����
�\�z����$�s49��N��0�� �"J� I���e�v �&{= �����~B!�BHn0�� ��RRR�q������,��� IDATZ�*p���_�uB��1c����(���92�3��P��l� q���=�L��M�]='|Ga��n��������~9/��k�u�Z�qu�P��"<#n����u�N�U������.����([����sP�J���[��M�8��;��2����Z ���$R�@�v.��I��y+�Q��Wm�B!����_�/�B���� �����{�5���� ���STT��_|��WS�N}�����&:rG�����mcK{��?(����z{�A�������G�u�/���l�=���HQ^M ��� 6��O�|_���YF�B�O�E�4D�gZK���������IA�P���1u@`���Z�R�@,�C5N���R�_�>%�nB!�B(�%�!��]���'�6lX� /���k���q �y��
MMM�w��S
����iYC:;;�?V��c����Z��l]�<��B+�G(t~-�e� �t�����$�UB���k4��B?��Cv�������&h�:���Y.��gu�:bAQUi�����=������R �m_9,��������������� SkS��mbXF���]� !$�B!�
t e��p8 ����3JKKp?�;����_ S4+���h��G�,����Nk)��B��</#�� P���u*O�N��)'�z�
N���{`��@��W�� 9/�D��hk���O&|�� ��h�gmM1�����Q[�����=
�8���=[����D�����:6�jw��A-O�����Rc����B!�B(�us8 �
2C�=y2HS�@����+V�X�r�����Hm�<ez���=����������4y@��%�Ef^P��}ind���y�:{�n���/��[� C�hO�����ztf~d����
Q�?%*r�!���'lM��t�'�W(t��o&D��[���j�R�P���Y��~���m��Ke}����*�R���{�����B!�: V�\YXX8��8
�q�y^�Q��G�*w�ZA���h_{��u�UX[f"��f\���qYI���nW�����>�f��� yz�+:h\V�`�y]Z0?�8'3?23?�wdZ�Z�7��>����s�Q�.�����=]�^mB��|�-W^54p���Qu�.���>V�q����9/ L���� +��TI�8*���)<w��`�������U�j�
Df�?��B!�B����+{���-�b`��%7��d!�����K�E UZ����.�
����������e5�VO��#�t*opV�R��9��,x�%i|+�|�e�����pW�WsY��]��#��o�%��1���&\��9��n <�dP;�� �xF(�?�b��"���
W7�4��H�F.�4�8���B!�B�0(��R�x���Z}}}o%�:G:l�l�=`$��
�E;��KZ�h�� �V,�z���F+,��c�kn ���X&5J���������\��������i������[qf��N��a�8\O|v$��
m
_UM��D]4X\��%��z:�"����Dq������"�E�CSP6�7�) �C�t����gQ� �'m
�/��3���B!�B(4<���P(�����7�xc`��.51������������OA+��O0�yJ�7� �T�]����@�&#/�
R���K����.�<�lr�+���=N� W�}`! )��g��:����'No�L�p4k���F�>Qg^>e_��r�P���%�@� |�����m�D
��!���9�JOO�B!�:�0S���'�x���A �M�6%$$$%���"tV����;B�iK j��9B��B���R���n���������t���I��U��M����T�3 e
�
�
���1m�v��[F���<� �r�����>�<��UC�i!�;�S\��x����p^`j{�o���|$ e�E�!�0����'9�/�W�p���[�����2�H��t��k��U�~���B!�BA]B�2�(��?����l6�<y���� ����.''G�p�5�455�x��,� >���^����V�|��5%��}i��t_���S��� �� 8:�
2�d j�SC|.�J)3�����w)o �$O�)���a�[n� ���V�#�A�/�/��~hM�u���;7g������R��Izn_����������&�f
"�&�>N3 ��~�p�|�Bb[�V,� ;�n B2�"~�g�B!�B2�PP�a�u����s� w�q� �g?��4��s����� �y�)/;�R��}i^� \
\0Q��t���#w��P�b�|�����q����& �G�$ �8��#a#���[:zt~o!0�'����9'n �Wr�.��wf�)Lv��w�����IA�zkw����q��,��+�1(�B!�:�.�� B����������N��@/��~N��< �N��>� ��6 ������tt���S�E�����9/E����)�4�'��M�Ug|x��f�-��#�zw� 0�����Zq'���6&�sR���Mm�e#;����u���Ye�*2�.�FVk�g������'"�B!���A�|� ��{Z�|C�jP?06,N�*�e 6���J��MF^��5
}����� ��X�� G 3�x��C��:h7(�����o�w�n�P;�l�E���E��1�4g���St �l�>� +�P8�F��V��[?����'sp.����O�����~
7.�|�w�}d�!�B!�.t�R��,�;B�3D�g��.�gYx�y���k���,�*�%84����t�]�D�h��I���}27i����H�]��+k�K@m�G.�r��X���u������ �Uz�����w�h���4��^U�>�kC��$�D��W�_��3M���1��e��w�/����F���PAj�b�9t���B!��%ep~y��K�o?���`St��L��&�#��� �[R0<3?5�.s�k�[[
�?y��������_� wZ
k�i ����j�;
���w�� �Y���P��/n�=�S;&wT�o �����"Q���8u�OD ��=�z���KJ&L� ��5-#�3p�V9� `u�� �eO��B!�B�B4E��lC��X_Z���eV�]G�3��c�}��0�s���dD�����]w����4��t���R?����@��-cf���i���J��gQp�F�?����I�����we�,������Q:N����y��I%%% ,��y��W^)���mv�;Th#l�:l��q& P�u�M� 'iD�W���!�B]:��w��A���[�y"2 ������B��.�5p��kx�a�����" $ t�n����L����g8' ��5]����g a��`�]��N �4g��@�o�%�B��[]�X� ����/<A��?�|��������W��W�� �j�v��%D��t����7B��B!�B9,�������1m�v��[�Vo/:h���~l���fX�/"C���
����v�,{���D� ���(�U�|�����)���K���� Q�N�A���PkS�>��+3�
��Xqhx�+3��������49��+?n���[gu7���u������g�]u�U����Rn�f�~�t>����]�[K���<zN��B!�p�)��%a}i}�� - �tZ 6�B�|���H��6�L�R�X�2������O �H�p�gu���C�]�9��
����s�o��9C�� ���J � ���v�1��J��������� ���/������_'N����.N5�>G8�tm�;0�S�!�B!tQ�L�����r���� B�-�zL��H,����~���3Gj}���x�qM�l�B�l���w�%a�J�Q��3��T&�����_N�� �o�kC9����B�M�;�������pTz������������O?�����0�-��vFX|Qx�����s]�!�B�K���+=_�z/�aP�,��`P
Z����F�@^v�=��@ �������f��6�p:��U���5 |(wwu�#�� ������C�%F�a��c��1 `����M�mM��iP�#A�Z�� ��ny����o�4i������ �x_�:bA���B!��V�\���>�{ �2]R"�]��F������K�� 2��h��1�"&�Q�S �)��V��
�'����K`��
�osO*����j��%OD�z�fj�"!^�*
!�����q%O�k*�T��- ����� Ur �j���i�k#3�B�"A�!�B!��:�)����VU�d�'Ov�i���J�j��3Xjw���c�N��f� � z**oi����b}����.��j2�0 �R�a��'K�#��$�����N j���m�}�������Y@�����E���tw`���0S�-RX{X�L��z��i��u�`� �U�������
�ig�"�B!��9�A�.ZR %�������-BR� �����$ ��/���6���{���?���V����T��T����]�{���4���L�����[�� �)�����`��:�����8�7�2s���Wf��}��� �6�SM�������^��_%G N�� Ss���
���r�!�B!4 0(����@)0��- 0'j���� � 7�� ����"Ed$v����.�_\��TT`}_s _t���??MB���h�sM"���M��:F�j p�<�>7��D4�*De���|�>O�k��<w"Q�9R�N; L�G<{��������/�p��= �\�}�|!�B!tV`P���'�)6�.��u�u�_������� ��1i�C�'='�d���)\�q�N�?N��W\mqR ��'<�S�J1�n�Z�{�x��"]��\����l������W,=j(n� �����k������@!R������ �6|��Y�������;g��@!�B] 0(��@*:h\VRn�����ge�O8�#E��P���X����rX����6 ��&k������'��Z�>{�T&�
�dj�����9��r��a�O3��mkt�'*{������� ���C&��Xh��`R���S�,�>��'��� X~�rj^I���3���<nj�i�x�R^6���u�����a �p��7���������l�
�7��+�_D�+ZB8����B!�B�<��K
�����j:�� �tZ 6��+�?[��h��$%�����G5��&N���0*�.sM����u���;B������j��!��bw�������2�=y�����j
">yj�q�(h ���$��*�� ��� �_��F����%:�_6��.v�a���� ��T8�F ��D0���! ����J��zgu��]��
�U{�;~�j�w�7p�A!�B�o�Ah�m]tZ+�n�/���_��hG���0D���% pP}�G�?w�C)�)���� �,�. @y����� 7����3��!&��<^����6]]�vS�u��:�W�.�,��+�3�#� ��yu�6���X�@{��
����~Kg��Z[��M���u������59sS�Q� � �!���� � �md�� �"2Th�p4k����^}l���6 ��B�@o
!�B!t���%���{MG��E}
��D(L5��+S
jJ@j_]}G������\M[�`�]�7�����xh��!qQ ��C�0>���� Q~��{��*���x��yn_�������� �{(]z9�������m;���{!_U����,�)�+u�. �]���� P @!^� �"2��+�o�`�9 �� _�m��3��w ��� WB!�B]40(�� ��)g���8|[�)���J�'�f� �{������X;_� ����Wy��(t�! ��>�f
z�I�v! �#\ ��F��\�~Z�m���K
�������&>aj�u�=��J���T�������1����Jit(WW&J%~��EU��JK �; #=B�|y�m!������'[x���������;��/B2��a+�B!��Y�A��O��|���sH���O��������^sg<8�t-�R� 5�c?���K�����
����#��{�*('n��y�Q���)��� a��.��#���� *v��������hHN+���m����������S�J�*���1�;���YE�O�����]�`��$S�M��4N ����>��A�B!�.j�Ah H��a������V�|c��y���}I���;B ��e ����t_�9������T��y! ���L)�:F�|���:��{��L���_�� �� ������2z�[S��4t��r��/ ��w&&��6�D�y9�L��!�s#G�59s�����5�P0J �1t7�������M
����H����t� !�B!t�� B�O�34! �ul�� G�B� �{�TM P�$0D� �Mc�*2���`������� �u����Y3����>:,� r��%���4�����[�F� ����[5*�����t�ts��e��� �(��;������������FwP���S���;/}���z)"#Q0�8������I�v���[�Fy�����B!�B�<��K
�+b��!`\������jP?06,N�*�e��es�5~#����� \���4��S^v���� 9F�Q��ot�7�P��,�M%��;�����UM\����-��vQ�?��Qg�
!574��J��� $���#�j�X�3w^����
���i�^k� ,�L��r,cP��w�]Y���B�L\%�O!�B!�.2�)�� 8����i� �ms��:��������Y���#t�����S��}.o1 Z���;�GK�D��>�Ai�M����rW�#6���uo����GW2��G��.��
m������:`!�r��s���8� B(���`[G��s��K��]��k �����O��;�(�^�{����I;����,k�yw��yXP!�B�� �t�~]� ��xX�b���+n/��1����F8
��m��m���
��4~����5�J��4�p��n��9�;b�V��f�m�bAS���q���D�A ����L�*�*{�h �M�|eT��up�y����Fw@G�*� �y���G�r�������4@ �"rJ��:!�B�����+ =/a �2?!������vY��%��o��������s�n �2 � Q�g��9���>5�� *��&�<�������f�nf" �r B���yw�JI ��+K� �e}��)2C�] *�����e#U|��[]��qS�om�8����<��*:h�����gey�SB!�B�����}0���28�h`�NW�=F����JA� ~�3 :���N��o�l'�u X�9�D8]�������;5� z�jU��v��']�<e\d�oS���R�����B����M��J��=f��u���������p�2�6�&��6s��*Va���A/�/�����N�xs��y����B!�B��68��c�_����F��2S�Up���*�-3�1�������3k���i2�"&�Q�S �)�����}s���S�S��������b]w?W3 Z.����|OD \�p��^��I`�����e5�V^`j{�o���|d? A��
K�
���v9�s��7��$y�X;��&'k{-��|�1�.Q�k��c��B!�B�#X�����J�' ��Pi <�$��[\a� h�5N:[d
�����e���\����y�*1���#/
�X�%���)i��Z���=�ornV����LPE��]�h Y�<���Z� �t� �
�`�t��S��������q����B!�:�0S���# ��?9/�#tj�[�6�q�9b����6��Gz#� P$�G�����
oD�j�/h4��Q���� ��bj�w���#Ed$Z��v�� ������������;��� �B!��y��2�e1� r^����o%�)g�} z�hc�*2�.�FVk�g���e�)\�q���>�����nz�����N��/z��$����l����L!��c �&_Nh�8[e���}A�"2�_�w�1U�����9A*���`K� �B!��9�A������ �#t�"���y� h:3{�����.U�!������� ��.sM���D��7�0~�B���B�L\��g �z�e��IXk�]�m�%*"}Va����, �Y�� IDAT���t �lN�(���j�I�dqA�2�YS�;����"�K� 6�+��tZ�����Z���H{�������|�!�B!t�`P����:�,�����*��-�4] �[��Vh2�2�"�R��K��e<)'�*�������N� C����/�dX��hV����}�!Q�k���R���Y!������:��cz�M�� (F$��nW��Sc:��U�-"���P�G�Y �N���3���B!�B�e�[?k�z����3�[EOQ^gWmEQWf~��_o ����Z �`M��`cY�����Q����9�9���� :�_:y����.u9 �[���j}u�:V�a*�Cd�;�X��c�?�:��vx!�rk�K �b�r�?�e*o�L�N`e��p��?���waA��A!�B�s
�"��3��; b��Hm����=�?
�e)' ��
Sm��[\VUn��Wfl �������Wfl� ]���I3S���f4FdJv���;=�a*8]���7F��:-�{QE9?��O (����O������m;%R 3hZ�qF���q� ��!e���o����������o+�_N������B!�B���� B}�-�1��TQ���� �g\ph�|�Hj� z�����.����o������ U^��?Z �k��� U ��������7`���S {U�%F���PA**f75�J#����ei��� �(�p�H��� �c�O
��w%����{�T���-VG�WG���U � �M!�B!�� � ����W}���
���
��i�����r�(h��<����V�(p�VC�{+�� _i:�VoW>�%m������SN<�#� (��\ @�� ��n���/J��&��h��������o/7����������?E� X�_��z+��Ve����>"D�����5��M��r,��}�����]�\����=Z��*��W��
��p;!�B���PJ���zG~����w_$��yM��;��*E�>��~f��TdD�R�9����d�9�J��x�I��_��(�h��W�&��@QA�z��O�l�| �� vqy��_��[z����f ��&R�� `��bi����Fu�� �U�GJ~���(��� �e��j�8�e:�;����s�����q��Q��#]J����B!�B����}0���28������A�H��g�m�����3��,��) ��P��Xm����A��KT�w;��������� j��S���@����k����wp�?���o�� �Uz�����j���_�9�~� ����_���.�FQ��'���X�}`����t���������H�K�����#����KN�6 !�B!t���w����EH���L��e��2xC�� � �.X�� �� ���M�]='<�8�.���U�nI�E�����Y5]������
_Um�Ot�� ��#2 �QT��< �\�L�Y�A��z�#2 q����L�S C~?�;B!�B�O�A���Ic1��W����'� 6���0o���S:�Fd 9���l�_�������e�e����������L%�R��t ?^�h{n��V���S�J���I�?+K�-�w=P�%m���g��"=���oC!�B],0(��l}i}�F�1��Nk��2B(������]����u*���#C�����nS}�;�d�u�e}���}9�R2��q��9�XB-"��x������[0rr �_���z����B!�Bh``�%�.`�cJ�k��c�� �c2 !� ������G���� ��(������������yW }{�O��._���:�K�*\} �l�9��~"�+3���� +.�h�\�� ) (�����&��cu�B!��0a� B���0A����X4�"�]%�����'�Xw��N�xk���pLo���X�8 ���<�����;;�*�5����MR�3�]U�~������Q��f����h�'��W��]$ N'����k�yhYi�����l#�T����,�B!��8`��Y@NY�r�@�]ZRJ�$G��Fk 3?������c����7��"5J��L����������\�q�� �����oio|G������
�����S�~�!��n �:u�>�P����z�`�p9�.�J�����+� �B!�PoV�\���>�{ n0v��� ��Z�B�����1K�]�R3w��MK
��+�_���Z�y� ����]Nk} ;��b���_ ���� �:�{�y~�+:A��h���wL}�nY��desSz���O����N���r"84��v�F!�B�>
�/�x| �s���,{���?���V����D(L5����N���vG��m ���}
l�$[�� �f��-Ia�>��; ���9'j�4� v��� ����Z�����E�� Ny�����B!�B�5E��lCn����� �Y���a!�, ��u���H'��W��9�������w��V|,����8\�
��?� ���_#O���Wo�R*���P���@��t ��j���Q�9�����zg���A���VyX��[�� 7�A�Dp�e�d�j�c� B!�B��
�/�XS�s��J�'" v�n���6y��.�� @�-x��v7��h�/i]��}wC�I%��+K�����R�����Dd ������{�u��9��a�/
��������7���Z���lX�M�����\r8eYC�����.��t* $���Wh��'�����g��( ��SB|
0�!�cwh�����������i������B!�B� _B����HA��g�>
����W�v�L��Y�;,�j������ ����v�� ���?�fM�� N��Z�x���{�N�� ���� ��)���
���b2Qa��������)�2��c�\��������Q�?�� �2�)�eK��iI�t_��9, �U���_@y ��5�� �MF^?oG!�B�� 3e:'bB*�v������j\"��&{�v�! w��GIB;�� �S�v��^�����H�S�.m��p54��� |��&'~����G�=�:�P���w$t �&u5<�8��&�(����B��D�L��h y� ��q���������K /����]�����Sw��o� �������RD���x�������B!�B�e:'rG���7Z�f��#t��M#8�*fJ_U~e��NmS��}��w~�e���U-�o�6x� �i��!a* �f�8Q��T������N[����&�� C�j�/���M��9p�5R��3W �Sd���Z5 \�������U���U�2�P�w�k�v�N���G |��y6o����!�B!t~`P�sb�A����8-�`H��}`l����3M��{F�O��1�����c��C���������:=DE����Tp��y��]*��=��}�F�`����v�o��B������Y�c�������?�/Y���ISL������.�7����� ���
Y:�,� ��w$�G:���D!�B���5e:W�%�em�c4l��/.#��jPS��f���� T��;��.���UP������u�
^QeH�&��U92�\nV9�� vMi�EG�� �1/; \�'�1#����� �wh���R� ��@��)�!~OIV�O��}sd `S�u/4�otEG�5�ggd����\F�����/�9��#���dU����� ��(}I?�F!�B���2�?�#to����� ���������
6�q ���2 � $�� �� �;���� �?������)��#45�D��g�\Q��w�]F�Pj��������Q��_BS��� ��xnx�����)���s ���[]^�\�=����X=a�'e����R�������.]M�*}�����:���1��'������{B!�B���� B�Oy�) ��<pY�d�H����{��p��YY� 0,J}���;-Iy�xQ� �6�K��d� a��O�T2�e�]� ��rE�wDF@����:U�`�8{��[]�H���Z �=�� ���"" p�F�}��)"��&s@m���`_1����
5� ����7��z���O����X�c�/���������-<��B!�\0(��9�������)
NO����� �ie�x�i�������� P�i-�XF��N.�9ra���K�O��9�\]����w��QU����s�������/A *�����vq!(�R�U���B��R������R�*RA[�U��(�P%aI��{&��������a2sg�` �y����9���H�z��u��<�������:�@'������\R ~kv���U)��� �����KPq��b�e~9a
���
�( ,�����@\]��i��"��wK��>QC���f |Y z����.G>|x�?��g5xFq�{n��Z����]� �� ���8��.tc� !����� r��m;�Z9%=z<���e��fg��o'�>}������d`���� t2p�lu+�8o/�b���.y�~"Z���������0icA^Ih)_�M����m��DsJ����4*}�������FG�>�����2A�����#^�������K���?u� ��=9j�&8u���4��t������W tO�����K �����]$2D$�zug�o�!g�G*��#�8��Q�SW ���$�U����H�#s��N�1�C� y���}���>. %2D��t���=�27�8��-�^6g6����8���2���q""����=�*}�2_���9����� A�)�,���%a����� *o�H
o�Jk~��{��D .0e �-�
�5���+���{�j\ ��O���~���>�m����,�+1�T����[wr��Y�T�����<�����E�p9E�|�UX[�D���� ��������@� _q��#yD����-���4�- DT�W�l����N Q���i����.h���Q� ���� �h�
p�CM��Fs�(�v8F�#��m� ��������]����D��6�vG���{�+.u��u7N��xs��on�������}��#s�\b�5b�lc���f;���� �O�G h�jF>�_>��!�b"�
G�� �������P�
<v��T�e���U��[(/������;��%���K��_q������@ �2 g��j���5_�cG��25{�RX "�9qN�@z�G�<���W��v ���2��(G�'#������&�?'�t�����U�[K����\�S���l"�\�|sQ�?o��p�:n|���4�Z��0dp�6""�%�h�rS�@�f��d����PMdN�T����9Yx��Ae7�c��Um��W�(SR���5���/SM�k�(:T�l � ��:7���Y+�� �q���2�����P��8�p�%q���j�0� Q��D����/� w���:A�fC��h���'"#����7������5��_�m}�'�5�YL��yI �)���������R{'f�$����s���&�H�"B����������1
�Q8T����
:��A��2�>�aw���#m��F��S�M �+zY ����_��S���� ����f���D=���/��K��<Eg�
:��_����c��B��2o�wl���k��������0"=g�����������}�#s~��A� e*�}W/�`�G����}�����}z"�p&�b����6L����h�N����g>��� �eu�]T0���6��WNh�[+� ����z��[���n�U ����������1c���3�o�z k����5 �ke ���
2����!�d�4&�w������'���w�""
E9D����8SQ��EI{)����������jC�]hL�(�c
6���-�) f��'�����L��W��?�drP�(����K�FE�2����j�����sKz"�p�?�i�R0�������������������f0)��2�M��Um�f��[D�L�sI\n$���9�#� �� �� ���AV��q��$�;Y���
�� s�y���Q�uq}�����M����f�19�Y�miQ'�#�����D���U�I3?���o�c��UO�%�_�U��������p������7�>�ybx"��vFn{Q��y��� W�[�/�������������"
��D�
�}�����F=s$ ����4� ����~J! �� B�����`��s�^�� OO�,�����9�{@b�lR ��� Y�[��u����q{M� Cc�M/�OY9%���6���|����3��g����o� �>6��!oO���LWx���%+s�
z&���������/G�,��M�9KH���Tit��$mR��XHzJ� ��H����'��&Y���f�Q��FSy&9�����!� [����_u�b� p�,X��~��Kl=�����g�
� ��m���MV�O�R�hg��oae}M"��mLN�NC��l��$�56�|_�H �; W�DTi�$�W�k���y�5?����U/��������#�������� }����}�Y�����?V �d����-�L�8!���x����#���k��p6�^��^�!|)���W�]#�w���v���.%�[3m�c��S�c�I �o�g~yGM�o]� ����K�'�: e ��������|�N�H3|�.3����5�C����%�E����EWH�$�� ��q��^Z�I��RW1�j���U#�^��M�����w2W����S��0�����S���P�l �?]����i����6X ������O0��#��)fq�}� ���� k��M �2 �:MC�;�XGg��dS��?�����&�:LA��pEZ�D�D�U�����!�n���������_�:=�h�I�O�\xI1��@�.+^qG���D"��-�s����MN ���������~���~Z��%����{7o����Zve����6T{ t2�>l)\-Xov�.������O oyp�u�G��^B�Pb pa���w�/=s�/�8�����d�?�>c> ������<z���x��2{b7���� sGM;��Z�,K{o���������[�lJn�����g��[��\�~��-[��U��/Z�q����nJ�?NL���?���z���4��,�[�o{���:xFB7 ����_��S�\��dS��\}����#�\q\1��.f4QMT]��X�� �6N�;�������F�f��*w��>�Y������[�n
Od�h���4��� ���q7L ����U�z"���74s��z�2 p1@(��_�i������������6�O6E�za����n%��G��P4z���*���V,1���_c��Y��N��0Fj���� �k;V0g��.;�~��$OR��������]-Ys���>����[�n������#"��;������~������G���v�J��tUjG .H=q����g���3qZ ���� ]i�=�!�������q[�.;=�����.yt��P�%�N&��'{Nw�;������~j��>/]]��"�Y��_�?����Tg2Nc�q�����?��u�������9h���;��G�R�����������F����_H\��
J`�y|pw>
�n��_�;�� p���[]����� =��~�ksG}0����D��� �����uq���e7��w�E�w���I_�aS��������_~t���xI* ��m�������0~y�� �C� qF2�rW�[�����3�!��W�`�BO�z��7���$���� \z��&��Y��e��s � ����iuD:+��Sgy`��b��r*��Um�fs)"!�ef�`F���'�S�����������@�����;�l\�OW�T}��rg��G���N\2c����f ����r,��GN����{� h���E|���_�=x:
� �E� �V�Y�tDJ6�]�_��z���j�/'��xJ^����]���Q�����U�[��y�D&+S?tE"bf�a��]�Z�{�������h��� 5eR ��dG����b��c�2DT�Da���rM��<�#���� 4�����A-�x����C� ���* IDAT _�*d5�,����H��W�~�Py�G���V��w����������.�q�5n�n������D���,��Q��Q�R���L�����N�����w\8y�����Q��3�Cv\��R8GW�m�]�,(W� }K[9 �|�P@kL�i�p[�E� ,�"�n�y��:)�?.����L�y�_�i����B����}��o�s����X�:.4�L�����p�O��8X[GD�e�������`L\�i�h�����Qk��^�o �7����I�� ��2>�jx5�����n%�����,� ���� b�m�f�*�O3R5���G�j����P���qj������?�^I&+��'j "����)�[K��lQg������e�B#�%|��P[P��28�5�}�4��U�f�uS�S��1"N�H��vZw/ �~��hr����d\��
�2 g$�NJ����^<#�8��J����8�6{WI�W��A�QL���������u)���?�vuY!�}O\�c|�����>C Y�=���7>���te��~k�V �ze�-B+q�L`\i�e�:� ��Ag����t��+{�EKN������1>�������Ey�<�S�^�'.W�v<>��=D �/�2 gd��!�= :�d5�M ����T"z<�m����$R#��M-S [8�9^R�N�i�b����j����������� w���$��d�:n{�p�h2w��� G�����~9HD�����tL�����T�y��FQAWy�����y������W�~��T��0��^����N� \�g�V�E�1|������ 374H'k�|�����u@���#��|��MLL5��%"g��ES��#y�v�������y��n�+�9���k�52��w~qm�k|���^
��#� l)���9���u���#:�����E� �d����FS-��L@�% 8�z��w��8S�:)�E- �����qJ6�w ���6������}��A������?^�����Z��:�}Y[��~0��w~�a�I�;S��t�b� ���9���=���O��z��D�6:�n�i [[�'��sE7�
z*c���(Og{� .��p�����8�l���B�B2�F����sG�/�mv���QC�e���"��w-��� �.��#��:���s &���_��_R4J}l����� ;����{ ]1�>d��c���i���3�o��*�%W;b�sfnT �(��te��~u������<��
��!�8g� ���d���{L�������/�n����//�
oV��=����V -�^���������x����HYVR�U3!� �l��w���~z�c��`6� 0�II��js �� -���L�q/���]�(�����X� [���<jI����+�Z��;I ��� ������������y{�)zc�x����^y���?l>����;���� ���C�1��R��${\���m���:���_����e�C+n�#FIW��2{����n�_jOg���EYML��+�DQ��_g����H�\�� z]\��Q��'�;i��k����&Y���7������G�r �� n;��;I ��5e ���>��A$�iLN� �� ��K6��>. ut""�NYv�;?�EgK���<XCb������E�c�,l(�S��hf"�em��F�c���+����cLNrk_1������F�[�_�
v���<�/I p��N��Fs I=������1!�
�k?=�iHD��� ��������h����7���yG%�Ux
?�*wl�.5�QYH���S��p�m�������]}�t�=Ys#����������
�t�$
�� � �Y} ���JGW�_�w�p�y��&$$2jmm��HY��}I�A��� ��?^��w�C��)��?)�����6�o�����,��=��4����B#�*&�wx���t���8a{5���w�M��1��f��JW����� 8!�8kR��f$9j����j�:��Y P$ ����G��b�f���J� #�����A�.L�+��� &1�����_l�p��}�N-i��D�\7F'BOw9��V1��kP��<�U�'k����p�w� ��1D������/I��y�d���~G�����w p��� ��:���{n��Z�����nk���$�����vDD{���
��7�����b�j��
��v6��6���ES��������O=0�����,*O��wK���[6�Zw � %��x�sl����8yH�, ���E����o���c_��
�����H��U�Z�-��YL���<���mI�����3�_�FC���O�Y �:��<��I gB�3 ���������DJ2 w ������+y6�������[����~�s�$� QS�b������~ m�{UsSwm"�haMTUz�VJ#:JD�i(��$��d�:n[J���<�d����4#�OE�M�E�O������'j
<���*��'j "�������
e^��[�;� � �@8�t��,Xp�� ��i"����c��rJ� �9p��cG�dg}���DD�w���L � z�CW��F��={?(:XQ� �����tb��'�^�u�N� ��T��MD���gr�J�HG%��=�����khV ��kh�qQG�cuFQ��{gP���T_�v�������g� �DF�QL���Ng\��U%iG*�# p1X�`A� ��~/�a��Y�y�
p���� q/�`��������5udm��G7MP�}�C�����y���wk��1��uG�>���8��p"�������R� �k�l�5w�������O0'�LT3�����p�`t���R8�Y �X���BD�~�ON4,n�E�y�a������%V��l�]"�j��(��S��v�5}�r��t���n���D�������$ �yd����=3�A(pF�r��i�/�����_K�W��h���������^_�?���>{�(���G�-�u� C_����B�����>w �X<����)":��)�,]��� w��:&(y%����������P��ox(S�uly�w��� ������ 4}�>��
�/o��_W{ jps��.Qn��xd��lc����������&�N!����?oh6X�� �'@(pF��o7x~+<DdI� q�o��P���F&�q��b�}�v���(��H{����X������ ���+������W���������K8���$�+��`���}f.��#�%#'bDD�F�����n�������{y����6�R���#^���3��bX8y�k��A����{��I��gtv�����P�_���Q�Kt�� �ue ����$���!<6�R&�R��&��������7��,�'���}�L�����--�K��(�`F���+���;����s:G�{g�|����*��8�a�E�b��'2D�a�['�d�5'�n����"�j���Q_��������8gs7W;|9���S����9�e��� �*�F z�2 g$����x*M u��A8�(d�$��7_o���|US����~�_��]��&d�da5����
�/��R����o|����_��rA�mcY���o��r�d#��V~�?����[����������
��� p��qf 0�4?��* �S8�,1��OE�J4u6 � B�%�3���jG�����~�bVs M�bs ���Y{Z��Ho�h�XY�wt����� zQ� �X6q�Z�w����+�K{���+�[s��Yu��:�� 3i�MmE�]�d�B�R�R����m}�H�cg?e �DmU�J�=��3���D���&��~��g���_[8y���7[��E����� ���8#q�/l�� ���S�y��c�z�� -���kw�J�:jZ��j��K$ -.Z �NxK�
G���&�y%JsK�H�h$���^ ~Itm�)%������-yg��Q�kI<���/D����t��H���������O}~��� ��NE �u ���c� /:�k|�K��>%�$�1�z�� �����Z����Xl�_8sC�t�����Ew�z�(�,k� ���d��W�rOYd������W= ��W�
�t��z-��W�����k�-<;�Tf����\!�7�5�hh�d���r~������:��x�#'� �Z6a��I/}�O ����_��S������8�����=��CN6��1S �����:w{�����;"���b�dr��0\:����m�����]�p�1��N������='��<}e�
�sY���V ��0 ���j�������P�[��n��KO� ?���Kv��5��?% 8%��8=����DF����cn���AV��^a7I���esN6`
��wu�v�dhs��e��\3���?��]��H�M��i�����8�}�f��1iO�rN�'"]V���<�l�6�R6-�U��V����������y�Q# pV`� ��i�jO ����nm�c�ulv{-�19&�H=��*�������n��%����~�,Q��+��_��_Id��b�-�$E����� ���������t-Of�y[�6}�~��MDD��k���<A3��d��.J���� ���2 �'� p�<��������l���SVNI��a*'������p�K��^����%��i����L!�����g��>k�E^g�n~�URz��*��'j Z�����M*.k�1Z ���:7���Y+�=;�}/��)��i_���H�l�9_������e}���Z�N$���H������z�����o�oO�$;�B�{�DD��&�9��������Y"U��m�|#z|�����E�#���E�������� �9�3������ Mj�h�15{e5��rcq��!
0�.��e}�G�(��Ih�8�5 z�f�q��M���mM���4z]�P�_0�:� ��Q{#)�[Zn:D#��1�)���?`����1� ���j�5[mR t����M���8�� @ �� N�z4)��=�4.}���L �q T;Q��b����0���d&.
����_ ��C�U���w��M��^Or.k{�}|�^:����,������Y6`f��x�&C���3Zn���?���|�:8OMdT\r��?�� �3�)�
���c��sj�K���gb����� ����= <����� ��{����u�����|�(������� OJ{Zd�K��o���,p����������f�Aw�v�SyN� @�� |C�-��������Q�"*�+!���/i���S�GW�w�mpWKK�����sj �(&"RK���oK���,$�)��G�PU?�IN��s�]T0����k�P���{a�Yr���#�^��� t�'��9���ZA�M�k|j��d�{Z�.g�]�\s�i�6�C�;������������j[���������Q%�x^�?#|f_CM����o �\}a��r2��t����^ �'�j��@��[F�l�?���+ 8�z��w������j�����=�� ����%q� ��r�)��m��#K�m�/�+�� ��W�o|�������.}k�N�1��RH�������������&������� ����<`1 �} � �D(� �����������sG��t�bb�)�<]� *"CDnI��hT���>��C����"�(�!"E�- ���ef����j"s}��]�����K����B���2�/L�Z���@��2�� � ;e��:�$+1���d���{L��m�1%��i�}����m�OR8{�8�{��P���=/g�S��P)_�^���]�Ay��R� m0��>3'
|��1�����N�q � e��~d�3u �pC�iV�a���<��cs���Gd��^xXS_F�T����o�����'�X,n�#��qMBq�������U����z�;����6�����7MdV�����g ���v�&��f|�� @�'��9���ZA����5����#�$�����w�H�M>��\nbb��v/����`=�:�!"�5�C��`��8V��l�&��KS]����$QV'0Fd���})h$��2�)��a��A _<�Y��vVx��&m��w�H���q7d��u���#:�����}�� ����� ;e�����Jd�N2�Q&���c_F�KD<X��?OD��:�����vT����%A�����_8[�ct����5|!F���� �E""��� d����o��/^���?�`�����l b�i A
� �Tk^
�+^2|e��_�8}pab� �;P�..)fQ3�arE|�E� �E�o q|m���L; �,�Ys$o����^xx���� �S��;|��C�\�XP�#���+�6EH2Q� jK'���>�� e1�9wIi�[�������\e�dp���^���AW9W$����[1�6�<��L\�ay�3�;�[g� }���u ������`��$vd/&��1�:6������S�����><�8u�k:����/�B�#��
7|i����>��u������'�sM��/���6�yMu�?����s.W�&���slh�} �4aM���Gy����i�)$w����kbw�r �>��B��������T} t+{��w� ��p| ..crL���}I-�;:��o�6��4.k��D���&�&����'��R��L\� A����_�v�E������n�3�EP�%���,&�-���6Rlm��[�Y�&�,=�d���Y�vTI&�W��������� &g�Z��x��� �W�R����?�m�
�u\��)?
� B����6u�S�#�W��&q�-�o'�[��d�3Ko�(y6wug����D��c�$2z e�SC
���9���0���m
/�����S�Z��B�LI�1Zn{�nq����
:������P���vT� �
��~[s�G1Q���D�C.1e�����7�A' �������� |S8��A�8sur���O14�LN64z��|m���)J�J���7����(�j��KC�Pb=��j^��UR� x���������w,�~�~��k�Q�&����d� gEq�/dzkh0�� Z����rV s�7��PMdT ��\}GU`���[�>�Y��e�� �� tP+�����Q+���~��"J�viN�.�>�� �(D$/�LieDD�4�==�F�l�|<���7k~��� ��Lv�����|��y/����x��,r��H��M�[f�������o#�d�HQ��Z���� ����a�����:� �Tp| ��� S����w�(1fw� ��8{j�w��f�K�,�z"���"z)������Nk�����<���� ��7�Go�����cn#����O�(���.n ��6o�����Kf� ����Ols��� "J1�u��\�b�����-��3Ji ������ t�q�I� � ��^P�uw)r�7�6����D}�]J^]}���AK�4��[��%W|� ��>����B�!}��E�g��=�O�p�N���� [���o��~��}cD���7b1����~AnI���3��������2DT����HyPn�it�x�������S�� f��" @��3�����Y�NZ�`��~/p�:�{��H����������9�=AR��W��������o���� �������,F��bz���������!�?z�J=����$�����#"���.�/ :F�8'.]�m�f;>�y(�!"Av��'�19�Y�miQ'�4��dr�9�/�7����Ms&_�����Wg
G" �� ������[O ��/=3l�o�Y�#%�������l��i��My%�S���@��5�������;������ y�)�HQ L*���D�~�z�8� @�>�/�(H6���X����W}z���`]}�Z��m�4�"��p_0q������ '"&c����1�. yKW����=��%'n�bs�B���}�s7W;|9���S�
G�t1 ����_�QS������������C� �%���Qj(�������
�|�{����C�R,����8��)�k�"�^J ���=u�� IDAT�s�����\j�L����������������\����H:���4w���u�^D�-]��k�z�I�SC������W�����a�?HD����� b���+�d ��/���[<��9�q���i��T;mD$sQ��Q�Rz������Hz����D��|"�6�1����I��d�t���k}G�����0��w|�����������XGB����������4D\j?����K�DF���\��d �B�v��W)V"CD96�f$��$��@j��6%+����U�����R��g��rk�g�@M�
n�F&m�����`�H��� >�D1��� �����|!�wD0f]o�����Z��2����F-7��N���=��OW����x����-�\��� ^i�Q� �� @��J����]V�zj�K��/���Whf�^�������>�-��g����o��`�����? ����������J#;���N����X[~���$C��cn"j��P�����f�� m�����c����]�mcY X��g�]�*zO�z�I�j��w��z$Y)o��Q'0{%��/ �7����_zf� 8-�g�����'��?��#y�v�������y��n�� |����1�wY�"'��U o<XQ�e��5?����6�\�[�����3������M�y�a%�^�q�������.6LD���%��9����u��L]��o���m���b��P�a"
0�G)����l"�t�fzk����M4D4p��M���!0���o5�^�:l:j� @��3�����t~���]��� ��8�<��I�����f��<��I�~��m.�|U��u�/���E!�����;a�������h��k��+^����9�u�}�P �����J/�������U�/�y�� E�bN�\ ~m��o�]���.�/���� �qG/��3�/ ���g~yG�%�I�- �2��
)1�\� >��������� � )�������'FNS��IM�G�.�U�#���.�Ql�a�^ 3��+��,��/�Jd��-�o����O�k����cs����`�&��G����l>>gBW� �5e�b�k{��� '��e�_�������������%`�d�6���Lg ���� �V!����hL�l�sm��G7M�p�K�P��t�����n������b[�����������'k���Sd�� ������ . '�;�[�Q�h���\ ��P.F\n��4b�6Nz������uW&+���L�����wG��7�y%e���{^I� o"�$�fO 1�>�0�I�f��$O��� �h�*R�?]� * CDnIO���
�}���5a�O��O�5�I��"�],����q�����u���}��w��f������}�,zQ��dY~�pT� 8[z����K�<���<�����l��\��q���/�-�������l��� ��5 �i�lf�SY��;>i<�t���,����a�w\�\zU����]g]v���d��3� ����7/��_��f�[��xsR�1�% / y5�3(�<�~���6%������������7.�ct�,dc5��6u� $, pq��_�QS."�UJ�� ���(��(>���� �mkZzY�V_���P"CD�� j�/�}� }���E�# ��������|c������^s$O�\���[�����$J`�y|0�=���]��m������������h����7Ql�>z�o��Kr� �?8���2 �B���Z���D�/�#5�Q��A��?4�q H������D�w� �������W&yGT9K�1J�* ��n�T"na�|�
[������t�E4Xs�y� b���m� bF� �?���gK~UC����7<�i�����[��z��b4T � C( ��E�����
�����������k�u������<� ���/yd�-T|]���Uz����&����{"�`�����zt����1�sH����� [���0Ys��8�<��U;y����������
&��[Ow��g��/���`������t�� ��O�D��_5������V��w1��� � � \$�&O��]Na\f���w���A�����I7�EL��D.%��i��9�>S{.T_�v�b;�&�
���~f]09S��H�{ �L�{�����5�.ZL���� ~�d����X���� ���&2D��x����%NDr�k������8����� ����S<����9��
�Q��qIxd�cM �3e�B&y6����`'N�uI'��H}o����i�'"�w����V�N0qY�J���j���t;w+�����#Y�ID�%m�-i[h�7�������������������C��9k�����,�u��pS�(MW�pnpi�� ^>������6wD;m����>|�� "��k;mWF� ��� \�N��/Y�[����x�D"��2�~K�������� ���&'*5D���A�r�D�j����M��x��~�"Ms�%E�
�JB#�
�����z���v�^��� �QH#�u������ �2�91�Y1�DPn_��{y�05�!��D��&O�"��& ��i��\0|m�v?�!"wm"%����6i{�c�|������� ��� ��,"�����sJ~�V�u���)^1���+��57��2X)�+Y6qs�x�^Tz�;�M�<u���.�����6#�i��D�C�R��� �2 vo�����(�M�,�}����I '�;����& ��a� \������A����o&��F� o;�|���&��P4�G<f������_�v1"��l���F "Zo�����B5�� V��`hI��"�����hx���~�u�[6F_��A,9q_�}�A9!������W���s�})�� �U��� m�)0����r��H��������h�z�>{s��+�h��4E��� skc�����h��ocY X��g��de��C�hX�X�`�(�!�? �h�������Y1oj�K/L���^G&�p"��,��Y Q��{��������$����q���F���F�yX��[�>�MQz��(:�XN�� �^o���.q���$�p���l t ��C��������g�w"���=��>�$)�T=��%��D/ ���m��������� a2g��[d�����>��#�N0�O9�+}�nv�n��?j�F�5�aLyd����1�&m�m��Z�i������2��M ������eY:R��h�bifN����H'PP��-������������r� ������a��big��o��>�}��� f��\ �=��{O|O�����T����&��D�������~� ��� s�|�����
���~/�Y��8�8�D��3�1ss��f_���~�����>���{�"�+�O����MQ�j��[��Uo�1;I?p 3�? ?^�F
ZV%?��x[�
"#�����������4�"FJ���T��9b���{x[s�[ ���/�K9�E ����� 5e�B��L$��\FL5� �{`x0r�?5b�K�6�c��f������iG ��OK�� s��;u`�,��^+e�t���o�!��R~��~]��Q��QP �y��j>��'/^����Vq.����$2D�je�e_������f5��������n������ �>�2p! ����u ���3���nT� C����e�1F"Et�X�%y$+U9�/���������]�1M8�DQ��`�sN�?�2{���Jf;J�r��G��?suvS�d)>F�4�������"J0���3�{�b�"w�$���& �)�%6\��]��w"�����ED/���6 �v�{�S�n������\����� ��p�[�?W1����j�S0����W^���7�k��e��<v"c���>���d�� �X���}������Z�M�=���/���`�8U�}g��u� ���u�i����^�i�����;YOw �N��)R�&x�Ol�g�]V)Y���}�p�}��\elR4�~UA%��s�UIEw�VXE��i�L(�'���HxC�����R�^H8���4����w������y�w�r�L��k>\Z�
�����.o���{%��J������\� ����y�4�c��x��� ������6k���������������]UT����]�uY�y|����� �y��������VQ!� ��4�^/'�d����� �xA^Itp�/�K�F����h��,���
�DF�QL������EIyP�����QRf�� �� t��)�pd�7k� �:8O������v �o �P ���%�`��y���O z�����x��7���w����h���W����� _s���������?�kj�>�������+�%���h��#y9�b1]�v�HQ��'}] ����j���5��G+�D����e���(12����
�~���;tWiG<�gee ��v�����J��>X�~$�����m������\�������M���� ����L�0��W��H)��:"*�+!����v�dE[8��n�0��������'��mN��m�c��R���i�W���_��
��6 r��v������<��;3���g/w�Yr���#��� \����ce����6T{ �<�W�L������������k<��M63"���� ���&MA^E��� ��~��r���DT0��1�����J���-�_;4��������<FS'�^zj��� s��/S����������(R�r��f]����N��� )����~���j
����������{��gkq ��B�@�*=���k<$����[�WZu2Ys[�Y�6+[?2��e3����yK�������n����\�����!�X��� �����D�*�T�w�0�L�)F �$�b�d��������t� �u���x���m��u�������w� ����-�PM�������`����N�����Dv$��v�56�D�^r�'��+PQ �c��j� ��>� ��kOW�V���1_��D�*�������y���c3��0
�����E���gmN������(q�����&q�=�/�+�tQ��Zw\��U�� %/�a|����iC���S6���7�2 �H`<��������1�>.
B������\FV���KF��[z3m���.q���$�p���l�%�l>���g`���� gk} �oO�����2p��
:��I� �6��f�vv�3�%2Dd|��Zs�5�h�q�_ ���+�`;>*���G/yt�D5I�p� ��~�=��m�� �"��}|�.�I������s|������iP�|�}~A��%���+j�?����zT| e�Q��pUvm���� �>�2p�Hl�6�;��U>}���2���X%�#���6]�����q S����O������O�q�y%�3��r:�L�+���nt[bN�^,<�U�1�&����V�����a�?HD����� b��8K��rM�#w��J<k�p .Bh�
����4��C�%�Y�`G���;a{�:��'�|w���A.Q� +l�����Q���YAdJ�x�((:�t��f��D�tQ~w&�����+{O�5��5�Q���� �|��h,�<$�����[4y��Z �"�P.�N]�ztJc�( [��]�����dE���{�����]?����� ���y%e�G?z�����/����� ��in�csjFz�;�z���?5��O���W#�"�V8��������X�i�8e�\��'��?{w U�>p�=3�;����\��k5�)*�������~Vj�-h�)�mZju���$-5�����)�)�]�M������9�?�;�"������s�s�{�93�9�����n�Gvn����T��u[>���z�� w'[�sc_l�V���Tb����d�m>��:�����BY*��l��>I�����P�����on8�i����L�J��SyA�����/�Z�������$���GN�ke��,$a���G[���
��VK�$!�m�<����z.�MQ����G��P�V�F�����/ ���m�xg� ��fd����0dd�ZW����Y������:HZ�� ��+�F�-����W��R��T^�S^-u��*.zM����{L����h}��U?�o�SI�%=ZK��vPH�����7:f;J�@����>y��� V � ��/��S. ���� Y��r.�z��g��C�<������r��vR���^-s5N��T8�*L�� � �Gp�V���>��Fw��������� I�(W�\�����K\d� Y�f']���AAc����� ��� �S�
j��B8_���TV�+�oaB���l7�|=M�MZP��W�Y��';N�f� &#��^j��cQ��M�E����$�����B��������n� ����#���lw1 `lqI�}��eiw��Z�T]����_{�����F*�\���v�y����)�WX>���T+W/���o��E����tT������1��v���O�����J��E�5�k�l��TOB�,n�lII��^�,zdM: ���wf��!Pe��s8�����jg��Z���Q%Y�Hig����O����L�?|<�����5!$�������!D�Nq��U��#$��S�
B,��;����J<Z�lwn���7�+*<$U��K�4�F�r�Q�:��3�� ��k�Y�k�H!����J+�+���(Fwr�s��� �
������g�X�F���Q���G��32zjg�����5$? ht��V�;�($�B�
!�I�!J* �>���S�
�
G����g>�����g>]�o�����&�1{Ip��������tn���������O�()N�+��)�l��� �V��sT�U��d���� ��7o����c�3e��
���{�ZW�����-��TB�V.��y��)<U���:�d���K�,dIHB�5��YqS.���(�K����<?�a�Tl�$����I����|�����9�l���N�Nw������+s�����F�!���tI���������%�3Y ��7o�a��m�eH����2������V�^swk���c���I�����5 �aU�}j!D��~���� ?VZ^�����h�����Z�CS�Tk��,����s�BB�����$&t����SN�/��s���V��sh����� I���a��������U�k� !Dn����k�g����:9��.�����v �"�2�o��&-C�t������)����K}��G�:9z*C��
!rU��]����&�0$S\��r��VsY4:�E
� �Q��l�`&}
Mf����18HH��A�%nw� ��W�%�_�K�WZ�i*��k������HI��p�9/I�fw�9 �Zl���}���w����*8Sn����&�����N�!g�, ����*���v>���Jym�?����Z}�
I ��� ��\*��e�������o��]�n<�q���x��M$Ix8�.�.��
W'�{��]�+���n�X�Zj4t����ve������1�m� 4t�����2�o�6�96���%[.mySK�\�2���4�s.������jK'K��u�����/� ,��x�`��32�$?�����2!��g��#O ! ���?���h��Y��7�+��|E<��j�����J- [���`gN%�'�=��gB�S��;Fz;��~�['+Ko�H4��?�<�"�X�����L���KF�z'����*��6�|�����,�+�����+�8~.K�F�z�V{�l�"����o@� �-#){��R��r]�\t�|��O}Y�-3��4����I�T��e]���p����.�7��k����� M��X��f�������j����=er��/c�;�;��������^B X��`OL���)��2����"��|oY|x��fE��-�/�+uB�QAiB��}5Z��-(�
�7� ���G�8}Kk_�3WU���9]-N]9��,K��Nf��{�� 4���5B ��-���/�Y+��Z���V}��V:K��_{������%��$_d�$���wR�Lz������[Rdy��s��Tt���~��#���>�D�� IDATE�u�o�)���h��[�+�Z� `ol��;��`O<M �z8:y+�_��9�� �5-�)dY!�s1��W���'n8�y���K_,T;;+��V�� RK��� 9�u=��-��v � %M�c�{~�<���aq �~�| �����{'fV09�+ �k.d��'?��� ������[���yI��';N��
��RWg�R!kuw�cQH:�$*t���trT�i�
�K ��^���� I�(W��7%Gw��& ��"){�1�[�r�����
w����:��B�.?v}�\�$Yw�y������Vk�R�����
�B�u���jE]��p���_cC�k����r�����]*V�, �m�nL�^ `y$e`g:���8���E���C2��<�"w^������+9H�W],���euYI��Tu�F�U�Q�5$��p�9/I�f�����V� )k4���k�g� +o����C�6�;�} ����AC��s���a���oe5v��7;��*��E&�}e��O�n��z�� �$O�1P�1K
�J�����B�7�:]R��/\�z8��F�B\*,}v�1I�#�Q\ ��~��v (����c-���a���&���9ozr���$�V^�+%�y~���� �6������[�L����� N9���'��j���e�32z%j���i�
`n$e�H�$I���w���4}�����mn��,.�x���{���sUv��� ON�$��������
���V�Z ��������E�v������y*3 ��2h� I �<�X��9�S����9���2�d!*�� �^�v�m8���g��R�C�t����c�8;��<J#��CFE��T�vQJ~NWMNo�h� �#$e�<�un���\� �����YT�nK* _��P�U��uu,(#;J�5i�w��|F��?��Wq�3U:�3��v,)�8/trS���|m�� X�$��X�q�$����yvh�KiM��p���I�� ��
t���i��%��7E�B�� ���\r�y����[V�v���
m�����XY�r���3��8c�� �����wf��!�s��L�������$2*�ie�,$IJI�XY�*�O���������Y`Z.��qFF��, ��W�.qf� `f$e�h�k�n�Uh�H�f��e�HYX*;]�z����\���,���n
�����T����r�vc�/ �H���U������ �����s���� ]���t.�M��E���7�/9)�N�
�����^�� �l� {��k�� �q�2��(+��Yq�g`�xu����u�LJ��C�R���t�$Lr5E:���:�N:�UX���2P�����]9�& �.&s�!���,� �j�� fFRvO��V��n��� W ��b���4����Xe~�������������b��G���e���w�<yY���c��y��`�� �BRv/������&G.8����]�*��B4q(������7E��M?/8���%���&���������#?�t���6���( 07�2�o���n>�|��.�+o���b��B�sVTh�*j6���l���_���j��AhGy�}���B�����B��K hxH����9�|��W���;�Uy��Aq?��T����xN����e�*w���98H�y�&��#��l�hCA_�P
!4B���oO������QeT�U�oYU�� �5�����*B<�SR9#��d�i��.zUh
��/)�b!��FV
��� +�~p�ba�X��/-�<F��^E��uG2��t�|��B�;��zv��/�d�=v
WJJJxx�#�<b�@@Rv.�����yZ]��k���%���&�PJ��R���pI����KB\,obr������{V�9�;��=�D}�bN�Z3w{�YC `��}���G��������[��; �S�� mO�;�������|�\���| �-���`�U��d�jheiYx���4!��*�&�a]�Z3�����X�U4ti��3��LZ2*� @ii�Z��v79��`��5%:!D�����^��1���q��2�o�NM�Dy���=���}, O��>�kTP���~�S���q�j2J����\�8�j+���r����%����a ���;����_���������� }��K��M�v��i___77��={N�0������ �?�|ZZZrr���� A��O7����s��UOO�����3gZ���2��>#��)��2�
Ic$e`g����-)KU����R�,}������|��A�jg�GW��"O������wN�i\MF��� ���j�X��UY����N7�aX������1 �����m���_���������BL�8��W_
�$���(,,l�������Z���u��y�"""BBB*w{���q��-_��c��B��;w��� x;���-�-w9j����K?�������*t���y��|�c�8���k�/�{���*AV��(*=pPs�J�j2B�Z�h,�pQJ����qd���#;��usT*���- �yL�*� h /^�������k�W�RH�$����\�lY�v�j�m\\�������! ��{�z
�j ��U>=�]���0S�d�������t��8+^�.��0�(�kL������s���x�����B�uQ�������uIc���l �n��S'!DvvvLL�������f��X�"66�����5
�������I��������q��a�v��Y��W����{'fV09�+ �kn����2�'����[����RK��Kj���[*���\dr����5(�z�_��b]L�d�
&��O�V���Z/4 �E��DGG/]��q��&���o�f�!DFFFRR��a����;w�|�>KJJLZ �g��c�������%Y���!��u CA�����=���e���pW���\�����9Ycr������s�Z�hZ.\��s�D4y����
GW'�n ��mc�� ��Y�f� =�k��B�J��� �(...((���BN�8�c�� �-��(I�N����������.]�Z�����N�Bt ��q�
�:�5���=��6S���sW�9TQ&������U<��b$I6~���I�mu��r���:|2�O�Q?�t�� �����QJ ���i�J�����o =z411Q�������O��WA�V'$$����p������������|���]�v�w���M�2�����������yG� 3e`7�^����&O(��8R�V�j�EH� ���cIYHBY����%iO��Em����}���5����������GfkUYJ7�n�]���V� �H��FGG��� :T����;|�p�kWW��]������/_NOO/++���|��' ��1����=z����wvvx�����Z�j�f���^{�������c���-j����Q����,�Na �l� I� ZB~~~���7�i���6n���\"�r����'f��Y6� �6�9����q�$����D�B!��v �|����%��������'9�{��pm i�� ���� �,_� ��� 0`��?���e!=�Zru������B�/% �Fh49���!4Bi��B�F3wd!d�|>O�r��"�B��Q�,���o
�)W����l� �`�l]AA��� z�!}�*7Y.+�L�^��M\<� ]� ! ��_\�$�B�o� ��\��&[c��U�\] 3�{�F�a�� CR6M����������oqt���������!6�?sq��ed$��k0+�bES!��{��# X ��`�
���{���x������#\?�d��gT:z���� t�~�Q������9]� �=��Y 0 ������*))6l������Li����;��� ���X$�����F���y]�r/�rI!I��, ��n���-�/}��n�� X�- �/�Y������ 2�c����/�)�Q��o:h
�
!�?�`[I!$�F��r�� �w��t2�����ePP�o��|���$7g�n ���A��
�m�x�� l�J�:th� �-[v���BS|AX{���T��P]�z��y�V���s�������^��s,]��
!J���N���$%�
�H� kb�l�>#��]�+V(U|?K�s6�������miz��_Q�y���l�7H��T���\��t� Y\Vn�r0��>#���j��P�! l3e`CT*��a���i�r��[dd� zV�(Bh�� zV��,��W�7MF��5���_����r��8����*L[��- ���2������ \�jU�!Dq� }FFO�()N��M�?���V_���&\*k��t}SK/g���Z
I��Z=j��&M��^��V!��$��eCf`L�s������W�MZ� ���4l�:*�� �*�X�"""�O�>���J�;�3gN���SRR�10�IX_yy�c�=����v�Z�RY��n�&- ��Q���YEy%�5e"�� u_7G������Q���`�� ���'&''/_���=deeeddDGG�cT�KL?�v��53�nX��~�b�������32...�����Bzt��=8Q����I�t�X���1�������c 3�?������_o���K?����%�B���y� �(�"�}7�\�.���5����5���i��u���!������p`���������Z8�_P7�@�5�T\��,���K�4��^ ����������,����OMM
�Q ��?�puu�����B�s$Y���+�(�{tIc$e`5�?����C
32B��N�\q��x���O���g�lg���K�����<�9��cj��'^;;(�^h ����m���_�������8##�X�`�G }d��2U&-*�:�w9j��:�Z��q�
���� o��"��R�I�.`�����;�ZrP*���F�9�[���o��Y�8�b ������/RRR���bcc������K�.M�Zt�A���IK��}<��b�)+�j�O>�daa����k��B4vU^.�)/���8�>Z��� cm� </��q �b XF�N�����111�7o6~��N����-VJ� �����6V0�;:�u��p 6�� ,M��FEE��������wVce�=�+�(*�_�����<�9l�0�X�&no��� K�h4���K�.m���'� ?~���c�F��o�>}z���_|�E�6m�Od�n�$f����*�w���}��v]�zE�CR��j�z��k���"##�x��E��Wi%ye:_��N�}�\J�+�� qwv� �d�( ��f��5z���]�
!T*Unnn@@��s�����3 6������;#�7�]�����{K$e`9Z����������e��K-K�������|�|�C�OB���]�
��eJ �f��M*�***J�y�����1��Q�z�,[�m� ]��B��;w��y�����t����K�.m������ {V�� K�%!������G��d ���_��{ 333�����{��5k�B��������899����� >|���3�;9���/������Y3 ,_��y��<��I���$�������s��8q"))����~;o1o�������J�
�|�P \ P�l��;������<i��������m�{F&�����w;�lm;w�oYB��:���0�B��$�[�m��>'� �r���y��<y��?��#99�������;����?J�!��<����I����S �
wg��������I'�
�|\
�Z�Fk�K)_ @=���;��6g@�Y��L����������Y�;����U�M-M�2�����NqsT��38���������eX��5)��[�>A���Ry/ �^���w[������j dY~���������KFF�2c�I���A��r��L����F�?�[�~s���+R�k�>/��� ��6�SSf!����S >\_sd�}L � ������t!�$DS7���w1N�|���5���������Q�h���|dg22 ���D�}��d����9s����v������n�<�9q���2�h� �� ��� ���}�����^{���������Q�F��9� ���� ���}�����^��m������ j�6��Hl����go���� �ER���7�����{����[; l O_B��;w��
v����Ik�
JBBBHH����k�Cqq����:d������������cx�5�2������_�w�����[; hh����,YR��eY 7n\DD���[��� ��s��cx���b�����&�L�{���e��� !)��Z�h���~�g��-ZX; ����� x ,,L�9h� I�rrr�}����V +�Qi5:9G�]u��@y��PSu�����\�r���-[��v, �����/�/.((�h4O<�Dtt�$I�]�;w������ey������AAAB�������k��h�Z�Fc�h�:]R�����2���t���.��� !)��[�x�������g� 39z�hvv��U����5��y��{��3f��l��M�:>>�g����L�����Y�_�sK�&-�*���X��ZZ�dI||���{�� � ;vl������B���� ~����k���/��HIIIJJ�����������������n��4iiT��.GR��t���K���������� �]!<<�����)IRXXXjj�~�S�NB��������>�L��"�q���� |�����x�=�.J�����F��n����/���\�r��%{��m����c ����|��B��jo,�h4���K�.m��q����93v��U�Vu���������.�$�J+�+���(Fwr��GA������Y�j�� �� ��%''��j�dY�w�����O�5k�����v�*�P�T����]'O�|��g-��� �s %sk,_� X�z���������m[k� w�I�&�T*!�V��3gNhhh�&M�{7m��R������G� MLL��>~��������+CF��������-�`J�e��G��$�n���~�ill��={��oo�X �.�v�����A����c���eee�f��Q�Gbk�Z__���'''�)������9s���W^9x�`�F� �����Vy;Va�?�m1&�b��k���������{�n Ou ��l��;5ep{k���5k ��-&���m&����_=}���;w[; j�6�S����a����w��AF ����%�����N��c��{����� ��0SU��i��/��}�������� ���2��7�|3i����������� �0�b��b����b��m�����uk���� ��6��| 7IJJ?~���OF �")����������������� �����u;v�x��������={Z; >�2B��;wFEEm���� �v, �H�@���+22������?�a�X
Gizb�����r6�-MO�v86����,�\s?���c�=���_�����v, ���4}]����+J�����w�
���V �6��bL��6?�:p����#����� ~��� j����111���������Y�tiTT�K/�$���������HY�������d��r���)<k��������f������wk �����F� �~�z22 `�.]�����~��g���:�n��K�.]�h�c�=�? `��m�����={�4wFF�)�4mQe������2w���1b��u}���v, �:�������
�b������3>`����)))III������-���=���#$e�F� z��G���g�X u��� l��$i��A�t��I��� ��g�)�J D��%Vrt����G�8 \���������o�>����~��k� ����&-j���E��DGG/]��q������}�w�� m$���G�^�]������5e�.G� :t��U�� b�X ��}��)))!!!�q� IDAT�M�N�e���1�f�=zt��]�*�*777 ����\��um7��W�_���� =zt��!+V�:t��c ��7�x���_>��B����1������M�6�T���(����G- 'L�����m>U�����A�-_�|����� P�N�> �V� F� )�������KZ����7$$���Ipnn����g��i��-�6��bL��6?W���{DDD||�#�<b�X f�h�����/<<����
����������c����/&# w�����[��������6(����<yr����/~��'� ��&N�x��E!�$I�������� n�g����������{�EFFZ; ��6��|��JKK����;��CF DR�a:u�T����z��'�|��� �*��i�N�>��� �7��� ����ih��9��_��_=**��� �[��:7���j���_}�����y���� ���~�0S���p�B�~�^y�22 �>�2
�� �����K/M�4��� ��#)�ddd���o����'O�v, �F � �*33�o��'N�2e��c ��
�����d�UI���3�����#�-�X���X�V����������O��1�Z1 PYE� U�B.��-���Nwt
�J0�E=�r�J�~������ ���t������u�6l��7�|3++K���W^
JOO ;v���C����k�.�s��_ 9x�� |p���g����'$$�����?��������_�r�>�P�^�J��W������7O����'#""�4i�m�6������ndd�riY��^���b���X+������o��c���5��W Ti��a���jhh�$IEEEaaa+V�����~o�^��w��h�"����a��m���i�����N�j�����������?�|RR�~WJJ�������?��Cwww�V���oJ��O�T�7//o��{���$I��������S���Vf
�E�q�$�����z�[a� �Mvvv�~���CF l�J�
��?<==�-[��];�^gg�>����E���=b�����v��Q��B�i�F������ ���.�P*����;|�����o�����g����rV�^=a����$e�J-M�}Q�B�_����6z����g[; �MV�X{���F����FFFzyy����8996=<<�j�a����Z�j�$I������'O�4�9""��\I�������a���v��i��y��!~��������������\��R�-6�)�� m����2vF��y������c�X ���o�f�!DFFFRR��a����;w�|�7l���?|�����.B����T*��)*��pp�{�������={����~�������1G�07Y���K��,p];��%{���7h������s�Z; ����bCY�����'.X� !!�&��_�~����4Je[�n-..6lj�����w���&{������E���9���c�bcn�n�=�'z�m�l����+#)c7��� ��o���{��� �p������
������� =z���F�����_���;���3>�C�&L(**BTTTL�6��G �����^�J�r��i��M��E�������2�����P��1 ������m�6Y�/_����^VV9~�x!Dyy���c������������?N�<���i�������k��L�RXX�P(������&L�����`�����?%%e��]=z����O�Z�F����;v�����5��o��e���_o��bl��K��}���ZPP0p��>}�|��f� �6��.�z�K�{
^{��Q�F�i�{�fR��K���� <<�w��dd �VTT���~wfdl�-&���Y�m������<���%K�O� �m��] ���{��g����P5�+��SO=������]RR��C���[6|�`�3el1&�b�����d����:uZ�|9 j�6�2,_�Q*�j��� ;v$# @�DR��32���## @CER���T�a���i�f���
��mKii����[�j�j�*22 4`�X����c� �Z=b�OO���D�RY/}
��:���#G����X�v- <�26A��qssKLLtpp�v8 ��H�X_yy��Q�\\\���K22 `�BBB���o���X�"""�O�>������Y�hQ�.]�Z�%C5H�XYEE���?���@F �ZTT��%K������{���[��8qbrr������!88�o���|�L�oYm��v���v�����,v]{A����B�~�zGGGk� �]���j��.= <x�����m�;����?J�!��<����I� �-�b�>�2V��j����j������ @���/�,^����@��<������$ !rss'L�������o���ddd���|��>>>B����9s��8qB�����ot��Uq�����>---99Y�b������5 )>>~��-EEE�z�Z�h�Y�v%s���32z%j���i$e�����K�����j���������:;;�#0 �����,\��������vww�h4�������1c����?�X�R���s����������7 ,// <x��3 ���� ?~������W/�������� �&��������J�T����������[� e����M�Hl\��j�z�������7�� �����c .twwB888��������]3>��/�X�fMpp��K�.���B�e��EEE�32B�f��%$$���Y� ��@ ���J-w9�2���j�~�����o������# 4(������.IRXXXjj��1o�������� ����e���Q#OO���b�EkV�����oTMqwv� �d�xl5e,J��EGG_�re��-dd �����1�P(L�B]�!�,�59��Dv �eiv����2/�����t��vP������t����_�t����suu�v8 ������V�
�edY��s��O>y� |�������������<ooo��$I:��g������v'sK,_�Y�'M�����y�f22 �PyxxL�4I�R !�Z��9sBCC�4ir��{���?�|��������q����=�p������������|���]�v��M�Bl���}�IgY�'O��� $%%yxxX&0 �%�]�6>>~��A=z�X�zuYY�F�5j��������/_~��977���� !���� z(**�={���g�����g�����a�N��5k�� ;������^~�e}��_�566VQXX����/!��w�Y�f�v�]�6��d�1���~��,O�2���OJJ����X` @�6�2��1/Y�_x�������d22 ������<u��#G��� &H�����3���]�vyyyY; `[x������k;w���s�����cP?���g� X� �{0�X�-���/U�
�5kVRR����5jd�� ��m�P�������=ls�3S����=;))i��]dd ��PS����3������{w���� �]���Os������v��EF�V�n��u�������[�;���v���/vw{�x�����a]z`������w w�xo0���6��^��������r���TYKIIIRR��M�:t�W�� ��bcc��_�w��-Z�3L�f����a��-p���D��V�Njq���bw_`�������{��_��������}{������� 2� ��]
{���%��/�wz�m~��)s�,�S�N�p���O?]TTt��/\�p��u{��!# j��2�I��j�*!���'�������U���k���B
3e����+�� �;�L�z������g�(��$I���X7` \�~/Q/����Z�x��������v5��^�������`��nw�^���;�x�p�����nW� ��~ ���|].g#H���
���K V@R �
H� XI +���u:���'�,,,,..������9#��5kV���� h�$ `y,_ ��2 V@R �
H� XI + ) `$e ��� ��� ��2 V@R�~���l��q��1s���v, ��m��M������_�^x��������:p�?`��:���Yqt���')SdY�:u�� �~����"k��>�v� 8p 11q�� {��y��_|�E� �v�8���-3� g�Vd��m��/���8qb������Y�j��1"33��9l��K�.Y0. ��v���s p�?` ,?��1��) u�V����
����?r���P�� � ���e�3�[f��m��OR jO�V;99��j������V< �Q������ p�?`��7��4�I� @�UTT�������V���zT� g�v�.���2��n3�}�2 P{���&�����a
�NU?���]�� g���|��Lc�� ����sEE�qKfff�����zT� g�v�.���2��n3�}�2 P'�.]2l����k��V�@=�~�3��V���l��F�9�>I ���^z��_�j�B�o���E�~~~�
@��~�3��V���l��F�9��C ��B��=�������������3g��f������CPW� �=����� �T*;t���G Y5^ w�����2�p�?`EV ����,�u� w��K V@R �
H� XI + ) `$e ��� ��� ��2 V@R �
H� XI + ) `$e ��� ��� ��2 V@R �
H� XI + ) `$e v&!!!$$d���u�g��E]�t�j�&�+V��������+��R�K��uq��{'KKK�� __���[�y��
6���'O�l��uDD�� |���e ��XHII ��Gjx���'
d�?U ��� ��DEE-Y�������W�0�O�������/_^�Y������G�q]�)���O?��C���[����������_W��u ��G MNN�2e���X�qT���/$$d���5?>((())�v�/ �as�v X���� |�g������o�����x'+**>��� ��*W7���=p��/��2::��W����X �� �^���/�/.((�h4O<�Dtt�$I.\x���233� ;&�������:{��� k
����l�RTT��W�E���r.\x���������}||����O�a�\��W^�����m�6g���������_���5�Y��,Z��� ��t:���G -..6,�(..�3g��'����o��F��]��������cbb��_���������� ���0`@\\\BB�'�|2t���={����.//�e��� {��g$IB���N�0!//��7� ������pqq���/�����3o��FAAAyy�����Y��w�n����K��M;}�������[��='L����_�����z�u���6m���G��M����WB��g����;88������g���QQQ/���m/��sg???�kY�������d8���� =zt=&e��n����8�������j7D
����111W�^��� �9sf�c ��d ���C�F��� /��\QQ�������;������_�r����M:���?^~��[]�V{���{������CBB&O�\ZZ*�r~~~hhhvvv
�|���u:�,�� ��:u��R��aaa;v��o^�re��! �o���.Z�H���]��E�����N�:t����eY>t�PHH������Y�����s��5��G }���������?��e����.\�e����}��=w�������!C������� ��~}����={�<z�h
�Vs'��u��S�LY�d�qK������aaa�;��j�|����[o���&�H��A�^��'�����Z�������kw�F�N�81u�T�������.{���o����X���n���<���iii�� ;vDGG��_��I�5� ����-� @q���� ��� ! ���~���k��Y;��svv���\\\����#F�8|�pMN<t�P���###��2�J�K/����l������7�5k���`��������/K���wo�DI� x��K�.���������g�R9o����_�|�8�/��b��5���B�.]�
!�x���>��M�6�c���,X`8K�R�c���\�lY�v�j����_Q��l������s�Ic5�*..��?���B1{��� ����_���$%%���� �P(:t����v�o���n�O5�~�����Q�w#..��������~s����s�� �j�| `�������
��$���������Y+�w�yg��}&������� �=�����������Q��5� 4h�q�$I�Z� X�r���F�yzz{xx�H�d���F!DDD�qT�$���������
34���[��5�� �<y�I��S� �W�X{���F����FFFzyy�po5������_�`�X������'� �L>��u��I��� �y�f�RY�����s�������������a>�|����2D���s��u�����a������&1 P
�2 �T���B����ug��iRf�2�ju5���e�.�J�RUn�O1���z�������*�z���_�f�"###))i��a����;w���j����P�V����L���^�������L�F����^�ti����< 33�0��U��W��2�X())1i1��m� T��K ����l��V���;w�)���B�e��������$I:���z�={���c�I�J������~��w��
������k����[���
�Z�v���&S���G�j�\\\���� 8q�� $$$�do���~k(((�x��^5��}��)))�]:�n��-wt�Y�f� =Z_�V�Reff���tg������~�o�U��W��2�XHMM5nINN�a� T�� �.yxxL�4I?7A����3'44�I�&��^^^����~�g�}�j��������������'���v��U_��NXX����q�FC���;#""�����=�����w�^�����q�������'$� ���Bt�PEu�C�Tt����]2,
����L
�K�)� B��!� �P�A�@P�,!E�A+Vs�wy�6�����~n��������;�1��l���bhh(�B^^^&&&:::~{G�MF�qee����A��� �E��?99����q���0L6���8�,544��D#2Wsss�! Bx�����fG[�������V�����3���yA (//��e������{���b����P�ONN2D��� ���$��D ����Z[[kmme��pp �L&{zz�����eu:�T*�������n�; �RyzzJ��F����k��C�R�L&B�x-!$�J�L&��SRR"��JKK
M�*N��D"��h�N�\.?>>��t������6�q��b��|������������bZ���f3�=���g6�i�������U�Z��������Vk__��j�x<.�+� 0 c��x�O&�Z�V���fwvv������
�w)���T�OOO����TJ&������z�BA9??�����������������A�A��w������yNK$���^�7 �z�"sEaY�h4��q�T���%�\.���~wD<�������<<<�����rccc����!S0�����+$��q{{���H�V���uE>�,d�6B����L,+,,������U(MMMn�;��������� => ~6e �m4������ KKK�T* ��\-//������|�'�hT�R�|��$� �:(��A �IDAT�2 ��/ |��������������w��D�A���P���mmm6��[�� ��������� �s�� ��n���������|��������H$B�H$�����d �� A � ��% � @P � �e r����I�6� IEND�B`�data-sorted-fits-into-ram.pngimage/png; name=data-sorted-fits-into-ram.pngDownload
�PNG
IHDR � � �W�3 bKGD � � ����� IDATx���w`����K��t�P@� ��Ld|�-��,���� P�D��~YA@���RF�Z'[,��
����������4I�K��.����~ry2�z���<�JE ���; ���� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ��� � H� ���w��I�( �KpR �H��K�^��z��� �8���U�V}���K4*�[�_�>,,������[%�,n;NEQ<r����'��7QEQ� ����{�� &p�x�9s����U�P��� 9rd��1� ��l)'UO{��i��� � ����o��U����v����?sl")�y233'O�\�N��5k>��c��-����M�����&M�T�X�\�raU�Re�����/8 ~�a__�����5k���������(++k��-Z��Y�f��M'O������;��������!!!�=��Q���FAAA���s�mIOO����n��)q������ };o��������
�S�� /�p��m����}{��i��5�Q�F� ������M�]�bE������6n�h����z����r���� d�b�������+��������� ;v {�� |��_~�E���� |�l�����g�������Y�f@@@�5N�<)H> %~� =��{�����?_�~���iv8w�:�����Z�z������?v��.]�4o��b��-[����j����[_�|��C2i��������?�:t����U�^�4�G� ��������T"�{���x��'� ?n>�O�>~~~>���e�L�� ����:��B��ZN�6����e��u���?�|� L?z��R"N� �}�����j1w������[ �"�>$�rgd�l�2c�?������ ���?8t�^������~ &$$,\����>�p����3 h ���P( ���{�5�5kVVV�(����S�L
������}���3f��xljj��!C,6��{�A���T��N�>��a�?������(����[�.**�|��7o�4��5k����������S����#F�HII� �}w��qh����{�����|CDQl��iFFFa����g����Y3�Ng���~��~��s���r���`8{��������333Mcf��5e���-[ �F�N1f��u��9�*PBf���z�j��z���o�n�q���/�������{ >|�|��P��M��A��������q*]�6m���[������P����N���+Z��JZ�5��\^C�IU�|�������~���V�Z���[<�k�S�,�f0RRR���s��i�a�|����C�����v�����N<�X���N�� ~��J�*������ gK�8�z�����={���r��R�-����H�32�y�f���m���Kw�{s�
��0;;;""���k�-{������s�`3e<�������^y���� A*T�0o���-[� ����?b��E�M�6�r��� h����zj���w��1
+[�l�r�l��B�
AAA��!88�8/i�����������k��6���a��}��'�?���>�R�""">�����X����S�N�J� 8`��n������/_�}����������+�6m�w������/W�V�=a������;wn����G����[o���bX�2e|}}��!��jA�!�����Y������w <x�����8U�T :tx��WW�Ze���g�MKK��m�i�� ��y�-Z�Iaa��Y�������;�pR�,!!!O=�����]����-���H���� /�]~%� ��cG�v�*U�d���C��/���;1 v���0���k�����J���/>����=����iii� ��[w��a� ���}��7���7��k �����N���4h��=��;�Y�f��s���
��5i����_~��g��5k��V�Z�^�� =��s�t��u�g��9!!!���j��|n��U��������S����O'N�0�8m��9s��-���E�&O��h�/��B����.]:r����������u��o�i��I���?��c��G� ���_��u ~������ ���Z����#}��
�^�z���?���>��z��a�������y�����j��5h���?4_z`�o���]���Q�I�&����~����#�T�b�8��/�H����cD��+W� ;�4C��IGg��[1�V��{�FGG���FFF0���S�c�^�:n���
���5o���7����� V�q��O �JU�F���{��q�����W���{���
> ++���k���k��~���8l�� x`����
3�� �|����:u���f,�����?���+"""22��'���s��'���k�����b�X>s�L�C�� ��~��I9��x���-[�l��xQ�u���]�V�Vm���5;��L :�Z���)Sf���vs�q�������@�-j���O?}��n��!��s�V�\���o:���ZJ!��$H����� W�\y��i������O��������c\}\�}�����������m����a�����m���� ��gK�8�z�Iu���+W�tb e��s��r�#��CF.�tJ�J�3g�X$�T*U�-��=��0#��s�#�, 8a��]�Z�����?�|f`BB�;��S�HG�/���_~��E4���jg��(���y�- 8������������/X����������3���{���f���6�|��y .4��8k�`04o��8/}��=�{��z�g���f�����?��i�����iii�������i���+�7o�o����<��p�����'��������������W��qcff�q���\�sI ����v������� �?��s����3]�.]��?�����sg��u��m���� N��)~���G#�b�v�.\8z�h��
40������b~���M���/Z��[�n�� ��������}{��������/���w��E���k����5�~��ElR���={~����|��g����s�b��M��W��'V�^��_� ,��u��U�_����b�2�^����O?m���w�}������������� :����9�|��!����[$ Y�������a�������'0������K�,����L�\��J<��|�6�u�F��v�����k�S�,�����m���E��/Z���O� ���;v��������3q��R"�_$�>�n���C���� �(����Y�[�6�@-^�/��R�^�]�v��s��%�k���+�8[J�I�|��O�yyy������������[ '������K�^����1q�
���=��s��m������Z,"�8�H��\����<�-�_����S7n�x��%�kQ?��� x """""�b��o��va{���)ro�zN��%e.^�����ZT�����_���I)o�����`�~�zrrr�N�F����Mc4h`������l��7-%]�n��q�DQ|����W��%e���k�wvvv���MD�M��~�z��O�>��I����������m�Li��c�Z�)�8�ZFFFTT���s����_������.]*�yR��/A�����Y�f�(��u�Y�f7o� 7 z���G 1 �u��r��E`��q��M��$j��q� ����7�?�n���n�2������a�#G�X<v������7�(�8�����5kf~����M�65�'w��{�_~���-[>��c������x�z��w[�n�i��b�r� ��{���z���>�;w���Y����G�)�*U�����6�`�.�L%�TE����� Q }��7n������D�OZ�fM�V��h
�������� �0a��e�U_KG�E�f��C��[��o���^�u���N��i|���mk��������tqqq}����+�9[J�I��#N���5{���l�J:��-E �<N�}��I��Kw&e�� k�8E��7�X�r��L$ ��
��<������.]� :t����n��������=��sg��={���Y����E�M�R ��?QI��uk\\�E������S�:�O�o�G }d�� 6h���_~y���j����h�I��������-6���/%%�����r�J��m���������B�
+V4�X�LSH��m?~�<`��_|q��5� wtttvv����-Z���l��d��J &������z�����7o�|��as�����e���'Z��x��g,��%HQ�re�����K�.<��C=t���������5���;[P���������C{���W_�h4�[j����KS/� ~�!**�q�� 1b��������?J?N {��J�*%%%��l���]�v���� ����������5k�L�4���sN�M��������o��.f��o��-[�{�9�����/����(��2�>}���3U*�� ��3u��*��������Oz���?��#11q��E�?���H�Z�|��]�v��h(9.<����E�����Y�fZ��|KXX�����>����_��gO��u� ���+vV1{��R"N����Z�bE��7�x�,��G�w &��t��F��nb���gq*�8��%���!)������u���{����k��}������/ ��E������o�m������ �[�z�~��w�/��N�8Q�^=��
4(����k'N<�����/]���O�/������A���oG� ��2���3fL�^��L�R�_bW9y��k��f�a��� dgg�F%''���K�����{��#l���2L�Q�F�X�b��q;v�HII�>}�c�=f [���-6j4���/A
�Z�V����cccA������;]R����\�����IT��e��-����n��J��[�����C��k��f��5ADQ�7o� /��\��o����7�R�v����_��� ���o����m���<���(��2�����OW}��T =dL ���w����\{�J�V��V���s�����8qb��=�cj��Q�B��w�I���/��? �u�������v��!e�����~������(�I����9[
E��������Q��y��r��U�r�������� ~��a����'���<x����+���w��-�=�a������������j��/_���? �[����n�j��(eT�f�������~�������
��� X�l�V������S�+VX����k��������{��w����t�5l�p��� a]>��G 1b�����o�������`�ki����o���i�����W�X100�^�z��C���� ;f�Q�?���
/�H=�Pzz��?�hln����������s{����e�5�s�t���Z�j���q��U�����{�
������I���P'�a��ImQ��W�������[���O���~s�� =��p�g��IU�|�������]^��8u�F�����K�C��N�RH�����e��]�6))���>�9+�B���333 ����-%����s�~R�~�z�*U�x�"9w��x����;wFR�����j�����?�oE�� ���pb� )�I����n�:�Z�;w�,���QNN�s���[�v����>��S�1:�n��������O>�d�S[b����E��3 �J5{��]�v����6�5���~2�b4����(�������}�Y7��|��'V�Zes:��A���{�|K�r��w�nqi"e��'�7on���"�"��O>������V��/��(���� Qhh���;4h���#BTT����~������8���9�7;�}�]����x�bRRRtt����-[���/��e������H�4G���_����g�~������(??����~���k�rrr� ?nl
��ys��)�����u�����s�q����H�����������[�����s������[``������gq J?� ���������C&77w��I�����x\{�Jt����v����$��p�Z�V�T*T��s���L��M�t�R��2"""**��''U��)����������D ���oy�|�a�m��.���b�kNMM�Z�����a&���
r��9�#)�aBBB�y��m����ggg/X����?�1c����T�%K�l����g�5�38x�`tt�<��iSc�S���I�:w���O? ��p���^�z=���n�D��&$$� =�4���g���c��������� .L�0!))��>psx ~����z+66v��m�?��?�/��B����G����� |`�������q��+,��I��]���W��)??���~�:u�g�}f[DD���Scbb�5q���/^<k��n����K�(44t�����K� ����m��[���I�"Y�JBTTT�^��2������bcc���g�� �b�����������n�z���>������~ =N��i�V��y� cm������� ;._���g��e�N�2�X|�8s��]��������s��)� ���o��yyy� �6m��5k��H��������K111 ���z�����{�����e�m��� >##c����������~d9�%�iqN�E 2�(�8q�C�]�vm���s:��1 ����}��x�����0��Z���? �1�e-Zd?/��s���O���_6�����K���;7f������pR�xF��T >�����M���Z���Rny�|�� _~��'�x��?�4N~5j�;����0�={�t���� ,��=J���k������������k��i�Z����7a��v��������\�����9???S]��s��5���Z �\���/����� ��+�t���O ��I����O���c���y��xC�z}�V��{ 3!�:��~��� =z���W�^�g��G� 9~�����Z�,�(�a����?����a1l��Y���<y�C=�z�j���7oN�:5***<<�}��s��5o����/k�FDD�Z����������(W�\�������7m��
*T�T����7n���i�������3,�v�����������?������K�.�h�1j���'O� ;�W�^�k��[�n\\��3g,�Z����w�i��V�Zu����������M���a����:u
����v���#�?M����~W�\1m����A���_M[F�����+W�n���'����j��P�Z������@��7Gg���_ ���{��� ;v�n��Z�j������?y���������G?��#��5{��7l6aq�8MIIQ��6�� �$$$�z��N\���6Oq�a��Ae������<��a����2eJ�:u"##���
�|�r��1���?�����?^�V�*U��m�v�� �� �nB�����I����-Z�������C� �Z���X����_5h� """88�J�*�/�� ~X��u�gj����!���f��v�����7 ���T� ���������C��nB<v�X� ��w�����O ��ZI�I��������QQQ���7n���w``����x�T�|y������� zH�RY�12����[t��������,z 'U��T����/� ����(���D��G���-_������#22���~2p��������Jt� \�fM���k����cG;���a�/_!))�����JTd pB���g��Q�9���}�������[����t�&M��*n�'N�x��W�\�6��L�����7h�����+j�B�G�3�<#��2�����E~�7n� ;r�Hxx�;C���_��1��?�p���W��; p�[�n >|�f�
x�Y�f��=�U{�h4%]�Er�g
����35x��\�r����]�v����7o.��ZN(����o_�{cY���E~� ={�$#���o������X�� �� �o����O�_|��&�8����V�5� u ___�]��6.�L�X�����K#1w� *�j��yIIIr��"��;w������'U�I9������7�p[H��K��_�S�X��S�={c�� IDAT�{��yyy*�*22r��1����;(��)S�����yS�VW�Pa��M��;��g�1�/�T�+V�Z���)P8����* ��� ��� d@R @$e d@R @$e d@R @$e d@R @$e d@R @$e d@R @$e d@R @$e d@R @$e d@R @$e d@R @$e d���LVV��M�
��k�� �y���������'������ �����(��'O�x����#���k=`���k��MNN��wo�6m&M���� ����2*�*!!�����Q����������|}}A8p`zz��+W��[ R\R�H���
5���S�C���� .�#w ���������� ~��� %�[;|}}��� | ��DQ�;K ������H�h�������o����W���X*����.yv����)\�7�w�� }��}���qo�w �.��wX�=����q�^�y�;���� r�{
�{���`w�Sx����G}���S�NMOOw"���a��|}}-%:���L)�� ��i���S�&''� �m ��)S�L^^�����/�J��o �g��y��I�v����A�c����2� ������_� ��� }�Q7�.��o� �c�
�
����p�������/�g������w����;,� 8�����^�w7����kx�{�������.��|��-[&N��s��G y����@�kO�<�d�� Xl����>������k4�-[�$%%-[��
�-�����4�w���`J�w�[���c��Q��oo���q�2�w��5 C��s���{�.\�p��YA�O������8�]�vW�\����F��U��G }d����- P����Q�F}������b)1Q�Y��lP8��R�� p��t��SY7u��Z��:�m�j��� �w��$%%� 9��o�y�����+�xW�L ��@�.��]�^!#[�p��J��*+/�����s�����m���(��� �x\q; ��`\e��,cF�H�7���1 k ��7INN 1b��_��ys�c�J��w<�2g@ ��x-�P`��ZX���L� �f�v�<x������kgs�2o��) �D��k,�T�� ����:t��M�6��Q,�2 JD��@�Fe�Q�Q
� �1 ^i��������aC������a� P"��jE�`�}����/�t �������;t� w,�P��*���ei x�����O�>������)r�2o�Y������� �����}�����/�dd���M7���M:%&�<�2�m x� ~��g���V�������(����2 �c :t��'�X�r����b�� �����=z�X�|���?.w,.@R x�#G����.]��g��r��$e �� =z�{��K�,���������� �s����]�.\��w��r��J�� �u������ 8P�X\�� P��'Ov��i���O=����� I �D�N����� |0x�`�c)$e ���>}�S�N�f�2d�����2 @Y��9��S�w�}w���r�R�H� 9{�lLL���3� .w,%K%���1x6��� ��p�Btt�K/������p���yg� P��/FGG�������(I ?cFf��i����;7!) dv�������S�� ?^�X��� ��������'O�<a��cq+�2.��G||��� �I����GO�8��{���7���v�������E�� P��W�FGG >��W^)�'R��;3e � �^�3t�����(I �n��]��������O�.w,���; �&5]��T�M���V=�N`��Z�#�, ;v 8p��3��ENJ\R�Y��,
���4���wu��/������mB�� ��1#��_��_�mO���wf� p���
g�LAtzq��,�2 �n����[�����3#�X$e �$=�r M������l}���*Qp:�r#Go�%�j �����[�;w��������E(� x�i��cw3�����g���+l���-N?u���bK�- J!cF�C� ~����(I � 9�gq������Z����V���� x���ow���m��s���;a� �� ����k�d�mq��� ��*�|�Tk�/��1#��u�y��� ���� ��Cy����� �K*����U���`t����]��j�j���r��8,_ ��Ck���j�W
���U�4q��[�, �+dee����Q�FddlRb�n���V� � �\����z���l�2�JU�J�2o���gQ��
�����{���l�2�Z�e:��y��} �$;;��� WHFF�(� �"�iP�rrrz��Y�F�������O��w<�2g@ �����uYR �Z%T�����*�����'+U��b�
Eed�y���7 @��p&���A�(dd�������10 �!77�o��AAA ���(� PZ���������d�9 ^��� X�v���R$!)������ P�Ma��,$_ R������O��~���������n����6%.��,�\� X���[nVS�����E B� /�����__����+'#cA�7�J���(�s l2v_���7��2V�Q���=|-��L ���7`� �Z�~�z___��)�2o���gQ��
�g���J~�]��f�h5��
�����P�^?t��;w�l���L�2r�c�2o���gQ��
H7i_���5e�j�� �]z�~��a�n���u��32�Ro��� ��\�����a� ���������y�#22�ER (�T� Zm�B����#G^�~�����jY��<�2 @�g5�_��9� �B��?���W�^��m�bR� �U
�Xl �� � �Q�F���_��q f� ���������#� �k�}i`d`I� �#��8~��?��311���_�p���{ep ��r(�R��ue��.9����l�����9q�DbbbPP���8L�7�J���(�s ��:��[Vp������`�r�t���(N�0����III����z�NM @A6��2%YA����g�$�������5��Eq���G� ��92JFM @AnX�Q
K�����f�s ��a�(��&M:t����;������p� $����Q�-� H�% �k��u� �s
��t. @i��+� <xp���ddJI @A�Gj5��V
k�d�p�4���@Z&� �Y� ����+�w���kW������;�� �M�6�Qp� ��ZU)@�(���*����T��(8�`���� �L�>=99��L��� �,m�j��H ����U �R9P���v��,��6W?�}3f�HLL��{w�
����1S �<��r��� �x����o��{�����c�r�� <O�P��6����3���� l]Ukg; H����o��m��=dd�@%�V����� �
���7l����R�J�cq1e��3S ~���������}���� �� /w, �x��� )�C3&�d����@�0g��O?���������n����6%���,�� u M���]���+L�F����pF��FL��� �Q���[�tiJJJ��U����(��]�1ye~� �P��e^-��Z%��M�F5�ap�P�2 \f����/��o�gd����| P�9z�-��:���L�;���,X�x�b��#�d$e ��� �i�� �,Z�h��������r�RJ�} p��t��SY6���� \nVSFUp�� ��� ���/�7o^JJJXX����^$e �:��K�'����O8vW%
�21mB��J0�l ����%�y��������[$$$����)))5k��;�R�� �V�d��,�?eb�k����5�;#
���9��V m�k[S�@��X�����&#��� ������21�t��t� �� �^��O��hx ����+�|���{����� H� �e]��N���jJ*2 �n��U3g���sgDD���@H� n�?2P�Q�~T BN���n{��C�j ��
6��1c���u���;��� �VmB�q�����}).
�����cwm�KrhZ
��/�|���w��U�^=�c��H� ������We���uI�j��� 7n�:ujrrr������} ���uI �� l]��K �i����''''7h�@�X`�� ����Y���%Yt� ��l�2i���;w>��#r�X� ��uI J���[�}��o���a��r��T�(=
�S�x ���t���$��a�~ ��#11�������o�5k&w,����w%��Y��� �[ H�-;~W���T�Q�n�&�� �%%%� 9��o�y�����E)�y���% ��l8�e����mPj���s�����m##�|$e O"�m��)99y��_�u�����E#) x��&M��mP����k���_}�U�-���� P��{�G./XS��M A��{����7o���eK�c�TJ�s�Y�Y+ �����t_`!55�_�~6l������(�2o���gQ��
�2i_����a�jv�+ d���}��]�~}tt���(�2o�Y� ��4�Eo��U�|/ ) 8��o�u�����D� �q M�p�nF�>�`�o
�����!, �������'�������c�3�) �c��,Skz��p��J
�,��;x�`���?���� ;� ��LP�#>>^�X �V��� )�C3&�d9��b���(����3Y� ��8t�P�>}V�Z+w,� o�a�;��X���(�V \�f�#�Z&Y���n�7��Q �����ZX����_
�v������e�����S�X<�2o��) ���KF�>����i:�j����-��A�j���[�FU����y��^ �p�������K����$e �"�|�^d�e����<C\��J ��R�&�Q��zAi����.x ���#G�w��d��^�z� \�B� @
K���k�d����-6���5��2U��:��U�� �oG� ���]�xq������AR (Ba������ ���3�E�`�4
�p����]�.X���'��;����xe�
� �O�Y�S�4 6��
��n��E� B�Vc* \�����$� @iN�<��S�?�p��Ar����y����<�2?W ��E�C-
���T�����3��9*��?�P�N����� |0d��c�`��yg� �(��:���Q �����(�.��)������
�$e or���N�:����dd��� �"�Zb�E�` �#Q���.!��q��9��c���{o���r��AR (������?!b���N�' ����3111�����a���%�� P� )6�?!�?2P�Q�~��� ��p�B�.]f��9f��cA �� P���S �?hdI�� ��*��qk�/���� ���/�����c��%K���=�2 8 ��,Zb�������Q���C.^� �� /� ?^�X��2o���gQ��
�JM�}r���e��ZX����G���C.]���C��������;o���wj� Ek[U�`��Uum��v}p ���K�����L�BF�� ) H�DU]ZV��������c���8q����}H� �� ����U�A�j�mum��"���W�^���K\\�K/�$w,p+�2 �$�t��t� �� �^��O��-�����111��
{��W���FR ���1mB�q��+h|��J��F�EN�P�\�v-&&f���������@>r x�1
�����
6 ���A���1C�X �2 �$!��+Y�0��?Mg��uj�n����:Cy�z`�@�2 ,dddt��q��3g��;���K �$�#U�[DA����@�.����l}�A���' �[d� �����;v���o��_]�X 'f� �{�����t���Y�h��f�k��glN�P
��u�[�n��u����;��� ���4]����l�q~�J
K�8��1��c�k��YW����P
��u�s�� :t�={���@~,_ ���� ���E{61����[���e�1w���}�v�.]��k7g� �c�"�� ���>?�����{����zo� h5���1�n$� ����i�����s��JAR ^"�j6Ja�S ]^d��5 G�=Y�-+��(�R��G�2�n�� ��s�N��][�j5�|�c��PS �������n�t�I+v��X7�����d�r��2*A��S�u� l8��o� _�_����6� ����z����qc22���+��*�! �<�����J�U�����r�,��?M����4,b2Kj�n�o�������S � �Yed��
@������G���k/]�T�R� � e��3S ����������C�|`�P����%�.jVK����������%�* ���){PJdgg?����j�Z�d X#) OU���-/2.���.b�>jAJJ@)a�����/[�L���+l�k O%��oq�:^9��� L���{���|�r22( � P�#>>^�X J����%�y:5]���;���L|�����4}���������z�� ddd o�a�;��X���(�V @)a\X��� � ���A�j�T�������J��u�@a�-����}��
Z�v�F�:�P���c�,��\ J���F5�ap�[ ����5��uV�|��@����`Cnnn��}��]��CWQ���c�,��\ J�I�2�f�(�R%P�������1&���sd X������_�2e��[GFFi�y��� ������NL&=��i�B\�`A�������' ��'f� �&���������##��� ����k���)#���z�:������1��i]U;i_�y�F�7��")�fyyy ���!#�P ������^H�v��1#[o�* S���R��G����k�:T+������ ^���Q���������; x�2 �lmB�q��+h4*A�TjA�� ���Y��:���X��O����TB���| �l�`A� �;!%35]b5G�� �G�� >�������##G)���gQf� ����Ro�=��&^�+8;F�D����������3�S��{M����eM,_J cF���_}�U�2e��(����2 � 3_��^������R ���:���RN��4�`QT��Q ��� �*z�~�����[�n%#��� ��p���E=��`JFA�.4c$
B� ���!��J!�^?r��������J��$ '�� �������_��T��T��R��xu�����J'��0j���W���A1�� ^�@�.�`�_�Pd��U��S]T���`U��2��� �B�(�7��?�LLL����; x6�2 ��dYT�
�S;Z��M�VT Ne���k����������T�� ��`
`)]�]+5]g�� J9Q�� ����'&&�� ������E�� J!�VJ�jaM��.|
ys"���*�H`��DQ|���� ;���$w8p�2o��) /����U�����Y�ur3���H��t M��O+#[�p��Jx�P �(N�0����dd�B� �GJM�MH� ��1!%35]'�j����E%���R������DQ�4i��������������2 �<����(���V��a*�� �Eq���������;����H� ����G��E%�d�Eg(/�
�B$����!� IDATB�����v��]�lY�c��a� <O���&T�(�R��G����k�ho�������m^}�����]�v�/_^�X���) �)�oN����S�\�����=f������g��
*� ��;Byev� �\R?�O�Y,����i#�H ���3����={����� \@�7��� ��Hi�,q �y��J~�]��#�����7�|CF%��2 P)�b��y9��K8v7#[�o3���.��#�o���W_}�g���+� � I (K�Uu�z1���cw�����c1��z�zi�
�b�9�6cS�+��26��9�V�p��g����/������
H� @Y���cs}��~�g�X(%�����s�n ���h���w����9|!wP(Ys��Y�b���{�T�"w,(�) e)�^��~�� �T�`���#YiX��������}b^��#o�G �1$���s�._�<%%��� �;�$e �8v?��� V �l�y� h���]u��k��HV�i��>,��a���l�%��,��
����t��}��U�ZU�XP��� ��0��� &e
��b=��mU���M��!RZn��O@���g l �&W0(Q ,X�xqJJ
�I x �[wA4j!�Qpas^�����H\��fAM�����
&�o`P���
%a��� .LII
�;�:� ����uW i/Pd�-Y�G .�j�OP�J��V��R��Ar[�|�� RRR������3e �P��_�k�*�� �g] H!e�������;
�����w�}�� d�E�:ip�J�{ ���z� H�-+x�>�a� ��c�v������f| p�O?�������woxx���� �y����<�2?W �`3�b,�:i_����b�jvq�������V�X���o���7""B�X�&��yg� ��<��k ��SX�WW��Uf� n�j��7�|s��=dd ;
� �}����������l�m��b��,�����t]IW~Uf� %���>�>}zRRR�Z��� ) 7��im-#[�p�n��~Z������_���@����/�O��{��z��� $e �N��ST��������q��+h|��J��F�}S�uR2�$fg�8I���M� P��7N�:599�~��r����2 p ���*� ������ ���1�t ���5��Q��Ca��jE�`^3�5�~ ��i����''''7h�@�X�)���gQfg e���3�im�F-�k\���La��\% ��y���'����a��r��(����2 p Q%�l.X��7v��P��D[�l?~|bb"(5e �&��K9�R��������(� @� ;v� ?~�� QQQr���L ���t�y���Z/Y�(d�K��@�eP��`-11q��Q�~�m��M�����2.��G||��� (�q^LF�>� k�^�vx��� �O+�Y��ti����>��2��z3Av�l�SRR��O?��7�4k�L�X ���x�
������:7�E��� dg]��G-�,������W�=��[Vp�����5]�{�IA^�������m[�����J���w�/ �+�
��$���+��V��X���� �?|-���� ���5�y�
@�����Y|�n�����?��g�H��=v��5|������� ��� �:��K�g��qERa��A��\)8S�b�����G�������c� G�=f��L�GU���2A� A��� �E7�=��{����o���E�r���2 ������5D�Gj5���7��m[Uk}j~��?2��l��Zx����6��y�vZ5��h�Y���2d��M����;@�2 �������� �5
.�V ��|}T� u�n]�����{6���J��k3%'c<(
�����_�
6�o�^�X �H� @*;���h����/�� ������cw����Ac�q����������Q �f�R;�:#f������j�W
�� �q����}��_��C�r�8��2 ��d���-uL���hlfjvU�y���fzm��
�� �� }^�\�JJ�\ eV�m[UK�e������o�u��EGG� ��2 ��M���|�uY�
�� KF�^�{�n� �L��g�,��-�.�s1��������O>����kcbb��pI 8��|��I4���A����3��b?�jU�m ����F� � ,���G���+��~�����{�\��c��r�8�� \��$Q>>z��g�(�W���Go���o�����5PF5��D��S���W-��s����9���;4S���
�z�?|���� :t�O�>+V������� NR�� �h*�! @�'^�,����t�,����e�\�T%P��CH���/�j��K|�G8��[f6W�W-h}�9y�w� *L��u��������
,�r��`y�B�:|�pll��e�z��)w,� ��yg� JP�����@VF������d�Xo�X0��+�X��3!>��Ne IF���f��� �e�;�I/v�������K������%6 J��4]vn������ IG����.1�c����������.[n��$K$p��G����.Y��W�^r�I ��
g�,W��u��R�uR n�wA%9y���� )����?2P�Q�~��J+��rr�O@u�-��d�%���c]�v�?~�����p�/ �1vZY��fV�Xs��������"� �N�A�P�WJ1`�e����!�(����c^S&�������p������ 8P�X �Pb����ZA %����V��0�������.�������Gu����~T�`��2����i�������M�{ UJ~KD�%�w���� ;��={�`����yWbL�E��+ @ )����4�GV ���u��y=���[����]r�<^z/' ����S�:uz���� !�')����2 p@a�e7�������3�5
�`V����}���� -[)@��VU
���Z^���B*@)q���N�:��5�� �5e � �:/�����1�5m�j�V��O��@���=>��� �{fw���n��L@!��jA�E�C"����J�%�� $^�� �� ��V�`�)�Uj�H�j��"~B����[k�"��%�M���m� ,�3���Kvf7��_u2;���sv�}��/)_��4vgpD�9/��m�=g�/R ZZZf�����O��7O�� $ :e
�"��a@ s>�zj�q�x+�o���-����njs� ��9�� gs�_<�}��3|���`�����)a��@8r�HYY����,X��Y �G�R�6�� ��~YQ Ct�%����U��e�;e�����u Q���p�� ] ?~�����G ���{�> �m~y���R�6� �o�O�7H���N4��c�%��E6
��%������.--��/~q�}��}H�����3�m�^ +BvTD���f��w�a('��
�F�� �Wd~������j���6��c�/ �o[�C������p��cH���7�� G/�i� ��:F�8r�R�}: �N�8QVV��C�"�� @���J~�����<vj���| ������p/p��%�n�� �8y�dYY��e��,Y��Y �E ��4\���~n{�#�'�I��i������
����<� ����S7�p��E� |�A���$_ ��� ���pIg�O9��
6��P�cD�c���&�������N!Iy:%W @�:::���.\��_�R�� $�2 �E�[W��� �t�oG���q6�������{�E� �p�~���(�>���F��M��v{D�u���P(��- ��� 4�����7��G U�, I��% �A!��+���p��� sI������4vV6��4v�i�y�iF�A�@&�����f���"/� M9}����__QQ�����}�dCQ `P�g��t8( i����������B���V� h @;l6���_� w<��j�@_ ��zH�������sI���� ^)_@ �Wd���W�X��Y ��N �AA��|��h8(2Fp�Q�H�g:]~�@S��#�]��������n�5k��O>��Y T�� �����p�.��G����b�p�(/K\��2��"K��#�]�����n�����7����gP�� >�H84%�a��p��6�0���c�i<p�n���Ht?q�M���w
�f�������}�Y���2e ��l]�w���(��9�-�'r�q� ��n���9s��i�=���gP �q��B �?C �(,��)*���u����~io��qWHI|E��k�]�f��g�AG�_��) �R��T>�,*�D�n�� hDww��7�8y�dTd ��X(J-�,� $MS�{Ch���qVi��PTE H����o����WSS�0J�� H�_��x�����+ @�UY���#t i�Y�B�q$ l ������ �a�Td@-�����% �����]{������ �~�_�./C�&\:�a��\�u�����m��B���LE�j����t�r�-c��A� ��2 ����0�� o_��lN�� ����GB��H��O����o�la��!-�Z���Uq^�z���U���+�>W���>Y~���5�yjXI��O�b��LQQ��
X�U�8 ��������( $b�F��p���p�G��#�m_:�u�D�DF s�H�_����g��������4`�1��}TxEo �w{�Z��V�{�=m>����z3[����"G�S���u�-�\|��/��*2�:m~yG� @��v�(�p���uJ&��o��P7G��0D,C�tW�e|t���T|� ��~n{�E���s�_q�P�$��tEG�"CD>G��� (�(���7g��a������"�� ��D#H|�C�k������� .4��};O���6 Q^�����Cg<������@��g�_1�P�$��=�_9!��< ����n�X���?�t2�� x(� ��pv��Zh\4�����������>c�1w�5+y;�w��MZ����@��L�f ��.1�e�j�-�H������+2YYYuuuz=�3 "AQ U�J�QU:�_,�-����q��"����)r�E���}w�m:�ne6����@�1W�L��[F1�Ao�3��T4W�CEaRu��|����NZ���yR����3g��h��y3*2 ��������]A 0�mso<�����:f�8�l=%��>��y�%�n���� ,Q��2'��S!!R�����x���
���[Q�����w-�)�h��
�D<���6��vK?�Xzm�0%oG����F�5����z��?��e��[� ������w/ �\��������3P�
��
������ � '��;l��X*�k ����/X������� �P� HUJ"�#�nq���Q #�}� ��~�����v��o������q R � �*%��� ����OT��s�0 �4�G������c*2 �BQ U)iQ�pO�X�0)�C"=�J �������;��9���o�� 5�/ ��\���#�����.i���������iwokvx8����2��\��jL��g\^��)�� 0�������������FE 6Z\>�Z��� QF��!?w��b�1����46��6.
1D3/1��k���k1�����M�&� RW ��������~�m����q ���/�_ HUS ���[��tz���`}\H� �>FxO^�n�x��B�h����N��6w�� ����{�D�k��q w�����W���Q���� RX0�z���nI-��������L������S&BA�K �_������f,�� :e ��l�%�����8> {o��� .w��e��1BA�K i���%K��� �hhh�X,j ��( �E{��K;+lK;�����_��"�����]����`�F�B� �� |����"�((�$ s��U��> ����i�K8�: �+��t��|�g�� ��1���'@��H�H�Pp[M�F[f M�8n��������w��H���c��U�/�j�E���m.p �������z�� �t�Y���\�O��{��f�( ;'���s��0 ��?���k��)|�i
����BC���Z~��_~������\��Y b��/�X� ���6w�����)��E����_ �����!��^��o���y;�����&CE� P���>�k��]�v�"�X_ P_�H#^ ;t�a�N��%F�@l���FT@�x���w��� ���N �Ik���%�*���E:�$�-o�e#L�rM:�[#�)��Z�z�/�;�tY���O��*�>���'�hhh��k��^��Y �:e �'�4��d�M.�w�*9t����C�[#�)��Z���U�z�q����}_���f��X�b�;���k����~vZ@l���&�hsW ���mn�����S��W�� 9=�@h`�� m��a=0�(��1����+z����[�:���'�|���>����C��}����w-�)�h��
)g�j��G C�2M�' �8�$� �o�d$_�%]�E����JV�Z�m���������}�����w�� ��"��qd��^���^�io���N� �-�M����F�z��\1�����<�>����[w���� �@�N ������9�ms�}���� ���&���o��t�M���=�|x���p����-0� ����� �`�LX��cB2<��s/��zd �E ��������ko���y������(�H����6��f����+8��_1�u� \��L�9�k��Q k�[F�L�1�M�1#i���4�y]�g�F77�u
������~��� ~��E]��Y �/ �������]pE8|te^F����X% M-4W���J5��E����@:�9N��8O��@SQe2c������}� � �4V���T�$� �a��5�����{wAA��g,P� H����6w����6���.��� m��ov�r�Ta.���@:I�-0����+2<�#���e��v��u��566�"�L_ HI9��r9g��6��j0� �;�$ h5�@�I�-0�'��+'�W ����wk����sgaa��g\P� HI�/�� �t,-�����3���;�4���h�5/K�g��,����)�H_Jg�l�I �H����g����?�|cc��Q��> ����% ����D��
{�!��� o� U6�"g]+��Flv:I����T����-8��7��V��{�TQ[[���O766� =Z�� F(� ��m-Q�����B���te�C����k����]0
K-��5��H��� ��O��v�y���k�V2 e����/�^� �( ����� �CI��B,CA����VR���5���r����9��p$�f��4��5�G��\�u�����m��B���R�F���!%�C�>E*�� �X]]��� �}�� E ���k��$ee�vw����%��`�u ������2��y������9�v���
�F�Z���Uq^�z���U���+�>h�+���|��>����X�� jX� ��d����NOHE������f��X�;���+I���|���L{���}��4 '��!"����)�W� �u� \��y@;�m���O���������g�P� HI�I�E3D�e`��I�R�P���/�#���s�_q�P�$�)�����?����_z��j� 0� ���I��"gh�fj�1� ��q:�����d�]��2�54D ����&x?�4�KZ����9r�<B���Fl����� ��s�e�]��Y �E �tn��P���lw����k^n�/X*i��N��L� ��e�� �
j�L���88���� ��=����~��e;w������> ���% �iw/i��l�-i����N�cx��� IDAT�)��yj������TT��w���h�U��Q����T\�3�Fo���eT��S�\�
��:[��S�2l��]�uj��c�� K�.}���~�� �} 8��8��� <���! D����� =�����g�U6v:{�������y
�E�j ,�6kXl���Z����#����f�����o�YUU����� �P�� �F�_��x�����+ h���� �sCC2X���i�a<`��_�Q����t�)������= ��1����+z����[�: ������������~�#���&m~y�N �d����
��v������O�|X5���B#)�m��Y�������:wDoT>��1��YK��9k�L6����~�����*�L|�TT��� ��}������/� *2 ��� @�I��2$�A
�G~���B�����{(��� p�=����?YK��9k�4��8����z���U�`�)����{w�y��o�}�UW�} ��� @�I�N�zFD-bs��h[�CZ=�J����p#}�NI/�%pf���*�����a�;�7��:z�DQ&����s���o����I��> ��� @�I�N�~�;�K� U6���0����pH��Wr���"*(HI��AD:�&[��,Sq1\��������&,���yB��A���� ��������> D�� @bX#"�i���/z��H�2>I���8�D�zV<
%��H
��)��QB:��*�,Wk]�� c{c�h�Io ��!^ }���y��o�~�5��} ��� RFS���p����8�-eo�;Q/Y�}�Q�Du ����!�8>G���+) dg�yY:=��e���NI�1 ���5����A��-�6U�z�q�����MU�����2���eoct�� ��;&�n��=�����m�����> �E �d��$�%S ���[u�?�� -o���W�f���*,����K-/�����{�4�c��_)W�����Y������a���DD #��qD���X����{����u������} PD�1��E�Q� 0�5���1D\� !�Z�F�!�b8�d� /��^�����p�7��yN�&"�����UR:�9�$��!;�t�U���^�1�#�m;����
���c�no���|MMM��v��-[����> �i��;v� (%]#���K-�G QO_��P�����W����'�B&��� �s����A���@!JI;���j�b�W�g*�&[w�C�_ _bg/I�s�=
�~S�� |�m�m������% �DS?<���~��x� I_;��X;c�����\� y� ���r��q|���/�d�
hJ cqB��:[��S�2l���������s����?��� ���c�{�cD��)z,���}������� ������^�� @tP� P�_#B�B�4 YoDD���6hZ���+�#m�e���J��']�o� .�
�:D7�^.�r�����e/�3�o��wd?7���/�5�/�03���������?}��Wn��f�� Q�� @��� �f!K���:�HtQ��A{���� � ��DDN�u��f��+=hA<���W[c�D
/S ���w���<��fJ���L��p��h�F��!������� m^s������TI�c�w�����go��q���j� b�� @t���E�u�Y��{"������N�K����R� ���4 u���t��l ���Q|_�A ���
G� FH��%�:!SQ�h+p������}� � �4V���T�$�]���|��555��z��g�a| :J���{�3�~����� �p��4Y3Yb�!�f�UWd�KJ�%�(����.��V��������4vV6��4v�i�2���-������Q k�[F�L�IT�Kl>]��Wdx>G����+����g���~�� ���j� b��D����T- ����;;B{X��u/L�U�Zi�����+�e��cTnjso���g�5�� P���f�'��L���3y��������}! �u������:O�8t���Y����?��'?Q�, )C�_�1� �x|��OGR��Ev()��I����K�Q�Q���.t���T������r���0z�8�ydw�$�e��~�re�A�������x��k��EE
�( �`!�`B)��2>/���
�Oz�%��k���`
����_�1��*�SqE��8~}��O�����kRu~�������NZ����R�_|q��7�Y��g?���g�@Q F�* ��� �Z_� ��� CD�k���ijst� ��0D��0*�3"",�����= G|;��] ��W��8����JiQF�m�*��!� I_��-�����/g������f�\
�C@<P� ��������M��]p�eZ�Q�~��2����(_���C�: ��3 ���0�O�O��P��i���y"��TWR91� ���<c��_����� � !} �v���@����m��0�&���0�H�����$v�(�����I]�U*�#"
T��GDu����V?��� [�hWk�@ �����������~5o�<�� ��N �X����_�E�'B���3a�,�.b��"?-�%��K��t,�E�J� ����p%
��������+V��?_�� @��S
{��K;+l�0���.?�~��y�����P�K�� �o���+ k6��? ����>-
H[�4�I�T\�3�Fo���eT���@�p����I��!�� ?>s�����/^�X�� @�i1�;�h3� BS�{���9�
&�������V`\��S��"�0$� ��c�i<p�������&]��h���6����E�t�K���o�d$_�%]�E��* �r���������������> @����w�/ (�M��e��BJ3� �J����+2yY�+�2>:�� ~Vb������2g�"�Zh��qMI�? }�H_���+r+i@��������_���� �1e ���"�1T5!{��n�`��,���\���� n�I�!�Y����IIA�������4m>i\�R��4������?�������Y ` a� �R��H��������
�/��Z
�>qqDJr������{lN�/�����mn�� �C��W� ����W�"��$�+i@������?��� P�, 0��) ��h")�17r���#����c<N�A�$wI����8�x��������C��'��*ME�v@X'O�,--]�x��>��Y `��S @)>V)/K�g��,���g��F�7��aZ4�:,K��F�(��+�2����_4 '���M)�:�������c����-z��G�> $:e � ��D�86@�����G��q�BZbr2�iF�#i���B���y�?���r }t���!�`��l+M��4�o����n �9N��8O�rH����������?���j� �E �k���6������`������Z(|��3 a G4�#[���qB���b+�h�4>��b� �w���
-�J������O��v�y���k�VR9D�i�������*** {�1�� ��� @�z}���7@9F6�eVL"wy�+��k�Pl��)�:$�O�e<]��b&������}� � �4V���T��}.��+2s��}��'�> $�q
!<���! �`���^w��+���,K��&}�A��a�%�����9{_������M�M����@i'����w�r�5&�w\ ����c�noQ�lJ�6��~�#��3&c^K�Z��8��VVVV^^���O�}�t��/��� ������P��0�3�e� G�s�N��L�.��x����"�H�'}�)C�Z��x�N���^���+�;s����_����"08�( /��� �H��" S�E;�)'��k���\�8�H��HH��e�A��hYWW�M7�t�M7�Z�J�� �:P� ���k�Pv&;�P��"]F��6H�� m��^�+�H{�5�|/1�:�qq�S���fv��|u��A]]]7�p�����y��� �AQ ^9�lW�|��Q�,��"��D��V(��� Q�^�-��������X�SR�C ��47U�����_���6gwaV�S?�UY41!����3g������{���< R����m�
���������nAQF����:� _�a9��t�8+ q�k�u������e�B�.����B�7���,��y� ���utsH���3y��h������ ������Q,�!c��9���e�����^�f���� Ji�����Z��{ ��'(Y2���Y�����;;B����u/��)����U����)*�h�,����S���t�\�m��h<�����9s�5�\�v���N ����w�/ �N��Dd�3�B�t�o��
$*�������(����=W�9���{���-�-3W���Y"u��� �N:��+'�]�<��p�z��W\qzd ��� @��\� }�����i���j�u��02k���3�+#�b_��p8f�����}o��� ��- HV� ��\IyEZp�bh���C4ahFlOSR��j����>�)�V?��Z��q@����f�����
�'����N�-��2f�Td @�2 0����%��D JF s�X����x�#� ��3 �g��4�M7�}��-�}U�f_�1��*��p��I�\���T9D��hGE�D�!a����+cx_�)**��a�������&�hsW ���m�
Wy��UD$z�������lK����}f����6��j�x�u
��R�m�_�Q��uL��-� �y����6����T��-�PXR��9���t:o���K.���� i�����Z��{ Y��"T^��'�d ���!jo�7��H�3E���6�����}!�t��w+ym���^�l?�^��1��$��������[G����/�"�.m~y�� "v�Hw�n��wk��;w�'� gs�or�]�+;���pw��D��4)�4N�5��s$��y����S�I��� ��V___yy���C_z�%Td @�O "6�J�5�}��� p$�������>���omv�q ���i#�S
�J6Cz�L�f �w1�e�j%/�=�����8�����x3� �2B|���x���c6�_{�5��� y(� � R>�l��/�7�J�$Z�� U6��4v�i6�>������g ����m�'�A�T\�3�Fo���eT��S�\%/�9N��8O�y�I��z���z3;iu~�� �����h�����1� aiq�*�hs,
� ��E�CW!��_����D���1��������������_!�/%���)//7 [�nEE@;���]�gJ-��� @ �uO�����F��7��8�Xzm��������Z�+��������,N01s���
�l`�y����r�N�u�V�#` ��/�(� �%�������pw0�Z����:��? �@'Z�|�����6���>Fva0�2�kjs���v�����= G*�����%��\�1WH����D���N���������,�� (�� � �&_�h������e�;E���*,�&��N���1R� �T��B-zH������]{w�w !Y�TT�������}� � �4V���T��3����������~��7Q� ��� @���W0|�
��E\,"���X���t��[��W@6� �������z�q/_+q � ��K ���Z|r�x��>]��Yx>G��� ���~�� ����|����L�� )�2 2�}{�Zh�.%�f)��N��K���7���A6SBH��V�dS��
,�>�a
��7�9Y���W|�����)������ .��������� DE �� W�D��N��Y�� �J�����PK �1��R#�7����x���O���pcJ�7����������;m6�[o�e4��� e �$*['��*��g�*-��r��D�Y#�)���Jd b��2���2CD\@����� ��}#������m��$����:_vL��"G����wP w�}wGG*2 -&B�m�j N�%I����[�ITh������ Q�6�.!)�� "�)�2��W��m�0O:������/���.���}���1�� �q��������744��0�:m~y���R�6� �S ����8�X2X����@3���p�0���f
S�Hi�R yL�_[�ST�zml�hL)�8c��%oB �������?,�����i��;�� }���sF���� y�$�?R�Lql��d�P�QJ|cK���������=����� x� �������� ��� �>bH��9#Y�� ����!<�1�� �(%{����:��9B��"����:*�50�<�>��`��M�������i���%K� :t=2 ?t� @�P��W� &+���Cf�v�H�[�Vvx1o��D)�
�e���B�T)� ����xO�4�i��.8�[�t���{�=����q �iq�*�hs,
`PY& �8x�a��t W�u��q�)
&����� �|��w����lo�E)���1y���x }vq�m2ymL��h�rS�No�a�o �-[��o��{�������q :����N Hm�<�='��7�Jw��d�:&�.a{K��qy������eBu����D��mTI�1D�N
#�{O���] ~"?�q&�� {���i��]�� @��( �-�M�{���%3GNo`��X�����gc�(��C��+����e���<����.4}IQP�e�A�$�s>]�����c�=�s��]�v]p�j� ��2 ���o��{j���YvIcg���=��W���� 0 Y2���Y�SQ����XUT6h�l���|�x��a��*�]+wb��O�_u~��H��D1LR�'��4��O<����~��^x��g���� �6�y���� {����6��1w�?��� ����� G���9��0�Z4ZmT6h�l��xt����5�����\�{cU{ �zK*r��>Y~����\W�Ox�e�!Q�Lu��/��wP����Hl Hm�<i":��/�ufO�[tg�� a���������6x�0�Zv�J� AkdB�z���L��[F1�Ao�s��r�yQt[�����tE���� ;R����! �*��s���J�>��|�����NZ���s��'�|�/���]�rs�� �2 ������������ ��Y"�X�
7 �0�Z�hh�>k�$k������fC���"�0����H���6v$j����a%sS~�L��c+W���}����� :' @Q R��%X� �h��[���s) k�b-�B��!��r�MH�( ���}4�!�T��m/�g�����n����y���rH:�`�?8��3����������h����% HaMm���=6�|�����~_�������Vk&���,�'[��5B?1XR�x-��**;��<��6�@$���"gr
���N�m/� ����/]yd�B���1v$+�����{���_���� > � 8 �2 ���.�3a�1R�>���N+0��������e(��%[s �����{H�6~�J����QE}C��*��������"��OkI���
���QB�38����v������]tQ�� ���(:xA�a�g �lMm�
�
"1�[{����k�� �,�u&���7��K��VLD`b�.4�� �^k�1���J�-�N{C K�I��mx��F�5���q*��ji��
)���P�r�7������n;���5k��[�n���v: P�6��c| RO�pk Cd�d�3��s��\����4v�z�q���rh���!�8�� ) 2R��+�>�>��o�W�gT9I|Hv�1O���!����j J�
��]�v��u����� @r�( �'\��Q�,���u���C\j ��%����G�0�c(/K�h������I�( �V��> 2��8������$�����9$;��k���O��wp��^x��v��YXX��C �^Q��p�����;w���7������eee>����I�O @#�]$Dg�*S�X�n IDAT�M-4.o����YF�#i�K^�����iF��D�?�ke[`"� T�j��>�������-vg�����d�SQe����/������<��7�]����q��QI9 Q�-��8n��e�]v��w���� �a���uuu;w�4 ��m[�t�����S �Q��Q�, 'S��V`�&X���P�&&B�Eb�#Z����� W�DH��w�Hz�p-0�����|e��"�s�c]�u��|+�W�p�OW�4�� ��Z�Z�/�{C���0+�� ��,�������O?�4*2 �|)V�a�������� ���Y�����`0� w�QWWw���`�]<? ��7��d`����\����I7_����
lW�~�`%EZm������hj�1x�o]� w����!�p�-�+ ��#��
����}_��c�����Sq����ySo�I��������@��������~W�q=�QE����K/���SO}����G� �c ��XQ�_}}}��3f������o��� �)�.��6w��zJO��:����k�0l����z��c��K. W�`�K$��[� ���& _����DUa��FI�6 i�rP�������%��p�l�^����+e�M��^~S/1LI���k�H��H� �)_7B�W����� ^��;9����y���������?,**J�y �J��2���edd�\|���� ����1a�Z�*� �A�OD�l�-i����O��T�H&i����k��`���/�[b�H���+c�_,�����o����J7G�����Z�l��Om����v��E�Y�52��n�('��� ?����+:n�m���4�y]�g�F77�u���|����_����fP��t���;�����X�w������|t���W^y�����>(..N�I``�Z�*�7t��&/�:e�^���b4�����id�: H9�>��P����#���.4���*Kwwv �^�f�k��K%�l�v�? ��t�(�xQF�L��G�O�1,qr�T��AT���G(<[����N�q����
5 2��:m2�l��5�H�8�\�;# hz��W ��>�`��1q� �c��U��#�Y�I�N�� *����`�%�� �@��;��Y�s$B"��S�h����_Q u��&��L(PN�#� /
w��u���UAE��M�b f�����,�b��Z�����}�p
5%�C|Ur���_�(_F�:�L��=��|�,����+g�
�?��
�?����~��� ��]�v]z��q� iU�����zC�g��'O^r�%�� ZTu�
N�J�
�L���e��q��L�����K�0E���� 7^�j���Y����&��h�eT�����/�L��[F1���b*�����'������Q���*�'�\;g���e����9��/�3"�l����� z���;w^v�e�� i5�DDz���o��������s�0�:�� ���N E�;�Vp��F&�
pn �pd�3%��r����T��M� ���YT���<E�g��� �"7^tv��;8��Y����.����cME���`[//��^,�i-��������d�TM��e8Z��>8�d6d��x���/[�l����_~y|G H����!�� ~x���~���v��1|������� ���%r�!\�Hp��]�Z(@�G����W�#����n�p���Y&T+/
���ctY�>[x)����zG�d-���-�������o��%���[t�o��t����{�?�A< H&����y��uww��� ?~�/o?����^{m���_���F���3f��5����'��SY �b� ��<�yo�[�9�x\H�b��������f� �s� F���Lv����� �d��72�?l�c0��z��7���4�u��^2���+]��1z3[���O��v�S�Rh�����f�'��%0�`���1�3_;v�����&N�������/�Z<Sj��� ������mN�|����f
S^���p����-�����A���X�����:&���=�kc��GC��r�d�k)�7UE<�K�����������w������@
����t�) ifO������8���w�3�����"�tQ�nKw�pD����~��L��oHE��$i��7��������������_��� hM��� �t����9���������@����]D�LA����R���6� �Z�\�u����6e��G�?YO�����B���K[��{�D��
J�C���w�}������_��� �H�� ��( ����!�!�(B%E��DD�L���U���Bc�Q�I A�Z��#�0�/�O��,SQe����/����*�����N0I�Qs���s��;�����������> � �/ ����|��)�D��Hg����g�A��R��h��-��8q:�dY@b��L��p��Q��D+��K�W�#��M�#F�����o����I����
�( "Z������R>����n���`�|�YXd1���q�
�pK�9������_ ��9NF�!��E��{�$>�2�`?���2��� ���/���d�1���� `��]o����W_��7 ��2 �!��/ :����l�J��F�A'Xd�t��#o��}���Wv�0h�>kd�t�d�����k�C��6 ��=�<�U��0|��������m�H�I���m4H�����%�(�g�����������.9� ��
��}aY�f� 1d�d_�=ER:ijs� ��9�� �q$�� ��,i�����V`�[b�`�@h�&��wO�{Icg����BBY&T ������D� � �Z��|Z�{cU{�"��9�����\R�SZS` �������������I����T���w��9s�m��o��o�yG ���S&��g��+W�Z�J�� �6����&����J����8Q
�2X�/� ��z���N�?����� ����/y���� �S��9��5 y�$�G���������OWt;_��e]�T��l�2iS���������o��u������v �q�V����V��0�d D�a�g �0{�������Ye[c6n�e�1�z����W�1���:L�f u�����u/���"���c�gUX�����n��X8,�0:�ab��H���v��u ����9�l�a$g��3�������yGJ�}�6�~)���3y�@�E�������n�����������C�_�1� 2���h�5/K�g��, e-�m[�n!����W_��|E�OV�*3;�y��P� �����5z�(�5��<F��q~
��������?J����<�:��#�-#e6�*\��A��#���-�r&� tEf��}?��O����Vd����{�y^�_����>� �� E�E��6 ��6���7��"Y���� k&��2���������|���T��1����a���c���+d3��:&���/�6���c�(�:a]�E���=���.��KD�<��5��m�p�n���'��$�V������}���/�������+��i�
I���1�%M� �����2 �Q��HW�0 I?heg�}>.BO������>Y�����(9��<r�a��Z��*2�`��p^��l��|��/����b��RR�C �d���F����[n����6X�!����=�V� �E ���;wE�K^����K-�� �� Y""�HX� ff+AS��������{p��b���h�/1��~�y�N�w�r��Q X�c ��#�c���b�Q��Z�/�{C���0+�� ��,���� 8p`���6l���[��-#
�#!�2�2Sc �BQ �HvKK�(#��d�7 �S�1��>���a�K�Y�cN����%��%r�D�{B\����e�`��TT�/ ��N����i��4����U �;� ":��]��� G� �� <x���o������[E?�T�/�S�� �P� -�w�n�z�twgw���F ����?+&���:?���~���9�X��k��\7���0:���pe�`uf���i�R�����"���eB�(�(�V��X��]�"�sx=+�LTQ���C�f�Z�n���c�Oef���$�} E �"�[Z���h�x��W%X1��A?KJ�fWKk�W�������y��Y�;����>��- ��9���e��T���@'������p7G5�t���o�q���?��O���� @2�( � *�(��"-� 3��5���%m�v���U��P���=��{(�TQB�=2�z$�zZ�U��������<�I��YC���^�%�����_|q��7�Y��g?�Y�� �LZL�J-�L� H-Mm�
�%�{�YE[Z�S���E�����S
QyIs�������D%"V;}������,()�Pl`Ii�3/�8c��� ��-��>L�l>�������������EWJ� S���t�\�m�����/��������g**{� DE�_��) ��]����\%]-X�;8�z�_�5�H�&��c�4v��?
AH2U��g
���� }�D�(��1$ J�����A�/��r������Q����� ������a(i�q�n!N�7$Q9D������h�(~�g����H��&�0 TY4QI ��A����3f��W��7o^"� � V� P�d�o���1���^��Y�`[��������C�� \L�>�a���Z�l��Om����v������)�b ai]��_����|����s������e1I8L�U_9�l������z�M�ZZZf�����O��??�� ((� @\;"�V>�l�1�LlX55��k���~_��9���{���.�i��'�f��~w����}������
�:�Yz�eB5c8���_�+��H��b�\��it����>y��� �XGe��:*C�G�7p�QQE���k���\``u�,�\;G8�t������'�xb��* ����&�hsW @r�.��*Y�+��!�����*�G����q���;;B���u/L�M�[@�Z�HW�������1V�c�� �U�����p�o��
�:�?eM���$JY"��mJP���!�2���>��Y��
� ��������>��#��{��g���/�Z<Sj��� 9;��1DK&dK�B+mg^�io������f
���a�����:�����8[��0�����/,��w"��~N���aa� >�=9c2�����Z�35�����������?�������Y �i��;�� v
���\����(��,�,Rv�@��������������~��-{�6���3o����=�BaE���,�|f%��I��>�=����Mb�����Wd ~�aTd ]!})�� ���+W�Z��Y �*��;�s�I'jca9
d^�$_I�#F6?;r�M�X���1�������g�M'I���_� ��q�����&-�Ds��B���"���M�$���8q������ z���> ��U�VUWW�}�H�)� �9�� � ����q�� ���rh������{e{���U �C~��B�����,��e��t��[��:9I �jWt����� %�����bX��2�5���&*T; ijR�0}�+{�2���N�<YZZ�x��%K��} Ha�V�
~aW�,��) �ijs���:��� ,eg�N/�7�lo����e>X��W%\G�l{N���V`L��`8��
b��U��d���F�!a#����2����b������ �:�Y�S%����T���8d���5Q� � &q��g�����?�������"W���K�S��S�n���{����G Q�, K�{nR�6w ���}�+u"���4�t����~��%�S���o�����"- ���'��������`JH�S~_o��a�g��z= K�NGGGii���� {�1�� iE�_��x�����+ ����d�lw���%'���qnI�L�PjQ~�P�Y����4����q1TR%����b�"gK���R����KKK+++ �q�� �F�_�1� ��<=$]���R��]����D
3�.��=�A��!�E�\�[����J����}_qL�ZBd��"e������W� �4V��$$�:r��@/- �O�.++�;w.*2 0x`�/ D'��U�J]"����"CDYV���v��Y���%"l����� k�����F����a��?�?_�ys]��%Y 9[j��+Q�����3^����O����|��f�����������'�x Z�N �N� VEm,��?��Dbu q$���8z���{C �N���R��-i%"�-!�p�s��6���O~������@�������%aiq��� �����������>�g8�(�({3_��3g��+�{L ��S �6���bin���Ks�-��=�/��h���_����8��:�
��4�:�-!�@�e �s�w����kY-
']�.3�O���\#��J����+2<�����N�;����Y�f�t�M+W&�� @#P� �H����4vV6��4v�i�����]�V`�����t��P��%����HT��BKH� '�N�T��7����7��V�'��L�9�k��Q|!f�����==|�'o�WZ��>�����p��wuuu��9�����g�I�� �E +!=)9�����,��5K�5��`���+�D��P�� $]:�)�������V?��� [�hWk I*�l i��bY��Y1�Q�p�LIENiM�uTk`��2Jk
J�&`�/�TT�7�+�b�Ff�C�Fd������u�]����&�\ ���2 �H��Xd{R�F�mw�������ke ,-��BDS �_~�}��+� CW�e�����]{.���1 �o 9� �F��%SQe� |�R@.L�#Q�RI��D�`���+g���v'�]��E��o�`��o����S�����U�� ��N ��"������L-4V]� G�~�iK����*��p�%�`���G'���"�(��tR��#��Z�(�c ���3 �XJ]��s��:~��'�g����'E����k������W�� �N ��"����=)��n�yLMm�
��/�zF�{<�� t������V`�7� R���%�/�@ G�:b< {����{Lt' ��j������l�ef���9���g�����F���9ow������� x���>���Go�?��>]��wuww�t�MW_}5*2 (� 1����5o<WC�����EH C}�y�S����� ���g�(�.��mH-�U���/�� �u8���� ����&��� �������O���OP�R�Z���>�/��}��w���������p�z�����[�v-���� ��( 0X���E��2aX��f� �\`d=~.�u3���,��o`���7�,�R��]��WU�\G�Lh�C����}�B. ����7
PQfRu~��6�S8|�Rs]�'�O;���B�5O
Tke��K��:,# N���[n)))���AE �P� <"��Dn�H�uJ��i�Kv&�� ������d��� $0h)�f ��hS��� 1 �q�E^���7
��� ��O���9&�/c���M�` �����"N�1�h�o��5�H��� ��&, r�-��" �� �`�*�����q�}7Mmn�'�8��`h���S\mqz��������IH�D��8�K��;FDo ����C�^�,�c����kB��_�,l��W���L��3��wx��Aez��:}����5j���,�� ��P� D�Y�y�HI����h )��N)4�[��SZ�����u�$$[
"������F�z�D�����o#"]�e�����?�����"c��O[3\X�!����j�oE�D���ohsvfe� 7��%O\|������� ��&& s��U��> �@������`�����)�/�q
�4�A�S ���[�z^�D��F��p@ ~}������x]G7[&T3�h"�9N���0����CLEs�<��c��������?����^�@��h.�^�i����V����jI����h���Z/PH�SkmQk��!IT���(�����9��BB0 !�M6�������$������nn������Lffg�L>>�H������
��v+ �6 IDAT�9���S��b�Y���sm�oQn��������~���3f�������mS�s���� ���m� !4��]�V��}��E V��J���B��@Y��d�]gD"��4 ��*�� =U0q�� ���1� ^�ek o��Z T�x�9F[��
m�� �%?.����������[��2��B�v����``q�o�<�����o��f���y�f��
=������� k����'3��P!�ih�P_F�����7DH��ce���������� �r|L����zI%b0i� @R��Zr�z��=%�gJ�����e;R��MI�`���/8��1��Z����C��;���\���:ga(�B�(�7
������ G`�� �DgL?�@HI9>� h.`p�g�Z"��B^�� vBr�%� T m��b k]�=���� �MI�`�H_ ��~�����fsEEE�D��)<�s��A!�F=�T@ Jv�xBA�����&��<H����N�4$�a4� 1�c�����D�6���-�E����_�O�w,��d� �������_�U��g��d�����|�6%E~��� �� � ���
+eB�E�F���=TK��
f�Y�
������!o%�]��g {���fR�x���M�'(�p ���z�H���Z�!�#u|�
@������`SRTb"�q����j4�
Vp
B����2!����s���%�N�VV��QU I<�h[����]'�l
:.lJ)P��"��>I�$���y�z�� ���-[sNm2�kW�&�!����x��E�+�z��A� �>�a��E��X�2�v���K�:� t7N� =���Y���-8��B���wuuUVV�Od �0evi�#���#�4�4/��� +eB�E�,=sp��%#Q}$��p��������i�����a;�� 1�C���" #|R����_�R� t7����PW���6�D��&?t����.�ow������G�0����U����i��&?��9E���!
-Z��������NJRI����SxB�&������ "�J,z&��n�1���K�y��=��6��>`�� �y�-�N���6��Yi��<�_x�+]5��a��hK�Vf�+�t�W/s���D&,����7'��4 ����~�} ��tn_r���X��x��$�3�Ez�$�U��_�gKg����f4�M�W�C.
��,Yr�����~�l�D!�p���D|��%1�\B��hs�d������%#�����&_SDC�]:<����= ��2.��#����8��$2a���-�W�L hn���&,���k����tU-o��f��� ����5i�o�:Z��8� �w���C����KO�>��;�`"�JL���;�/!�B�������!��P�n,�� v3%�`� ���Q���9r�%�y,;�����B#��W�2��`���i�V�7p�@�����BfI��1F�0�/;�2G�w�tl�a��e����� �P�p�/B!���4��3H�� 4��d�c����4�{��g�f�Gz��$��?!aP�>�d�|[����{�� �*�!|��[�F�$�,o���\o:�)��,���\��� \�0Zh�k�J���F�[o�����z��w,K�>B�xX)�B!�V������z;�����$���Mf��DYD#" <GF���}!���eP�h���SK\{W���|D5���U/`��e5�����^h���AO��#����������f�qV���O��0
����@�4�W�c\����y6�����3�*
���4��;����������D!����2!��������`G�'��+��� ���M�w�WY��o��/) ��@���?3;M ���l3�*���������UP�g@���W�2��`�&��`�N�^*�#��h �����
m����jB]BW AwC��%�?�]�����f�a���J��� ���g��L�k���m�z ) �y�(��(�w�u��C���}��>��B#�2!������� ���^�H��!&�qu�]'�:C+�Zw� �Sff�o���H������i�d��P
��� ,�E����_�O�wL���I@�S��*�����H�2��Q����}O�G�D���q�/�(�]���6q��~%��m���F��� ����lFSwq ���p���SJW�\y��ALdB�/�} !�:�)'��� ���TF�*�N�� 3MRR��L���C�����S�3m�+���� ~��
�Ro�K��+-y�@�x>P�TGlAx�����A��4$~)�nh�U|��_w4� �,�E��m��AI���t��e�����WR^k�h���Ra^����
���` ��I�^�{Y��)�w�}��>����1���B# V� �B#����)QQ���RN�')� ud�(R��U�gf���I���k_t����`�8���8&�n��\5���
�K�� ����~���&e�������~^��c�$��6�v�����Y�S���DLw�%�z��vG���iX,c*��������_�ly���V<XY����
R �T�t�+�jU/�����?�����m&2!�G� �B#�r@L�Q,����j��H���@'�:�;3O%onBCKu���q���*�� a�3�,WR�=q�d l��%���A������ ���p����u����y�(�Fkt$1�2�MD�]K������~��}6 ��go��������3�[������>��������~�������~!�D���B�(�1[�t����ff�)���e��x�_���a���V���O�m�Y���j����T���Yi�S1�MQ��:
����hl;b� 1���.x�JQ8S�8�E��$�I��1�y ��� k��KJ�$/ X��B��{����B=�(�(5��{������:����]Qz�����Po�RW(x��������3=*��<�c���;w�5*�w�B( V� �B#���Hu���<�P~�# ��;�zC���t+o���?��"����d�M��&mq�
��D������Q I�^*�����G�C���m��� �q�3b"#�� �~'& ����}O��n��� ��l�4��f=>�� |�s����G���!�"�P!�QT��4� &�P���&�(� ���g @�N��f�1�OAe}�8�&�E��Zw����F�����\Kn�F���y���J��/���<��& �q��!�H��?�=������z(�/�Z�j��m�� �P���%�BhD��O����0( ���`�T+���%�~�.nV��A�2s_�B�����|� ���c"������*Z��H���;���>���w���$E9��k=@�B��\ B|M�m�_�O�� ���N~�����A�H�I��=�5��} aG, (�+�V�s[��=��������/�^���w�����������!�X���M�H!�=D!�X�v��D9�<g,3o���� 3D���f^u ���������`��j����?i=�� �l��������9�0���q
L��E��[_��Y��� � o�N�Ve��TY��q<&��l0y~.u0��,�/�m�7��W&�4r%�9+\U+��&����!3�Z����[�_�
��5�`xq�-��`��5o���G }4f��>==B
����=�ix!�}k��Y�v��= B!�K5��e��LV�b���@���4�#R�=qspw�oC�k���1CV��p�i&R�O���.�_-oN
��)��-eZ��Q����� ������O�� {���k��e �� ����m����M[�q�#c�E=���������XO�+))y��W����� ���!����kKJJ�O0 �P��3lC!�T����t2�)�I�/H�F�M���[�� G��H{ ����������+fu�~����~��A{��$���Y��e� �Ne���{�H��mFS��y�yZ��������?�|UU�� ��Bhh$�/���L�Kb��"�B��M��39j��l5����K����Y~�C{7��� � ���� �|�O
JQ9+\;5���.e�i����sd)��s��_j k*� ��N������5��c��y��q�?��6l�D!4b$�/�8�!����03 K��g�SF[x�iJ���"���~�cF���OZ�;��Cf0�-�f`IKbS�V�I�r�
��r �US,�t7�j�����>f7��)��=��l+s\�O��tU�S����������V\/=|C1o:�`�~��]:.}�����`��� e � �����2LdBh�a(�B�C�
Jf�|3����5j��2��& � ��<������T�V�eL��F d��
WM11���
t��a�7Yr������x��O�}x��X�5�==y���p�
��9(?(�'���Y�� y"#�}�t����Q���q� ���RMa�����������[��o�/,���P��|����{��O>���ywB��pC� !�<33��_�H����[��'����k���m��6 ����S�p�d��'��bE���>E�������h(�z/����*j~u�����[�>���%���^!$lC3
t
�������]5�Awb����p X ^n��3�i!@;��
�v��Gz���`����k�qW<<N��}"J�"�7���M9���C�r��r��$Iy��w��< ���+�6����V:��>[!��� � ���� O=����>[UU�� B
eB�s�U�gf���Ifv�gg���#���4'9I�'1�a�d�yr�d[�>8J��J���<.�Z�����M���9b:C��?^ e�L��s�%������TQk!:�/L�]���6qF��6�.��6F����� �����F�:�h���PU�Ig�+��ZR�����fO���
��E��>O���FO[YY��O>�}������|^�B��} !�:�H�C��������>yK�|Z�J� @�B(|+O��K 8 f$q X ������d�f1s9�XtK) 1�h����4*/�}�]�b ��AAO#3 ���$"N��/JU��tE���M���S��:�^G���)��rJ�Y3w�}s���A*�|��g���9��e��~�� ���� [�& +��>��� ��o��� ggg��{@!�+eB�sKu�o���eI�������'�xB�]�.% �3�(�+����b�-�"F[���R�����3��)�K
���k
�����.���Z)~�m �<_5:�~Be�w����X� �� ��r e������{�[���'H Ra�����y�9���3�.�����U������}���q: �����?������z#!���R!����.~��BZ� ���^;�)?Y�I�&���^o�,�P�8����%o���6��, ����D���m��[rH�a��W�j�{�x�s�]�~#u0����D*���g]G�jy�Y3C����H��xfzo�Ly}����&�{�o8B���l��G��J�`fq���0�=��oW����0�A�A�� B!4�U7�6�l\j�D����I�6ak�i2 +��$=�j��f�
��M)��4�,���F�n����N��E���H����o�r��N�E��
J�P�r�F���V�h������'_���
$ a� �9(��B����_I�}.����kK�~��������vB��R��b@~B%�{>imV�f������o��zV�������3|9��# �s@�h��FC���I��:��cR��Gt^.U�S*�|B���tY�/DLw�����D'�7dJ3}���tU-?�,S�t�8���U^_�����<�����NW���{h�8����w�j��3c;�w�s�@]G���$��#��� y��� |���>�h��)�>B(�$�/�X)�B�Xb���:��7"
F=����.b}�r��w&��v��4@�y����L����$�\���Q��byEw4�o3���<�?]��b�A�k o��G�T (5X��G ���u�D�0�WO������_�gkg�
��F�c:�/�=��m�Gy���p���v�� �O���^{�� ��s'&2!4Tp�/B!42I3z�_"�S'B�F.��8�n�o��1#��L����J�^���RG��� p p H�(_Z ,� ������ �M��bT��@�@���V}��� �D ����\5����� [��z��
@!���_k H���:���@LdDAH1�
� �����?M� z���V�[��q�W�[�{Y�����_�����o�� ^x� ��BQa� B!42)S��'E�' �]3V~D9MF��tU�Y���yi��`4���0������>�:l�L�K�
�p k��2� ^����D��� ��9�����3�w7�����V��X�d0�8����@�e����*�;b������[����������w�BH V� �B#���^��%��nGR�KJS ��SI���� F���W�������g1���
���S�TT��6W��9�ZsE�8�����! ���v�����
�{�/ �&&��^9/�>�����Qb� �����s�� ~x�E
��@!�V� �B#S��?>�W � ����}La_���6�9���N��L�Q�4.�N��;q���������-oLfN#���/���2b��r=�j L~a
P6�W1w&iT�������[������Pp����y��E�E����|�������?����� �� C�BhdbRy�23�L ���PD-�9���� P��.a��Baf�Y{���K��%-W���.���
u��I�P����b����i\i�� �����P���G�5G�pQ�/Jum�@x!3�W�������;���K.���!�P|q#����[�B!��H�u��m�O&���3���8��1�}ix��W�j�����L+S
P�� T� o}�k�2*��q�q o��J�����Q6�}e��K^����� �8#?��H;:�����m[�l������o{��!�Hb�����4�$��+B�sD�f ��n;�l�6r0��+���f������jysR��N~����~������W��SD����h�<����M�3-<� �#&���Zy� ��$WQW��:m�r�����>X�t�_����/�| �!�Xb����~B���7O+���� �&�-<�� y��/�#} ����-'�d����tvoL*������>B
���{�V����Y^_�P���P�@8����|�>����K���/�D!�
�2!��p��yZ��]'}+�Z�����j�uR=�Q�mg����I�����,fB��[_~�r,��"�m���[����dj����/��>���f I����mS�s���� ��<�<Q~a��� G��3G�ivi������� H�/ �����]yZe���v���>h�
�=Z�RS�}��%K����;W\q��>7B����_�Bh��Y�R����3F,���F���L��`F���������� L c�!2�����US,����p��^���
�(_K^!*M��^��,�\�}�mJ������E'������3�s��ma���Rax���� ��9�:���}�t����]��}��*�]��������7�������xT�B}��2!��p��bEgA��@�#����1 ���+h(�?����F*7��U������ K������[���H����;6~�?}�����(/��V5���(|�����=�,o�����
�����
6����l�4��R
t��f��:�����k���o���UW]��O�B��`(�B
W�'��|o�i�����?���%-��&��z�
����F*4p������
�n�[W�}��m��E�}�(�y�=y�%�0ez���M8����2�T�}�'��) ��w,:���Q��;������T#@�e���&�S���p��j'�A����������k�]}�����!��
�/!�B���L3% o���4�f�O�/KR��h�2G(��;&�6*���!`�������Z�����}`u�{QA�Zq16���q�I4��"�l{������ j�� rV�� G�Fa^% ��$��+-f����Kc-������o���W_�5k� >8B��%�F��%1�j!�vn&��&����2 `���K 3�g��������� IDAT����[y��Q��l m]ah�����]���ib"�#!O�l��'MX��L{e�����JW�r�I4�3����(��H�� �A1�(��I��GF�l�*��/��E>hw,�d0�8����������n�����w�;tO�B '1y���~@z�]�v��!����rk= �D33��_�H����[��/e ��~"��<�.�����J����=��[���%�u+�+�5�eJr � @)����)����K~�D�'��{��bZ�420�� ��qF��v�z~�O�D��?�qEE�F"s~�B���k�J�����K��hxI��
!��� ��P���J��6����&�M���Z���8� ��{������&���W�r�'����QQ_�\����`{������2}���^o]�k�
�A�|R��#]�RbC��8dpLJ����=E=�5���q�������%�����(�w�i��
��$��K�}��'D���f�t�|� $������kjj~�� ����7�pC�;8+]U���a=7�,s�w�!�$�/�8S!��M�J-�3YT*��HI��H8Ne�$����2���g�s�7)G��zR�
J����}�
��K�)U�Z�5+�q4��)&�j���\w/�n ���� �&i�vJ�o}��-����|�����H�5�h��i1Smm�M7���K/i$2�Xw���i�P!��
�/!�BC� V����'�N���8���v�Q��K�Hd �����s�7q&6w�,c���� ��lKngJV�V�k�����0��9�X� e�S��� �_���X� ���}%��m����m�x��w��;�*�L�W�-�A����Uy}m��GM/�*g�����}y�H�����?w��
6��;W��uW��0�B� +eB��� V�(����Jz����tD��J}����m��A{�����Z���k7"���U Aw��&��GN,��W� %�N(b���>�\��l�pF�xC��T71i����*����*�ip�-���P(�+�N��)����r 2��L ���JKKo������\we�h��~B
eB��� V�/�� y�"N�%���v�N�=��+D8��2+g�{��HY�B�mA���g�V)Q!DJj�D�����:<�����;���=6�:�����Sc7�8�����.�K�f�p] �3fbF�v�k�o�B����RQn�<�9p���9s�{�� ��z ���qU+�f�\��8="�� �BC� V�bw�F�N��B���" 0r
Q��I`W�oJ��/��c�_�i���]��%�Q��6(��gc}��F�)�3����;
���7y�/��,�zseG�����v�{J>F�ql���7�D� D��W]Yu��vQ5z���@���,��!�D��g��p�
�����������T�O{dl� (�BCC�Bh0(���`��t��O�����&:g�D*�! ���/P������ ��f�Zq� ;���}D"�li���O>S�q�\ p�O�dr��}=��[w7�� B]T\�$����{����U
�Hw&FB L�|�Kt `tp�.*t���0� �7_�i "����0eIbd��������?���?���bzrywB���� B!4�"u1�J�h�<� VV���dZc�� S�#� �V��Yi ��f��_+��ks61�]��W�6��AC���&�j��b���_ ��)�X3�q�\�1�N�_e_ �(�u�H�yy" �W;�\t����;� D9[GueU�e����*u0��y��&9i������p�
�����O�[@!4�0�A!����"yFS��+�1 ��sJ�T���Q�����J��y����"�p�p�$2�������[��)�J���X)g�F �5� �P��kv����; ���A�����P ��l�k�i?��
����s���4F�~���\s��~�� ���#�J|�!�pql�V�q����L���:����#�V��K ��^�a~fv�x������mf�����V ����~������{6��{6E�5��R�W �+J�l������ ���[�<T����o]�[pl�C�%�;6�!1a���u�?Nvd�BI]'s�\��B������Wq:��\s�c�=VT%B!��0�A!�\���$j�I 9�HL[~�M <�Y����]'}�N���wb��B�Yr��� ���>���v~� 8����"!o}y���S�L-[s���: &�0eviF�UN� 6�����8���=����:��Y��)���thS��Y�w\K$�,���H��N%�]g3��O���X�7Q�B���=����_�KG� �����?��? .��M �J � �B.���8r Iu�o���O((�O�����w�)��"4�N�VV���R����2��\`�I�7������<�����7��`�ZB�Z�����A��_o}���8�n�B@������� �E������ ����W?3aviF�z�D#S��:�W\����T $��`w!�W^_�dW�r�u �*�+(�r^�}������&�dltH K�� 9���~w����-��`!�P��2!������Z���0J�N�� l��t 4�������3�����n�� l� ��K(����#`�~S(>�$�`�q�q���|{�#��k�K��EB��l��I�8��/�t�����g�O"�8���T �jE�|d�����/���Y�� ����5���� kd�H��%��M�����7^b�����8������]w��U�V�X!_�$�
�i#�J � �B�Au���8r ���R�M�'��h��~�H�?��� }\� ��B}!�$m_�~�3� q��
v62�9sz�������'����&��F�k�
���ous�+��� T���C��JQ*Pj���r���&)���� ��D���sY�H��inx���/�����B�����t����������g���/Y\\�|�`C��0�A!�TL9��jZ�M)�H�(��@���5�B}$�&��G�)U�S � � o�NT�v1X��:A�������_���+h$�_�
A��Kb��W5��/�U��l��
�1-�V�R�;�� �J�!p��)�o� S���F��.Q�~R=����w0�m:�'��M�� C������_�q� � &�� �B C�Bh�����Y�� ����#`W�� � ;�qo��P����9��f�:��l��7 Q} ��
|��z�������yT3sgf<0 �h[� O���n�S�W���b����('o�R�;�S�5�h���Oh�� / �����uV�(���F�M��Mz�8����<��=�M����f��������N��� vB% ��B
{��i9��Q�����Ic�;�������4q����W��o u� ��2� c�t�{�9(�S������m��B�B]-�KJ7m����� ����M ��Q� �8�M��8�M�n��)�!�� �&��X��������Iz�����:��}I�����_=�w�M�,
�� x��[�������g���������+����8���B
-Bi\�W
� ��!�����t�')U�F0�d�%A6��/P�/�+d��zV����&_������_�
�$n�7��y4���1B�+x��P �IL% �U���9)�^�����| ���m��z�b� �Y��Q� �dM{�`i ��?%F��O �l��z/�q&;�i����gZx4_�t�d'�a� ���/qV������.��ovo���
����$����[���"�&��������3u�+j�J�q@�u���N;k��+V������0��`�B���D|��%1�\B
#R�"��e��H_ec��w���c� ���w
�=��6��,���O�R��������U�\�R���&,
�J�\B���iem{nU�)Q�-e�q�o�k��<H� �'9]u�J���.��O��������QS� �"�8C�-�e(=sa��m{W��l �N&d�5�S{:U/ � e�i��|,��>���9� ���U~p�c��y�+/��)7L�����U� �c e.)_�x�����!�P ���D|��%1�\B
�������p��3���8��G� E�D- � h����z&#������D�E�p�*Y4.18&�r�u4$��W��5_�����e���P�����Y� R���t %��>����(/a/d��,I=����D�| >���wQ�o9~L���h�p��e�_�
�%������-R�
���?�����!���w ��B
���:v ���� h��6 �H2y-�|��r;�9b1 x�`G��SY)�����}q���Q���#F�}��Q/ zS�|���VK��!97l������(�Y��2e" ��F����V7�&2 ��� S/~���J�)�+���C r/�u���M�-�-(�-PV���LdB�\��~B����� ;��i|!�����*-�NS�%cm���:A��%*�:��|����63�������;h �,�gg�g��uS����Ea'����3�Z�
Sf<� a_���������3��Q���\Q2�`���`�f���}�Q���D�v�>��*������{�������r �����w�8^���]�r�g���q#��0�A!����Fo �R����6�Z%q5��I��Q�DZ����u����ff���K �V���t+��{�P��O-7U����|���y��/���;�����M<b�-W��z�f)�I��|����O�S��>��-v�>xj��ek��� �/��<�!|R�U��M����uH�h��x�=��#{�Vy�F�K-T���_�gk��l@5����ZQC.�(�+(�r^�}������J��� ��X/?��B!5��0�0��B
_��R5�$f[B�����NG��GJ���sy��|��K#`� �����dG���Pf��|N
Z�UJ���
!���ExE�Q������Du: 1���������pU��8SF>�w�L��[��y�����g�^s�57�x���k��B!m���;��A!���AP� ��)U1�d��v�B�[.������Ny�`�a��\�P�j e�%�(jF����H���j��'�t Xc�+��$�a�r:�r R��%��^J
V.� �<��4/��6�d�Nd ���� i���z��F �+��
�M�Q���]{������0�A�s
�2!����n�e���&���-! }��8�*�<#�,o/�S�u��L���o�1��j��ZR^��8��j���
�����q0�TGG~�JWM���
��b���W���~<�)�)%,�����bp����*>y������JQN�C'��i���j��7���z?O:NQ��`��p����r�k?�^�Dt���}��XC1��5k���? � ��p� B!4v���o��)m pQ�)�qW�O v5�^��c���OH9�W9����:���>�r���b)��iH��@����x�* ��?�||/��|������. �_.�P-���0!�h� *F0��~%/� v
���f��[���D�Y��*>)�����s�������_���Bq=Nv��?�h^Qo�M�������f��\��������~��'b}u�B# �2!��`�n��Q �= (�~ x��n��hg�7��^fS��m�#�Y�_y9�XJX�����9���"e�����K\���e�|�|��rC�6f�0 ���G.��^V��C�P��o]�S��2*�gk�[E�T�'*���|�+}j���I�����eU)�Y�����Df���� b}i�B#�/!�B����"��h����#���4S������g�����J���95hEj�&# �@���s���+_O ���I���B �]�*�y\>3X.# qV���q��/��l �0(�z�:����� �H�\F\��GI�!� �D���Z��c�Iy}���U �9����������������� !��H�� B!�W�N���U�2eV �A(��[y��Uf��v7�6��T�e���#��:C����i�. �Y���������4� %q��W��!�%?� ����99����%|Rw
XS�>,~�_�*(�';]G��k�c
~� ��l�����f�l��{���g���� �g�{j���C�-&��#~zf�tvv�x�����71�A�s �/!�B}"�")��Di Lr�4]�e��j43�|���t+o�H����R��L3v*
#*3w;��V&S�ul���4�h�H�_4(��0!��D�q�����xZ2�����y��5��RL7����X7avi�#���#�4�4�;��G��h�()�y������6�o�m���I����lFS��4���s����� �{�9Bb���Bhd�J�B�O��0=�M^�"R]�}U�y�h��V#e�K_:�b-�A}�VC������-�K�M��
**J<b��$�O-q�]!E9��B�Ui���X3W<b��-2�L�~kf_���F�����q���d ^���a7jS ��Q�|k��l��������&M�D!� J��u�M��Y�v���{�BC`�����##��D�L,F _{C� ����
>��|e��P�/qh'M���u�X�apL�_����eL��I�3^��.���^fU�!9/����� C��p}|��PW����t��u�U�S���Z����gZx4?������v�?� w�e~��s� �����b"����a����u�pk��-))�>M� C��"��!tN��'����q6��YiC�<���OZ����x��0S�O�E����%!�e��
_9D���^��y�+]5�{�\�$
�7�hK��!�J�� �4�{� 6nvY�8�E��tU-o���Q����pU��jCf_*btN;���������tu�u1r6�i�������x�������1�A������'�3
/����Bh�('����1#� C��_�_��/-oL�8�W�O��� -oN
�����Y�q�� � x�|����L2-<V��y��U��>GU �5��A���2�,��c=V�}�����}���� ��������zo����'��� �� ���H�_�q� B!�'���T7�6��5� `Ba@;�T_�*����U��W]����j �x0�9@�2,i�L��%�%��6n���}`�
$Q�q�,�x7X+��Mc�U *���W&2 @����H���������3��������c �Bh�P!��+�
�q �$R� s|��_����qC�(b\���T�V!����Y���J����X<�qE�8y{Q�8+\���~�` �8� kX]�� OdhOhF�
�a ��)����c��~���y6�m���<� nj�G!4�a�$B!4 ����'�*��i����{�6�� �������8 E�Z�� TE� g0��S���cb�V
J�g�����ek��u�[_ ��������t��Q��X��)��,o����8D]� v�����/X�
���ge����b"c6����
���7Tc 8B��
C�B `�I�����m-+�Zw�����i�~��:�T+nT�+��*������0?3;�|N�3��g-�0��?T�8��(P�-;}�1iL��U��:�W,� ������bo]E�W�/J���NY�B����cQ�z��M9N�tvi�#�Dx ��{ l�"�8�� �}j 1��w)�|�$��������?������7�L���� ��� !��p�� B!��/�O����T$��VE�K�'j�8��� ����28f��� �u ����6 �[���z�R =� ��M�_�TQ��f�����
R�>��p�_�)����W�2��`�&�Q^X��g^������,��'?1 Q �g�A��7|!�� �B�Q���:����� �U����Wg��z
�Q��z
3$����zi�} d�Mb S� ��_IH_F�0�Ij7��i������#KnQ��c����E����p��T�Oc��P(�h��@ PQQa4F�W�� !����A�!�P��a� ����
a�kG:��H1s&�0���`R� T��L!@(����t�q���{C]-����\,y�@h�"��%!k�u[t�� IDAT26�8�l��/LJ��:�����I��}_KN�������v�?�5y�3����k��2r�L�P(�p�����[�����'����������� ��"��9C�B(�6@Kk��F"(B�1V zVq��/��K��"�t����� ��F�����{@��G�#��"��5� e� �,����[k%�}j�k���
b��� b����G��Y���@�_v0)��}�-���\RY��x���� �g��l%���G;�����/�i�M�Ph���mmm ������b�S�9���u�!�� eB�!� �l��(l�y��.6�! ��qAJ�BG���`����u�-��!2C��?�)�>������^�c�H�|{�o�9��#f6�<�����A���Y�0��#�Jw�
�
��[r�o` G��.#��J'���@LRD�����K.�nE�V�q ! ,{B�8
-Y�����������u��h�2!��M� �B�U'�d���|��Ejk�n�I�8c8�@�n �l�( ���y2T�R%o/
����(�>T
C�s�'� �Tv�1cv����)�����_}��|_�=����5s!B�3&F����%���������"Q�P#e�N%�FO[a^%�������2�r����B����K[ZZ�y� ���z�� !��HE�J�:�!�=D��)��W�w!yB���YB�m�o������ �9ET���g����&~�p��@�/�F���<�������)�H�������K������L����%e������Y.�\��R�����sS�L ��37y�|(^�Ex Bh��h���<��� ��'��51Q���8��W��
;�'MX��mm����N�<G��y� W���Z�zW((~j����s���2
������;��c�X��!���K�_��R!��H��7!f��������0]H�B����rfLdL���(�~�.�4��W�'�='��d)��DD(��4��7����'��!��P"?s������ ����%%@�q�\wc�� ���������F�V#{��������b�t4���Du�F�P� 5L���h�~���-�6y�S�f ���p@�8�����^��y�������-�p�����������fB�<�
/��!�����OZ�z��6�����A{��L���f��a� �[yJ���M� X �'H�i��&�|!p����.r���4T7���g�,��1��W���W��JD�hK�V&o�������&�������������7'���G���m����<�wW]J �r$4��o�����qV��11�7d�lJ�<��� tR&� ��h����u �aG c�\�vFO!Ry}���x��� ��.!$��HlF��-3/��y�(� �B-1y�J�B#� �����!/zP
��%���7��Rff��8�q�K{n��F���~�K��k� �<7C��D��K"bf��Yy�H��yu��*�[�Nl\� ��e�-�K��|_��{����es��;�&�
�1�hp�� I�w�����3��� �����.P��o���P�yGE�� ���� B��`�wZ���2��j����oS��� ���<��d�3���� @m�DHp"]��%��MI����F�����w��]�Q���\9w,-�����P��_o���f C$qonV�>��![$�Z�'2 @�� E� �,`��-�����;B!tn�P!��H0�;�U_��#�LR#��:�@{���W�P��s����%��y%,e��}\�D "'8�k������
uA��f�y(RcJ��!q/3�E�Uz� k��4=�s��E7��<�g��1+���W����
��H���h�+Z��k���)'��{I���E��?�m�Im�>0�Z��6d!�:��
e~��_�9��'���� �Bq���z^����q0L������N\2��W�b�
1��S V���Ros�;�I���mW����W ImJ�l����4=��F;�}k��PF��#���{��]J�;3j��+.c�sV�"M��odo_Z��Ts�l��G�5gm�vf��Dk� ��
�eMe��d��}�������e @k�{
�7���1f!�b�
ejjjJKK�����;��yB�8
����^�Ij��5 8����UJ�.� h���,���d�3��%���K��s���v/��y�+���8gE�������<j�Pv^wy�3b��|������tU-�U4� Q� �}����O�27N����^�q��#��F��$b'%��dcR��'n_j���]�Q��� .�����:�Qo�BE�+�INN������II��#�BCH��H�\��b���lJ�n2hq��g^�L��of"�|#������{!�>#V4aq����T��^4�����2��>qH�8k�{a]H5s�#N�<E����F���Jl���a�f( �1i��7/��L<MY_�p��� '��U�W@ D�rz��>�p���;G�����U
!��JW(�n����������� �B� �[������"��l,R6%i�d��%�@���r#�k�R� ��%�������z�+�.Z���fM{�`I��^kzS���D&�O��S"��Y������Q�& r/������0 �3�'�� e}M��V��(���� z��>����F� �zC�U
!����s��)S�� �Bh`�JaG�ww��9�/#x�O��y�Q�����������[_ l�gR�|�^�BK~����� Ru��%� 6 �q�3�% ���S��*����t����+���dgy�� �z��M9Ng�J�H���.�-86�!��� ����(�VJ���.}Zr�u6�I�4����o@OB�W�Z��{����SLd�7��J!�Bz�
eB��g�I�����m-+�Zw�d#
��E����w�����i��RG�U%�9g�
��4qe��p�E��u� )���������&_���/o��sj�i�i���� �&�h6S��&�M�<E�9��]��1�7�0evi�#���#�4�4 ��Ov4�� O�l>�����@���6g�����������,��(y?Qa^A���������b���+��7G�R�������_����;w���)oh��l���+�I�R!�� �!����/�*��E- �H G��MI��|��h����i\��O��;��]�m����c� ���O��l����;���c��=3�Y��,��V �Oh��J�v
$�N���T��7OrFmM������Jii����
����'��6Ov� X�N�Z���9��T[� `��5[�l���O���|)R�B!�Gl�2MMM hoo��>������_{���y0�Bh ��� �Uc���R���#�V��K 3��>�~�/�3��*u���.��%�'1'p�����)����t���
�l�
��������9�)6�k�SK����C]��_�K�b%�(u��������/��t����iMR���e%6��R�����)���R�~"�b�� {u���tw�ZR;��l���U���������UUU���)EB!��b����q�����~����?����z��G^~�e���l��z>�Bh ���"��CP�Z��o�2�W�2 �w�-��T�:�]�g�u��{�� %���8�c�|��hIt� �4�w;��_� =���� q�������z��8����: �����&1�5}������|��lq �X�~���j���)���$V ��ykWR���O<�� /�PUU5~���zr�BH[(���������p<���K�.�5k��i�n��F eB
/���0�N�����N�<��=k�������
�'��>���wEo�
����@o��<b�-J������]�*�i�9�@~�H��o}%P6@����t��y���C�G"�JQnP
g�+�j$��k���&�i���O��tU-o
v
P��J�E��M �^2 ����?l�����j�� �NB��bk_�x<� v��}�����lNJb+oB�k�
�qI�] v6x����T�`G�'���(�3����������L/�:�Rg<?v��������IQ�%�(}�1�`_��f�S���w�viHW�
D��K�����r
@�_Z��@_����#�
�������h��[WZZZUU������!����R�����mmm'v7E�����C!�B}��D%�ve���h3���� 7zo���LU�=���`�������F�I��EQ#�� �
��q�
=� p���{R���d���e�F i�ogS�������<=���=P��,>� ��L9��P(�+(�+��N�Lk��� ���Kv��j4�����~m��� .����#���X��'�\�~=&2!�Zl�����w�����)��I��D!�8�7�����lwc�������N3� !v��5+�1� �:{��[���<�1M���{IZ���)J,y�1��3�R��&�u��l��a?�q<��g��_������<� /�6" Sb��u@���[��z�R �~�}�����������
@KW' ���W�!��7Wz6f<0 �dZ� �� q��SO=��SOUUUeff���A!��b e�|��+Vx����2 8q��}��w�e�
��!�B1Sm���4f�5�����>J������ ����|�>��FJ��YS �b���H\��'���'�3$X �0�lB���Z)���<���a�*�F���}�g_���Ry{H�� ����kU�}I�?�� u�����������|����7Wz6�x`��|� �>��a��O>YUU�����o!�b��ddd�����������w��b���B!��4�?�N�^sv�
= �j�V5z�D�l �@�H Z�=R|��� `�1+B]��QA��C�a%AO���3��{����[��T��{����@+Q6y���9����(1���e���X ��k|-A���|���y�T���m�D9� �?{� Ey�q���� ��I8�CB� HU<@�('�h�����`K��U@[���W��P��=X�*XI"B�}[O��5 �(�!@X �d�=����$��g�M6!!���6y��g��
f��}_�� �v!Q4 iq��H{��Y�~���+Q�AAz
"���Q -K��.Q��k� ���_�:O5� w�^���./XV�_++Z���c�������qB`������ q��n���0�B�YUI�c��H�G=?[��7m�����#�S��Vm,��lM��L���Y &(-��v�Y�����)��S�)y���!��No'&WU�ZL
ZMf ����p
9���j �;<e �e��Yb��k���b���;w����l�� H��o>�GW)�J~~��-[��� �t��4��:S�P:� <8���t��Pe�LS��/!��`�N�v����.1T|v����'<��9~�����1 � �T���0�L,�a>]���
�jN���|�R���FNOx�HUWH���\73?#������7����K @�5�k�BSMDSD2�d\��!��������u����FEA�e �2w�u������mAA�K�� ���PZ �������u2,��p�r���,�������cDm���&:�� \�[�$ Oe0)�i(� �: �:J���<r�W �#gM4I�I�^�h�����cDT��^�xU��"{FY}��#�$� �7[}|XL_j �EM�!m���'=;?=��`���P��6<��[�w��?�8##��� AAT1$��|�m�� ��K�. =z� w�����}��={�l����7� � Q`$VI)^h9+��DX@���U,��|���]��0SRlo m�OS��?�A������hBX1;y_}g� aoM���w�3;RE�������`���I�H�1�����u;�]�|D��� ��"�z
i�d�6�l�z�;F�JGj � ��h���k1&iY���av���'�#%�����8m�*����033��n
AA�0$���f�������?����$''�\�������="� HLQ/� ���K ��v�O�� �Kr� j�i
��E�MU1�'6������A��m��D��0L��� ��7t�) �d���w��Z+_� �b���
j�������v I���F�|�NuMU������R���b�2 �DKC���'[A ��Y��H��J<y��s�����9� �tC���7�(~q�������M�6m������ ������"F,fQ�C��p�� �"qFu�)���2���A�E�N�0�*� p��z�=}^���pK-"*5<��c4#F)�[i{x���"��k�F�V�2 , LV~���������p~�����a� E���8���6 � v�/�9J"Z���sA������U65�DA����x1k��f��+���eZ�mu����I�s� ��������K��>EG����D�jm
�{���"���Yc �� ��h�!
mEUm��S�Tj�j �� )���y��B�S7
����7�Zk�KU �b'
/^om��X �\T��S�������deL"��P�=���E�i[VZ6<��eeQuu�`�t����� ^��JO_ H���$Q|)�0�F:��YM���_kK� �sD'� <������c���\��n AAz�x�2SSl^�J�3��gz��E�OY���fw+�����M������J��-G;ol��b���s�<l���# )1�����a�)T7+|�E��ig6Z�[�|�E��T0��J����$:� �?�iE,�bI����#TM|�����&mO2[ P�bk�+���xB<'��WU�e�@���1�A,c� dR�D"���y�����s�dk9�[M�6�I ��,
7���mo�I�8Y��du ���"�R��AA��.�O���y���� & ��/^�q����[����O���/���.�^A���5����&�#�X�2-�6-�&z ��E��oZ�f%#�-T�vli
�'JA�3��'O�B�i.Vm�[���1�5S�$WR�����������5�b��7~^���:��rT2�����), �Tb#_�1����*M^ ��//�{����{|�o� c����3����q)PI�rm �d M� ���;?���?��=;; 2��� ���$��n�+� r�y���
.�.� Bg�����[�bEMMMsssjj��+.��� �\����_CA�����s �2�a��������F:�.��%)s���$��M3��G~��<���`e,&� S�z"��CC�� ��������;7�~'3�T% 1�2�~|��L�� @�D!�$p��A��0i�����XC��&���� K����o������v�`r��P����S�-&+y�? �6���>�q�r/���R3 K ��<d ��'\����/%�E �5T
Q*����a+q����r�4�����j>/ I6��_���m�u�]��%�������oA�������*e `�����_zb+� �M�A�Z3�e)b=�~-��\*h�e��=���A��\bI+J��H��� @��d5Y���JX5����T\cOo�Z��Y���� (�^U�J�h����S��F xh> B�4b$�V�S�Q:��!+f9�rA�������%F�����W��Ei�{:����K������A�5�NvP��_tA�&���Q���0�K��
�T
���P�-qF���*����F�3��H2hK s���Z6.�F�mpbI17��{\��
h<e�&��%�5;�067b�:����
GM'B�*�XoL��,���`b�p1���F��+�W�x�\��4��� <'� b&������(��E�����,����b����Z�F�%�q�O����0��We����7%.��������0AAz��+e��=����i��g�����]!� H�1|���R, �y��/�\)P��� �K�S�9r��9��R ��p����l��a��g��s�%�����( �eA}�-��(�����>"bI�5��f#U_#7��n�:��a���U{��L�u ��H�����&��*Yy ���m� I�/�2+U�� lIf !��;R��3�'�{[�B��M ~ ��
Q�����q� IDATLW�*_�Z�DG��w��Q� ���b���-bOSTI��������O���tKJ�`A�!�e>���+V�s�=N�S 5�|A���T���a(y%� ���$��T6�>��KV�� h���w��N�>��x �Z��j|J _JZ 04�D��^,��b��8��3�������HN���u��eN���r{F �"�p���2& /]X���A���[�����x�I��%+?�2|Q��M���9�� �vX��huE����l�5�S P"�e�AbHS���^7#%a�u���������\�����V�s��R\�-!� H���MNN�?�����x������M������W� �%����f&����p��%c��J_ pY �����h�Ai$�7��:��S��hsk�K.�K�����6��p���}�'��#��+���G�����O�������O�T����.O�%�������il%]qp�/���z��� ��OG�Cf��.E `�,j�(���0e��`-�&�ZV��2�a&8�ORO�N�RI��{��kd�O� f*!� d���{t�2��Rd ` +2� ��B�� ��h��7I�2��� ����W6��w��CM����V�\c�3j � �u=�V.�Kn_`�)��������K��>�i�}��2�aE���I�T}��U�VB6Uh#v?��0���uW�����������.:UY��,j�|g& ��Q���f�uT]i��r
�]c, K\c,9��m�L��t���� ����b�F� �edN� � �2�1�A��sg�OP�jQv�&m�J������,������m �-?�A�w� � �%�����x��o�ET{�^y7� ��JU�%�/�6�k�0J B�
F���f1eu��6�, �/�V��@]��f���}�����,Z���R� �J#�����f������:� ��{ 9 ��bI��w����|d���)��+
�uE��e� �U��� �� �
i�9x�����m��R�mX��^��d:#��(e ?`A�XDW)�b��� {�2��;wnL�� � ����e�[*2��k\�a5�_����G6 m�Qd$�E��)��]. ����9����y �
��1bu�=c����3-��i��"��R)�s|a#��5IYh�,��
> ���������g�����I��9��aJ%�h����F�h 5S�&+?q������� ��oGR��t� ���B��E�JD��v���G�9-����o��:��g�A�� H $����/�|��AV�U^,������� �[��o��!�\�P^- ���2v���u��" �� �p�}��@����f" �VQR�f1 �!vS�*�V���x�d���;1�W]�)_Q5s�Z��W���H_�|� ���R5-gS�zwq���LV�8��o,��w�4G�C���\1��o4���tUU�`}(�kY;)W��B�/�5`��a���� ���A��I�cY6(��U�M���J� ���� ]���~��?������Y�b� A1�~|��V lz��E_���0�6"U ���5������. 3�J��mj�)�2�8)�s b�n y�v��/z�oKK@� W2�==����}���FeA�s|��� Ll��-����+�^-)2 `�������(cv0�z&L-n��}�9�0Y���M�|(�������������$�R��z� @�A�8�1�O�:|�8_\0�B����}���w~��)����h�4 f� ��n2c�5� ����P -���B��A��I�[2��o�f��� � �f2� ���u�/���4S���"�����DUH|
�Ssz��0$E|��%���aFkS�V,���C�S���� ���?u��� �6�#*S�~TpH�Z������������v4H�K�f������e�:�G@� �����?�G��� c1�����!���'_�/]��7/�� o�������0~w� ��M�|x��SF���z<m�iY�AA�J� ��
_{��I�%�V� )�m�/&�'��L$�AF�q1bqB� Zf1Q�,
�H&��;]C� |�% ��$����?�rZ���#����/?���������>�L�a�G���0G
�R���,BNa2���":2�D4�I<|��=u� ��h�[\�� T pZ8AE@�Rd ����}P��G� �1���X�e"���BA��Et�K P^^���O���3 ����l��)S����A��F��Pv����5���*L����F0"n
Cj�v8%���PS��Sj%�MQD��cd�%`:����J��pk- x.JG ��d�/F� W�D*��W��Jtj���t�)y�R���[���������4��$ ������YM� G7aQ�Kj�����
q�vOL1'�� ����!�r����,=��`i� �eE� �Rb��2�J� �CDW)SQQ����o����������� �XQ���A1�k��_9��n��� ����3��PV�_�Y��H�B�y���X;S43iu�QPV�(/�R�#��
�>�_\z>�{q�����e�R����� /� � �J�8F��X�dO���.��/
{k>��x������:S�����wzbeQ�� \�����7� �%:�]>M5�:�0�5����5��S� ��M����9��ds�?�AN[RmG �L����7�����/�3�����_���%C� � �2�1�A��sUMd .!� H�]�LAA�k��6h� ��������/Z���w����!� ��:��5>����d��V� $F��h��� 6~�}�2�B�X�r�����������9Z��t�j��%�l�B���2E,�1��R�Hf�e`��W��{����O��9�kn�v�m�
MRz���ajq1��*��iR��5���h5
L�~��b���`k�7 ��j�o ���,��+7����1s��5k�����;[!���� �}�e���a����
5{�A��u+�P��
��`�I_�Pv��4��
�M�9� h����A-�$�n ���t�b��(���*��������:bId�8>��.��m�F���"+/A�B=� �:�,�
4|�1���g����j5GV^�3��B���m I�mqK=��L���|�<�Z�Q �2������ �t.����z�� @� �:C�0Ow���6��gK��� �~��_|����2�y�k�s<� �)Q{� � H�Q�[1u��W��;���J��B���dZ�mke���K/(&=�'�/�Uyv�/ �B����U�5F�� �H����n���D����
�4��R%����<z�s��� � �E��{[O��m����%VsL,��gu��Y/��U{�@o���ed� �F��G;����K0���@=�� ��� ������9r��m���j������7��`oA�!�����w���G �Gv��9v���n Ap(]W �9�F*J�1�'������"������=\����[��l�������"�x���6d>�����u )�����������s|a;^@��F����� ��A.`���� n��|�A��@{5GV^�%1B����GNw�'�;!D����g�68���g�6?=���"m�J������,��p,�0��� 0 t�F*���k���o�a�m��.�������@p��y����j��<� �Ct1����w�}���s���N �����
6����.���v����Q�� ��=u~y�
��h�W����(TV�_i���jU�"?7� �a�pG�E7��O>�^��,�:�y�F�p ��.��/��%��V*{q��n�2� � ��~@X�=}���$��������� �#�����T����j;� H��m��/���_v����-�]�,�Jny}T�
��V,
z�������u+y�?NgM�fG^0��6�k�5%���"�fZ;)W �TT]!��$Zl��*��mw`b����g��� ��)G\���c�#��N �u]A��7 ���S0|��Ww��
S�N}����Vk����7�WA�~GL�j�-�-���:# ��h�,�#��p�+&��Rv1�2�~|T����6�]\�0RA
a �����7�� -6��� M�`�������_I����7���k�e����LB�e��,��dVz�"v��i�sT�-2�����|��k���'�i�vCA ���/�A)��3���z]������C��|Fp�� rMS^AAD���{_�S��o��� e]L�(���M3�����"�1M)�z�0}�d y_[����c)�.CV�#��1;FKYN�T{��&4 ����/�n aC�����'�����B"�^���w�H5;r\c,]0 �m|"�E��V��?��4}�%�bo��h�R�)���7��-l���#���V`�$36����6� � ���{t�2<��?���AAzez!��N]Z�%�V� )���:g��Rd�|p-D����FE��4��\`��� � �YN��b��$C���a��
1;�?�Y]g�����el�>)�T��!�$����.Ej -��&d�lU,+���s��rM9CM��s����� ���cd��M#� H� :Q���������,Y"~������������A��Q� ���m���:��C��V.� b,��:�e:���%R_
a�������I��5I{1�X���b@=GY� �"+?q��� nJ��/fs��Q7�22�0����hI��/ ����b�����qs�g)u�X0\U 2h,��� ��UtY���LO�R��"V ��� 0��C�A��D'�8��� ��� ���c����V����!� Ht(����� ���g23$)���516�����|tN�f���(�}� �� �U��>mA��ORL � b"� �!�
�R�L����#��>v�V�M����mS@��M��d���}K�2j�����@~9G����;�A 5;
��m��edN�M�9 �cX^�e��9��#&.�7�z�7{�#q���cG�� ����}o�h=cq\F�cq�F�sA��D�R5c��m���L �zA���[v���{� ���4A��K�]]z��W���U%0�f"7����4x����<�9���"]E:��� T�\x��L��y��u*!�&+d��&k���Ub���bi�{��r��I�t8�T5�[z��.�HaU [� $-F���y�0�i)�I�A�����+�`�H_bm@�1�O����z�����Z�R����nf~z��p�%�BC����6����O��U{�-��Z����>;�� A������4s�����S���v�{���]�'���� �O1b�C���R�TB��-��������v�?��X�K��J�� �^� \��f
���+0�9.EL��9 U�#�s }i@)����SW��Q�i>c�%K~�z�VwR����Y����&���"))�����% �{��o �L'0[B� X�w�YM��M�7/#[���9�FnA����w��e �{ Z=�� �=T�w��2��m���KmR��1�������}b��s�a�K5�$!O�"!LWa�[k6��"e�eR���7����N�g�"���b������;\��J���@&�.B�xnV -�T�x��D�^�iJ��@"�
E������[Ey����h�V"�(�(M|C�x� )�e�$ZB��>PeT��;������"��|bA�~Mt�2S�N��y�|d�����*�VA� (�z�+F.:�M���LM������DXG�M�fW�r�jE�cIB�C&��� G�����D���I��G�
&� �4�����i��*�f �LW��HX1��
L���Z-*�^�&��*v�{�DyE�Z',�W�����j;I\2�� ��/���y��� |����9�a�������..��}DA��b7Q�;��X�>�M�Q�]1x/�S�s ��0l�`��� �*� o)�BTs���%��(/uq�/���D�F�WH^����*���b��O����k,���zO�zS�~����d�mm�j[<�h�` t��j`6 L���|��� �D�lq:��t���"� ���N�aYv��
{����o_ �����y��fBAbEL�w :�Dk@#����e@�����M�l�Aq� I�OO���M��[|�%��E�V���u$Ly���' [�<{�x-a�������4�=@ �� %��"��� �,�`v����j��� (��:�KR�/�F[8 �Z �?� T��3���z���C��Lj\���s��D�Mjh �0��dZL��D��l �����5#�N����|B��O>Y\U����\�kb5��AA��E_���_�M� A��K7�zU�����:
S�Y�,�T7)�o ��1�G_�K e���
�L���������%�����j#� `S��/I�D�8���`|��L���Y`���k6�r��*��,��4[�}�m�� @�!�W�\�>'�����s��g��|��?��O�.]*�UW,���s���/E�Ad��7 �����}�}E���W+~��|}doQ���s���}���6�a��v��rY���D�oe�^�DiFB%t�d y�_ �c5�_����S�% ��vv������0��L��m��/���_v����-���@L ��.}l0x�VLV�������B�@� L��#6�T����7���-�t� ��v�|����{��e��]AAt���������������,��_���'O����AD��:��C��V.� �Vn���=u~�� oc�W&:E�(������� ������������|(��i=�������� ����I�f���f���I���L����#�.9S�������W�T��BqIu��
Ro�F�8`L]l�6�� lLM��P�� % ���� ��O�>>���y<>�����]��;S���[n�%77Ad��(��������~���%Q���n{���{`c� ���#^��"�0Da��|>�����]x���>�_\z>�{q����b���<)ov � ]r�_�=��ss_�����N5����) �� ���{�����5Y}W��W���@�rD�����L\1\������-+- O��rg�7���� G�jvRCC���3g���|����� �\�e�}��u��
4H���[� ? �]!� �
eu~��~��*Vf�u�d�6I�����M�i�.�:�;-��J
H��_,T+\��-J��+/"��"���3-��i��"q�9�����i|��WU Yy 9�����H � O���t~�z��2H�[��caX�c�)L����)�>�h��oC��j�
�s��{c�|BLl�G;w:���(E *���O������?���CA��Bt�K^�w����B������ Ho�Y�N��p��b�l}� �D�)�iSt�Z��J��V#��E�3�T
���p�����b�K�\{�� � �W]���%\� X�1;����/��3��yM�}D���.�=�\����OB$o]������/�&����}�9J �����-=�R�y�2����bZ�D�+�(Y��F<z�U�� y ����}��}���:%6�"9�$Zl,c
q�h���La�xA���t���'�J9S�I�"s�M7=��s]�� ��G���Vb$6� �;(�z���eZ��2�U�S������v3���+�7�K� �jw�s|�f� �w9��K<e�*� �&� !�")/�xl�pk��O����'�{���#��� s��:�� M}a�<�BTF�f'�" e�+W| z�d���$���X:l�5Jl:E�����#�:,pBd6dE������k�!xv����y<��z���S����m AA�/��2���'N�=z�4�k�.�_AAz�!v����x+3E�&�wR���
����|7�(j@�-��� ��< �<})xr{��$@ ��Huf?cO��~'S���"� }��7<�\��)Q+i� ����6a���l���y�6����"^(+ N�i9��A5q�j>R��H�3�|nbT2Jj\���s#����c����GP�kR&Z�"�/����!��"3y��^x��{CA�~Gt���O?�����o� p �� .�� IDAT�{������?zfo� ��c3�yW8�&�����O�^�@���D�Qm5����>�h�y.I�)���g��0r���������;��!<f���bI���>-��p����=L��y�t���[������s ����V��J�������� !D� I~��ll 9�a<��`�t���.�mK#�����9������9c��n�Ad�uL���G��� >|x����_���?n�
��}3� A�R�x��/w���,`I�2�� �t\�2"�tzQq��V�f�����1vg�:��O����$B�� N�������;�|���vj=�v:���� ����������S[~�b�J0�����M���zOU�> q�����\�����|"H-��i�s4�RJQ-T��
���R ����Z�M�%u*�T X�wK�NdfSw~3� ZXX���� H/�7 �����}�}EA�l?�|rT>;M���k5IK� <tM�����"�������8����pSU�a��i��S����/
���={����&6�5��GdE�]�I?>�)�RK^��$��i�����Z� ?�����TY�)]�nCc�����T��e�����+��~����#�8X����n��Ju�����j>/� �� ~ A�w�����/!� H ���������G�
_y�RK��7����2�+bXR�Qi5x���@��|�X#�L��/�2��"��� &�$ BlC�'�5��o����/�=5Tc�D��!g*�9�"�0 Q�q �������cb-Gn�[�mX��^q����h�V"@^��.3�1�r�a �H*���V�\��� R6S�iAEA�0Q����;x�`SS |�������y����� �hRV�_�����yAto�S�������:��l&�h��M<A� �PfV^��������5��F�{kcQ1^ ��z.��,��I `�KU���r��|Q�[' a �6
��Y�wFy���LU"\c,9��S� iv�|�r�b/���z�\@���^������kU��l9-����oY6<��eeQu��P����"}K,�|Rp� ����~�� � "�)� �-�� ��2���_�h��c���?��_���;���c�����C�CAU6 m���9a�Q:yj���k\Iq&3C��L^���J�R4�}����|�v�����O���3m���T� 0 LK��$C!ZH������>��b
�c��"FT^�� �� oDa �� ���vHib�5 L, �� �?KYy 9������cQC�vG��mzk��5�������TY��1�r�������":��8��l9!���X5S\����ed���n�� ���?� Z�U�c� � 2���}���>����\.�K/�4�����7�x� w�q������AD��V������"*} X�Nh�Z��LcdiL�U�Oeu���~�m��aw���D���KJ7.@�5 �
?�P�=�;���My!cr��&i\���.j�������IE4"T���l
Yy ������ @���\,]t
��?����k!&��y'y��%k�z������ ;<e�&g]��}���S��#��m�S��(c�Ui � � /#[4��m�p�Oy{*v� ?��_��NrC����ua�� r)�(����r� `��=bx��f�Z�=�5Ai�r�������N�4��u^YqC�(�]�|5��W+�[�~E���U����A��'��&%�����D�2��3�*!�����I�V@c� w�mG*�������n�� ��n��/���K�H���P�r���j�B�Zj� 9R�����(�? #�~������N��FGl���E���F�M��g��5A"�`��i
8-��Qq]�6� �\JD'�444�_466���!&---��BAeu���_D=�JD���/(�~���SSlT}�� n#�!:�����4;F)���� �������n��� �{z���LUEF�N2���ElD�1�U%���'�O�� /{�O��Wq���@�����>��#e�5H �?� ��KX���-���v�E0���|U��/�On �Gb���3��sA�d��Sf����w����NNNGx� 9rdl AA�PV�0�gB�FK�>CF������m�s��LZ�3D��j�t� h�&*�2����v���T"� H��B�;�8J�1I��Ws�Qle����6E�m�-��p��;N QJ������ uy��+���fW�4�`08k�,�������Sr�8��i�sP�������sA�4��R��_\�p���[�v- �<y��G �0�"� ��i���
A �X$o���s������:����@���b�� DM:��M�q��� *uIbw��d s|�j��>J�1I+�Z��f��/�xZ� nL�l�c�f]C��W t���}�E�&�������P(t��������b��������� �
���o�����_���k��6�{�_���Ad PV�_�V,��e�l�W���i�x� �0���V4uT��>������. 3�JgT��r�.kO���c)���gLv��L����*��[����H����>�WU���g�&b�&Ly ����1����� T{�ZI�$dZ��!.����`r��h9��n�k���-�_������k��'\XuZ�����;U�Q�!�l���e�^�� � 1�o>���=�/���� ������������/� b��=u~�e���52�]%����`��Jg!&� ))�u$������6���i��N @���d���?9��o� u��
���/��\��]@�J ���&+�/�I���9����������WA�k�����_+�0�9���fjU�p 7g� ������b(� ����}qO�����"��)�l?K (,�f3���EY>����)!��W]$�������1�2�~|�,�tR&C��i�t,c��d���T��&t� B����a���p����<� ���t�t}����;>�XM�x�����KD �v�`-�&��q��8����khh��?��� � �G���]�E�x��g��� r �}\������I3���������'� 17��{�6RGE�^��������&�*V���KT5nL�\c9�1������Yy 9���1�%�$3c��F>���sw�{U)��H�������-+��+�������~������]x�� p�[B<W�mX�w+ N6���q��y�.\��52� �)����w���/� �\����<�:�%�/�D����������/HT� d$o�;��� �iJm ���p�j����/RY�)]tJ�d�����y��b�7eV��n,�%9��Y������n��*)�>�ho[�����2":�$����w:4��Pp����>a������������/=}����kmkm��AAT1$�p �0 !��DA e��\@���wZ�Mr��.j|�Yc ����AN�A~O� mez
_u��|�X���x��@�y��3�ll���c ����"����{���j����2*�a@i���� �a�;;b����b��8F�Y$QF�`�x^��A P�|���BUX����3>(�K}}}�K+~��?�5 AA������^{�-���� �>����;��S]]�{� ���4A��EO���t�����QANh+���x��V����u$LZ�������"��S�H���q�d��J��7!�2��!��O�����J,��`#�@ST]q�'��),"R�RI��{��z��q����r)a�:���������Wo{����|f�k���' n�81 �BA.y�����J�9s��7 ��� ��������A�����-~Y���� m��(SV�_����[�W6U6� ���^�iU�8��Lo���bO�W�y�g�"�����E��H�� ������{����j��"0�`x����R > ��PQ�R��Km�&eE��l��J��F ��F��CZo�`��O�z��k[�:��;�e� �G0$�<��c�.�����������xS� H�%�����#�a��;�l ����2��#��SoU�������PZ ��a-E@�/���'�N{���" !��<Im�JE ���<��� �q�T�����k%1�/CA��Kt^�w�yg��_C�y���.�^A.2�Z�P�<����$�D� b�8����x���. *D� G>�wrcZe��*Z������8�x���qs�gE�_�L� `��jJ6buL��!�s�� 4}��q�[d� `"��D���qn��� � �Et���34���(E+��A�<��S�����:��2?��rp���1�LEhEA�� J�b�^45�6-��0 �RmS����g�%L*4;��5;�$L*����U�����hqoM�U]���z�-F�[y���CBsMP����c�L,nvh~T���6��� ��]0a��lhonj���b-�
+OM����� ��� i[V
$��.��b�#� ��w�z�)���b�E���s����^A� ��������>�D�wv���R�F�Z{�p��'|�� ������sGaD�XW:���\� ��8NXG��k�yc� �2+=U���(�0��k ����h
6q�������K����m|"��U�����fkc�� <��0c5�_�z���$�Z������������s"p�w��M��Kk��� e���k� � }���/���7�WA�K���l���O�O��D^����E+�����B���F!r|�%���I2��:J!�1�V��wb���4��T���L���3��ICc����ta #�e!��W� ��c����(��`�G_}���A�en�}��-Q`�� b���� �==���O>�dLv� ���+� �������7x�� `�����������n�b��r<. V����k����#��K���;��J��H��L|jX�T��*�o��kY7y��� 4B{���:��tS>�s�T^� ����-���H��� �G���������wo�AA� ��P3�/wD+!�o l&��,��.�#Z�P��h��_4���_�}�z4>���&U�x.�������E�����K�����6�4m�H���V s"� ��������.�UW,���s� js���N XZ�=������%<s�LQ^�E���Qr��,� �2k���Y��w��A��#:�_A�OA�E�9�k�D�O������^\z~�)�'t��:��C��V.� ������k��A;J�H�"��8�i`b�K�Gd����B�[�)_��*��� �[pe�g����0 ����NUwh
�E��
;�����5A E�<e+�7�U�� ��VYYD�Gwg�%� ���;�" n���7?���oC��o3�Wf��Oo�v)�0��i\��/ o��<������ � � �*e }�Q��_}�U�6� �D�V�GT�Z�������������u�E7b,5�El��&���} �.|��W]$�PG��J�x>pN��H1���p�1��r}G�W]�)_$.(
: ������H\����>+/�~�E����"/�L\1\>"?��&X��".�$��"����,�*vPMC!>��%\~��p7/#{���xA�e�3��*:Q����R l�u��K^F�@@����[BA�KC�Lyyyaa�������n?� H ���D�� 6P�A���ktAa��
�LM� ��SS�m���% �
�J����E$�y@���,�nq���E����' ��{O�i�2�D��Z�a ��+����8yO�C�����i�-u!@i�s��X�M�F��A&��w��z��n��z�D5 J�,���S�mX�w+ /#[�O���WF;S}FmD���6 }���#�| wO9 �*����l�$AA�e\.�UW]�u4>>>v�AA�@��#*bX������c�����lS��(mn���_=��X��u/y�!�����C�(�v0��|�\��5g6u-�'g*�9FG)� Q�W���k�E���{2D���#]� �*vH�� �Q:�(���� �� � �6
RV&�2�����������Oi��#��� � �q y��s�==�A� LM�=x�+)�dfHR���k��m���s�P���� � j��fx x 8�v31�A Tmn���]��2 ����k�c{�K �K(������E �v0"�������1������#��r���@�6 ���� ~�$K�cv0?,J�{< �.0�5��%Hi:�q(�>�3� v���� ����?�� # Qz��edN�M�:�X�v�a^��)�{������;w�"� � �%�P�}��w������?�7S�A����OU�,�:����2|�D�@���b^u�����z����A������K�9�N ��l����'VS-N ����g���Xj�s�!&��������� � � L�� ���mv09�R�'��bO������'����*b2\C������d�.��'[A b���W�q�3y���L����O�_?���k����k���#uNAA�>N�|x��{�_���A$*�M��s��
u��������t&�d��-uz� �ODi�&���"r�|�%�e�����p�Z���:���-u��;���%�9�%�l���T��1��%�l�������L �< lI�i�F��eK�ls���\\U!�9r��u�gq�-�>�`�Ja:�
����� �^x������499���!� H_�o>���AA.mT�Vb��SF��X%F��H���� F�����*2������"�Z<e�=e�%
�x���qXM�&� p�_� �93gmJV�z��� 3���Q�W�w��G;��v��&0�" d�'v:GB'�H5�I�2��T��v'�_|��W_EEAAz�E�>���O>���G���� ����t���J��(�r�X�n
�6(�.!� �j��� �K�u�������XY�������F ��9G�:&�JT�>�W���$dZ��!N�L 2,@@i &+y�?N��.CI0 y!���������_K ��6���Id�R�������_|q��]�G���{AA�^�o>���Rf����_A��QC����� �a:�����)��Q](W�����^K��;�kX�s�'�����
e5�!�F� �Z|G�
|D ���DA%LS�KZ*K<��N���k�&+c�2\�>�[R�i�0;��+�� rk }�^�(%*�i�P�L� )�A ����9�h:.�;�BxA��/
�;�?���_,--EEAAzC� �q �B:�� � <��!R�T7KTT��6 m����9a���N{�b���6�����$����5\�Fy
u�S���>?|�VY,# �!���Gy��F+�I���:i���_�
>��0��l @[�
%�p>u��qw�lt�a��w����v p�b' W\ ��$�0���W�,9��5:��5��<*�> ���x ��v"k�$��}���������Z�g~w[�W��/ ���������������3��� � �e �2�����r� /�����*�~��W��� �������Yc ������5���iZ�*17�U�O7���x.�+)T
��h����S^k�=��w�����<������5�����H�,cI4���Ro|/{^���������^��)n��oRo|/������
@��}�_6���� ��{��Y�d%7�>JJJ�ah�^� %���T� /�C�Y������ >�y���h52V�- !`"���$
@S����> CF�%�������v�����fhqAA��!Qf��9���������B�����c�/A����� �D%*�dj��Hc��o}< �RmSRlo m1R�����l 8��F�6�������P���p�I�Q{z>$�}�d��M�� !�����5���/����*K<���z�+�Y-�,��uhFTS��wG�4Y}W����}��~3�HIp�"�mCM�V�Tj.Q���P�-��� �E��>/uJ��$�` �;K5#`iOS��� 0�1�7M�
v����� �� � �5 �2�=���������������1�� ���~�zZ IDAT�N�v0eu���~�Q��aw���D�`��������}w��z'���"�K��'.i�1�3���� �&r�&��������p!pA�Yf��z��>��ZJ����$�`z�}�������b��� j���1r��u�{�f-�����������5��
����3###��AA�k0Q����;{h � H�e�X����8��� ��h�OM�=x�+)�dfHR���k�Mj�u=��N/=����$�p�_!}� _@Xu�W]�����s? $L}�X����"����e��2� #�I3��K�)Lf�� W�yz���p�9�V5nL�\c9�1����M�Ej -�P��0���0��2��'�Z�Z���l^�=� d����W�����A�M�K_z�������/��fA�~I��w�m�R�w�8�.W��F���G��c��e_ ��c��%iD^#�/����q�}��K���U9��H���_^u�ry��\3��OBv��$������H L\�����%�^*��t�) $+�s�}D���<|��Z(�$w�5� )�W��������c����X��}I&�� cg��p �� �n�sv�j�L��A�����n� � � Q�H���
E � �0�l���J9��aO�_l;��i��Q,,�vY�n������J_R|F�=*Y���|� �����u[_�X�
�&A��! �|F),��H2;��~�`�
#���\3 moz������{^�9h ��;�l<s��s�����x�I �C [
�������2���Z����d���\7��Mv� i������UL�����tk����5�����h��G;\���Z[�����*��A�i�ejjj��9# ��������nK� ����������rX��22k�c�M�G{@�`R��X�D���!��X"�6�,�������� _����G�{���@XG��u������(��[��$aR!���1I���|eH���e�|�� �-������L���YiK���S�@= "�f( ����� �}B����L^F�@@ �
=R�<]�PC��H�(}gA�i�e�.]��_L�8���������zW� ����� U;��;(U�(��b���&.� ��Hi7����� �v�0�����@N�>j}I�{k�� ��AD�tK0�
PR�����q� zf|���"y���* �_�S� ����i>�6�Y��"2�%��Ze5�}��M������� ���K8^��S��`��v��o�ER�O!� � D'�|�������+2� �}T�w���S\,�i
�2�K)��BpR���%��K<� $�eX������}C_�(��et�a� s�Z��;yzo��^I6zo�u�� [m��1x�jY���e��$�P=PJT�})l_����3$z}����O~� B~z���� ���������?��sjp��M��� 2�f��o��� ��R�L�lP� T�V-����}EF�J�T��hk�H�"����W�/_!_�B�|m�)>I��#�8S�Z��%�1��������'�U�j
&Lw��[���������c��X�4�9�pr�d:�d�?[�A�N���M�9^�r���W������C}����&�i5�a��x����h����
�A�-�U����K�\s��~�#��*
�����3�.0FA���X}�)*����c������{����o ��Fe��-�
G�Q�&TX5�?�y2!f�h�(Sa�e��Q/ri��������h~�sk���#$�����mp�� -FBy�jY��FG=;������>�� �
�?�X�o���}��� ~��������~�������Pp���2�XAAz��D������������ L�>�h�w� � ,����i}x����S�]��&*�����"���3p�U�����
Gg��8*�IdX�JE����8�u{�l��.�
+��i�BsM�deLV� �(���c�DU�E�8F�o���U���o5���~�����)v��q�����_�����!�AAz"����s�M7}��'�)���w��U����>s�% K$�@�zk!Z��� h+*XITj/b��^�B �z��W�b ��IT��VA������^ ��@!���y~L2�3����H����Y�yN���|��~>R��s� ��!����y�]��:b X ��Oe�>�Y�L�d��p��nz =��wG���G� #R�8�q�T7�&i�:A7qW��t�AI�����0�i�� ��Dk��,i�qw-D��Y�)�s 0����tr�C���"SZ}������2������3oz� �.�q\����O��K/����������c��1���e��2nr�{mA�7�3��G���^z���<��#�� ,��aC�7�k�����
VI�J[����z�����#�?A �i7f Y��������K�������i�� �[b�M����I��\�����SN��� ���@tF�X���q�����I�e�jo
9�0����Xe��~���j�� n��}��������~�#�`IU�,Nx��Y�y4� �K��o��_z����
��'�H�2������A����:��N����p���y��Ny�#=��������C�[��|��&[����$�f�h�f_ ��Q�� ?p�� #-c
{#�,�7���T��z���\ ���V�-Hn:���v�V}��f��������V��cn�p�H;v�(((x��w�� D�q� � H��N����{�/_.;�b���mA�U�u�)�P �� ���!8����[�`k���M�v#a^>��h�a_�v'woD�:����d���A7�3���]��^�������pf�������uAo����1����3����^5�6��i��L���[��\�8����������
fP�Q4��w��5g����� �����#��AA$�D'��� �����^A�����J���G�sC����lsSX#���sy`��B��#bL�����4C�
��f KQ�Q��Nahvei��E�������sQ="���_� *2A/����~ �VB�DC�Ql��b.o�,��&mA�\A�0$��e�A#�m���{���o���O~��h_� � �Mt� � �Ha+�8�J��������F)��Ma�0juN���Q��#�F�8�; ����L����eWA���<0q�j�\�8��Qd��.���v j���`��0�b�5�W1�0���� !�4X��f�����ho�y�4;|���?�f�[cE��={����7�|������AA�n����5k������^��� � ��Y#mf]W��YGn��H��h �rI�G�����p�� �����a����������b�.H�y"i�F�o���@)�{[��wW��]t���'�v'��� ;Ld�Pp�p�vHNO�)z`�#)DO2���@��L#g ��+�&�?����������H��;�.��#�����v����+�6?e��h�����+2�*}O��Oq I��������{���'
�����{������N�<Y��j �~>(hF%U�4AA���D�>� �5[�l��� � ������$�Xuz��Xu �I��?\��h
���-p�07�����p��%GL��]R�3�Q�d:�Z�J�.�� ������T���:. wuqcY��M���,��?i�g�
�� ;L4�p�.�k�3��-���)�`��)�@�3
sO������'r �]=��AG7����x?��*KT<,�|�l�PN:U��X�$��_q{����-C��d'�������JJJn��`���jF� �\ "_:r��o~��k����x�A�e�1��#����#�inHf�aa��f�����Y��}-���j�h�b,0 W�4D&���]@t&�eY��\_t�����z�� ��$w�"6���^����$�%���g)�l{�<c�}6��8�
'��)�/H���%�����:�� V��Rx���S�G2U^^��_����^����#��L� �\*"e>�������W���� � �� �\�����;�4rYs
��ny7voG���%�*��j,��0��6T)D� bH�^V���1�P��-���k��x��Rw���`� �/�����.�A�^�Y��c���o��l= �� �M ���D2<)DU7d% �\�a�-aE����3f������Ojt3!� r �H����(��� "I��L.�j��������k�0W�ns�a���$j ��+� i�k��
[��2��3�G�l�s��fx�� C� ���a�J��6� 8??�+m���'c
�\X'^�6�����(�e�L_{�A&�D��T\]�j>�l���g%�Pw�
�����L�>}���?��������A��`�� �]��*�J�T.Y�Y�vM(�k~2���S ���>B� �"��kgM�� ��
��K� ��:hD�����lDLIgdG�O�)3�&�'��I@�t� ��4@���a�*5wv�a�t�61 1~q�� ��r����@U����l4kt< <xp��iEEE?���e7*v3E�xAA�
�2� Hw�X�D � �#��eg�&��G�7������sFa�v�5�pS[_
L6M�esjX<��dY�3G��InArnAh�Q�L#?+��:2��%$ �0�h�LA���ZW+O�N��? �.�y%�s gP��H� :4u���_~�������yX�� � =
eA��BQ^QTj���F���b�f� �e}d:),�&�����yR�;# NW$��P_ �;���t u��S##��|��b�A�E �VOd� �R $ � p '� ��F��+���!_�:�����~O�^#Qm���S��=|����S��]{���F�sAA.9Ub#� H (�P�5��i3H�;8���FW0�S�b#+�F��������u���� OW� � J��@�; ��NG
���^�2����> ��� ���V�>��������Sg��3�l[Y[>Mz�:H�8\��B(a��+�[6eU�5~�)�������2�rG����,)@��`?��\1.|��T|4�m��F���G�M���?�y���� AA. (� � ����6���
��������U��x���L�~*Zl���^� ]�$�����O=���W�;��%G���ZQ�`�Y' wu)�n���~��,�]]��\KN~��"�=�p���3u1I^t���s})3O ��@o��qEo�&� 2�pp!Z���e!O�,m������������M�M��)2aEL�a[�d ���;�:=� @O��#��z��������zM��5����=��]w���A����� �](�PS YF���+
�� �7E�5���S���v X�L�8� ��K�����(E�B^>����d����6t%�vv6��w�8 ���i[zA�hd�*�,ub3�����_( ���r�/ P�a�3>�7��>?_�s� � H�EA�as^���LP���D$6�J�c�. J��i�`h�]U����Q� �!/ ����jc2�)�R�n �T;���6����++��f���m�t������o���;��_ h��:� � �eA�Ha��c[G-����������D"]�$ N9�����S�9J���b��}Ta��9! ��w \�����[�����$Yw��D|-����N��)Y�jHW;���6�`�y���7z� !>����]o�8�)b�
� � �eA����)VY��T���V����K|��� "a�+!�W�`Zv�M������/ q x"�:E�
��F�XC�N���6y��T��#����0�������6^$k
�y�p�;��q�j��q��l �gd^�j
���U|�!�H�S^�DJ�����#��AA��A�fW%B�s� H�`�gM
���6�_� �v}$�����(:<
y��h��^m�����645�;"�V����)w S\�� �t����@�r��4�F�^�,1����[UY����.� pD�!Ha�s,W����_�!�L��W���6 PY�*���m��������a���?��[G��`�'�����g� �q��U�NU��BADF�|��NA�>GlSHM.�o��#�" ����W468�/ j�[��LK�5��a��������������w�)�k @� ��1�O� V�&�E����x�1Y�f���hD�"�#��������r� %���>RZ} Hy�rk ��t�Fs��+���t�D������ ���
�|�� �9[��_&��Z � �\�(� ���y
��O[��;)j!2 @)|�� �i�>����u�|��p�����h'���[�-v��������M���v� �"Cm������2��3����Vm4(�>�h���K�B��y��� .���%� ���m|�_�e' 8>���S�[�d��$�@����������c�3� � =�8�2>��h4^�:� �E ��d F�Q<%l*���Q������C����e� �i
# �>���
�+R�}l Py���[�/�B8B;�2l(����
��4"c2��A$����{?�#F"Mz���` j �����m��\s�M��b��A$i(�Z� ����p�
?�0*2� r9�]��� A���V� )Vy��@���]���;�
�6.������^0k�����o����������1�6��j �y����'����<�x���A����
fB�?Np�X�'c
�m]K�m���O�4r��i�\������qF����MY�k�_o���,n {���� ������[{t�\�� ��#����3p�L{���3gg��~��S�&O���C-^�8�A�ED���1c���}���i3� H�Ie�"�6RG2'%*�=�d�A��Rj���Z��+����D�m�_�uW�fLo�� � �u���*
;�D)Ot&�+�����J��hT\�b�KA2�������n��d !�Oi��E�jO�,��&�\��
�� M��S~��/������'�+���-�y%A����� |��H�� � H/""����.+++++{���ssso���^xa��M�g������;��?A�g"3���VD&��\��b��9�b�-�&a����8'�^6)��~+" 7�D�� ����j�DE��C���2�CKv��B �S��P��~1����������T�����-H�{"���cW��?y6B��b#�H�Rd��'����V'�9��/P�I�u�}�lL75�f? &�J�*d�644L�2e����>�h��EA��Q#Tyy���c���p�� 7�t�xj��];v�X�bE7��g�3[�A4��})Z�l=���8xc��L��gMl�L�U7*�x��/�>#P_�]U���>�X�`be @o�4��;�5���I% �$�g����3
�;d�����U! ��kY7���M^��m`<8�'������
*E���\�� �k�������-��$��|_�N?��T��$=8"a����� 644�p�
�?���R� �DB�|� �SFPd ����RE &M������_� H�1)��f����)k&����������b�L�+���3:����5{���u���D��������,�u�=:}"gN�" �1�M4G��������~5E (�N�I��6Tv�M~Q�����"XltL��F�6 �3�/D��� "�a�����;;����w�I1��b�I�5$>��t�%��j]]���g��x��W���~$5n|4k�S��r
� � ����~ {���-N�AAz:�G�F>'%F��O���q7���6��E��||�}t�Y�+����}�m%�m�AW��M��^��3 g��3��i���b`�����z�p�R ���#��>j����V�i��E�����F �-� n���
� �l �5�y'f=�����k�n9u���FS �KVy��Pk������o����������A��D'�������c�#�� OHH���A� ��:�����J��p��:U]&� ���g�)I= ~ ��<��h�n.{ �RW���c�.��,���*|.� IDAT��qEz{��R���'�!�o��>�Pj� ��3&Ku K�l�jj
��j+@a���������Vd��O�\�I��H��)6�Q���������O ~���� I1j�Kz�� q���� KA�:�e�������5��p�4TXH��� � � ���F�jkk�������� = 8�x��M�6���t�{:=s,
A�;X�YSCh��`��/������2j$��u7
���/'N�n���0�."��� � W[J���r�qwu������NgM����d��K[>�7+i<E�m_��� �#�~:_[��2�^9r��C# �Q+x� BeR��M�L����ws�zZ�����l�������������������������}�Y������N���L�_� ��z����*�E222�~���+W._����=##���t��a��9A�Gq�-WL��#��u�m�����w�y���Eo���,l���}Tak�B��FDqI�qp��s��"�[�,SId�1@���1�3�� ��_��-����;�y���e���0EK
vz^�s���:*�����=����p���> ������7��M� �\ D'� �������t�VA� � �N�d�o���t3%����&O�RMI�l>�DQ�E��hh+ `��B[w�K�<���=��=[H�V 8jZ�-J�SJ�U}� ���Q� ��R���p���6�7�L_��@��� �o�����,�W&*>����AA� Gt�2"���5� AA���E����f��k�P��� �A�
z|}:�
b^�F���0�D�F! ��o���5Cv1g��)���;����x?m�������t&|;R� M��� �y�, �0 ������y������3�F���3gg��� � �#��)�o��+V$&&r ����t�� &t��A����di�����/��M��F���9�|�
�l��:�� 9���jX� ,�a/��`X�Z `M1Wm�����8�?\����"�W�!�����mG�/B|L���� ��5���?/��P��q� ���,R���2�I��^�����@@x�&�^������F�A���N����x���KJJ��� ---s��]�|y^ ��� �'��f��
B�6� ���_]�(M���� G���!d�(&�COb�o����7^���Z*��Ai���\Tz���G{
+^�+2��� ��x�_�v�h���J����w�-*)���Rx��� ���9y ���G ���4� 8��p��e�R� �/M���s�\ ��OJ�v��������^A�SD>|����_�>55U<r���E����{����A� pF��(v0
��^�~��z�������e�BOS'�`����;�a�u��Aq��lI��K'�d�Iz7y]zn~�#*[�B�� �� S�(C ��n�{��(�$�m��pt; �/�'�s�C����o6����0�����;��$�K���e�jobow:���M���+���Q
�AA$N��7��9e�^�T����T�O��� � �1k����m� &=�s|:}
� @��6�8t���5�G������ {�*Yv�l )����$�)&�WQ�-�L�0�����%]0�����j�v�f����+����3>���@gR�HGS�� ���q�����@�ZW����>}znn���hxvA�\�:SAA�����o[��O�����D�8�D�����)e�s���'� �L6Q���,����Y�E(H�gZ�����f����]H��T|�q4/�[F(���-���������B� ��n�X��y���%bj���6��~��x��� ~ �$��g?���#EFm�1�|AAz��/]q� ������~:r���n A� �U�Y���`k���M��=� j�I���y�|L5��!���6��8crcYV��J~���������:$� iL� �-���"� !��HB�qrQ��$��������vPi� rSL�U�jN������( ��G���/��\'4%�����Z���������������w�d�����Kw� � ��MtN�U�V�v�m�����z+ �� ���qc_�AA"aO�gCgFo�+��p;�01]u�������� �Kxw�t:��d�) �saH��`o��OJ��%>@ 1����� ��2�$Eqb�9�PNCB&��#�mC�������m��b��d�����~^�Ir�}��*O�z �� �K�$�M� ��E*�M��G!� rY�S&!!a��---- ,X�`����?������#!� ���<��y��SlM O�j;_0]��~ _��k ��Z�c����4�(M��\97�������7eU�������
�z�����n��Req�������_I=�J�Qa���H�� �<x�5
GO���A�N�b����n�u�l�Q+�s�^�x�`~���{u�]gg��=�� �D�}t��]f&
\v�#5EF�-� � �7Qg���%K�,Y��;v� �w��<=��(Z����6l��V"~9 �Z�(�N!~XY�"�`�^��'r!��I��Br������r ������ �j/���y�l�������,,�i��X�5C) =e���b3W���
�l����X,��� ����]E�
Hd��Ylz���[�������EA��&�P���?����e7������ H�g�gM
����m��\? ���S�Y*m����hu���:l��Pq���� 0RBA$���JZ��L�`�~��M�p �- E
6������w7�������dkj�4�8�XD#N��gTRU�j.6�q��Y��G�
��i��� �{��wd��,{F&�8�`�� �t=��{�{jii9x����e��� ����xo���3�]����z� �����SS��� �}�e��u��u ��e��&���qW����_��$�]g�����R�����(C ��q�e � ��/�P�����W�{�d4Jo�&�O���*:*�L�W��#�2�H%��Wm�=r��"�/��(2��7������O��:� ������=����>���'����[�v�x��p�{W� �?,:�y$��mY���r�`Q&�����fJ`�7�v O(��W�A�F��+9�D���[ ���l�? ��@���������=
!�]��"V���j���s=�rn2[��������}(�kd�����n��H*�) gp�(I1�V��U�]�A*TdA���(�b��������� �X,q�� H���cS��"��h�:z��L0�(�(P�v/�M:}
wuq��Zv�-Hb3}��s�����:�gR��X�=g $���-{�����"�)y���� (�/b
���I����y�D���$��h :da�j�"or T�wd����o?\�~���C~�&TV�eJ�,�S� � p�~��9��AA���/�F�" �-��~A�
�� �IH��� I��\�8O�J�Y#mf]�#%�G�I�������p{�+�� m<���� ��]]��o;p��
uW7�e��dl,�rW������fN�� �3� rW� �%'?y�+z{&� z{f��W���� H�����W��ML������_����������i{�o��S��A��|��9E/�;M.Jc]6�M��J�����D^�����m��o0��ux��I���V t�&H9�AA��@tN���r��V�����F� � H LJ3_H~m��$ahHjW���4�<�6 V0���j������$�WD6�$@ 6c���}��SGM��E��{� WZ @gp�O����
bG�F�i t�"$�$����w�yP ������}mA��XR���A��^� m�/m��uN~��Ya�Z�)W��M�������s��(M�H9� �� �s�<��3��w�,D�������A������LJ3��<�xj���b�4�<�6 V0b�!C�2Fp� 5�� b�``Q� J I��|�[e 2���8���;���L���t� *K[�ll�Pd�z�1�{ p����'d#���P��f�j �~>X�h^�����B8�a�t� :�@�����s�����)/> AA$"��5�\���&����s7�|sRR��G_}�U�l
A���Si"A���m�a}4�&������K$��R���@{��E��m�.h�3_ve�y�3h
z�P���s������K������,��!�n��t�g�S�
��F{
�"oDd3J�����S��k9��0[��j m]������S��/��K^(O!�0AA��2C���e���;��#~�AA"�������Zc����]e��a��
��E��b U�Z�PZtmL���, xFM�A�����i���|���Q+�����>���7}��1P[>]�U�kT� �p �e�L�������j7qiV��
�����+���d���Z��PV�� $L^>�
� `3W��"|�N<�h=�}�����o6���]�v�z��g!� �D$�����g�x��8mA�� L�a�5:��@��o ��g��)j�Q|w{9��|Q1�Xr��P1��8t���56e"|�G/
�Q*�u��L��f�$e8G8z��kTIfS�*�C2����3��.�#�f��|1�j�� � �'7[g���/��P��k�jM���% �?r��SGN�Z�� G jM��WgX������fr�������f n��q��< AA �~��B~��D�|�%�558�q0f ��� b}uT ^�=_ ��Jd���q���Y� h���R�i����W�~��U�
�����C�-��� �6OW����*I�1wW�QZ}����Ea�f0�7S Q��*�&c��
��(��4,������[�K�*�O�@p��_�{�X��6�q��Y��� � �E�g�y��}����?���+���m � r��9_6 n'&�wo+�z�t�����fo�w�����a�������}���t���2)��*�m����q�x��=,���b�Kn~P"�D��{!<��}�� ����u)�r�4z� ���_���P�H�������j�w����_o��Y�� AAD�e>��������#&�����_�zu\w� ����z�1��t���
�h�vd���B~� .x� �G�T8�w��z[�-J�Y�5P� ��"j 2,�������{�������b�`����:�#�� �� &����:J��cWpJ�4� ��!.TRPM:�PH�� �u�����W :�m�6��.;[�����AA��]%6+�L�8��6�*MA�"�:_���=u�
��]� O+����yY������)����P�s| �3Q,`R<��k Uny��g&�+F�����eYg6�����������qEz{&� �}����/�����$����u���� ��������+7����~�v��_��z�ui�o����S�)���8�(�����!.D�� ��Y#����"k�S�W������cAA
.t|���z�_� �\
X�K�N��"�n����
Si,b5D���Eq��}D�$�&������/ ��
����)3O�6+i�o,�����/�p���1 �J�uc�|��7F��\gr�5�.o�r�iH~0� � [� BMRqu��_m
�|�� ` :?UR�h�h��``B� ��*>���Xw� � H�\�O�<�����e+� H��9_�:���6
+�0�����`�� I����Z�,�A��9�������#���oO�������������� !sO�>��z���Hy�n��~�����N��'���� ���G�6?U\]! ��v��v�I �j�����-�q4S��Z�f��W�Hv�xV�����<� ,�����hoAADJ�N�x�������r� ��[�l��� � =����E��Hv����#\�01�������8�3��&�=O��K������v�����kd�0� ��qE��~8�m��,�+BK��s������h��s>�z�������1����r��2�����Z��#E�]]�h�x GO��:$�����G��� ��W�m���{��x $����:W[�����6z�� +��5A������ � K�������n��s��m���&��5���o���!� q`R�Y���S�Y���2uE���cM/ n�&IT
4*�h��Vn-�s�AV��s �@%��:SJ��V�=��R��u���
JM�YK�Pz�P{�*i�6�
���q��g���7�P�6������!����� �iv��i76 L��k������Lc�I�,�E#ET,*�9yb��I�#������ �|���a% 0s��� �e�3IC �q}�;�l���6�� � �H�����>
_|������q;� Hw"�] @���k+
;��Ah�������\3f �A�q��&E��o�[_]L�o��#�$i 6;����#��|Kv�bm�k�Q�d�g�w������2KN�Ty:�� ��'�M���6���z�'^�2���] �m�Rn~z������� ����v�0� #ZT��#����;���dDZ�n (���no�� ��Ol�����/<rx�(� F!�AA$B� �]�vm7�A��� ���-k����N1~^�Q�]�p�u��t�!#v
�w7��J��, �r�5H\�1L �Z��S8���O\�2�E�+K�JO}���� Ido>* � ����F�"�aM>�~Nz$�h���T��-�������Q1��$LHu|���SK�.���?������'��*>wv��� � � H ���Cb�� "�kx��&��2��{������ct �LMeW�U�Y{�M6�4���K���[���8xCi)D���f�v�sRn�;j|g�l
� #R�8��wui����6�����t���9 �����?li�
I�� ��-����([1�4�8�X���b����#; L:=�:�16�����^��R:� @�n�W�l
R�,$�N���_.��V%�d$� �)�����O>�d����|I� �t3=��{��K����1cFnn�OWK�,9uJ��:A�����
�| �ML7/�&A�p�v�yd��y6 n2ol�"�=u �9�G��c��Za���i
A����R"�^�?q���I����n%M�"�"{c#?'�h��L{?����K6�EE �~�Gu�E�g��C��lFc�O
N�z�WW�U]� PJ� )�����W���};*2� ����D�/��r������(�L�6��_���!� �D�P�� 2 H0q �I�:L4)��� $�dJa�1�L
O#�����N�>Rn 9������3���eY��b����Zk�IgJ!�Ao�LW$2���&_N�0[$��%KN~��"
�(op�K?�Va������~x��������U�NM�&����� ��U�� `��nM1� H1���������������R0o0�����v�N�u�����;w��w� � 9��\���O�_��_�~������'N����A�8#�P(�MO�&��9��C�$�fW�g���&}F@��]�Q���`�H�������$ ���%D ���#�K�}v�\_��lv/+� �bd���#�8��-��*����UYY���[����s����W|���9�V�<�?����>��5����!�������A
p��J������� ��������� �����I~��_�
����8�������/?�����w��AA�o�(�p8� "=B�x��vA���J$M��f�(����M��=�����z/t�`��g�d�P�$j�=�u�o� ����j���DmAV^�,���^m3N�o-���MY��Z\�sQ�0�$��V��Bgv���mM��t~����y�cUu� ]B��3��L���4������b��2z�6k�5�y'f=1"a��}�xJ ��u>��b�����AA.��D�A��L�����YVV6{���K�*^��;�L�<��nx��}>�s�gAz/�D��2k�� -C;��6 q(N-�hre���k��)�]U�Z](
J�~�_d�
1��:�s����4��A��\z���n�T��ZZ�p���5 �� �vc_5����?�� IDAT�H�X�����W����������!��y���2
GO�T~��������/������>AA4�N�ILL<y����g�}&��\r(�=��w�}w�=�����������x����~�����,Y�YA�^MT����b��6� Si�����-:q=GR�:6��O����+"��.� |��3��3�+�����t��<��� *��~[��� �x���z��)�RH�1��wc[�����1Q�� �4���&���� P�W( ��������< ���9M����OQ�AA�;�Gu��+�������o ?���5k�������E
!d��
p��Q� V�^���/
~�_������g������"��:v�{���������_^a[pM��� J�2"������s����w0���s�M��-*>�"qz5��x]�I�!��Do*�]m���B�Pbm�.�f��W� �� ��:b�i�����4�>�;�)������J�6�k�� ��Rqu��_m�s��[W];� ;���p�i+��`�'���U}�<�D ��_���P���/8���x�9�� �'�����*AA��Sf���/���c�=��w���1�������?m�^cG�z����n�������"��.��� O��#Ba���SS�L �V�=%�8�q����(�bh0�m�����:�k�J�5�> K�����:��� ��o�[,��)���� [��[�h^��������"k�S���fm~Jp�PV�A���f�mBO��pj����2 P8z��`d�[u�?��y �AA$�s� ���#KKK�G����a���oK����5C~�6l�8��}V�T�e��-_�<��"�\0��%!|wb�Z�,�3�*;O�u���]����8� l���6����=;:e��
[�v)&��lW!@(P��&�Ym�����^�5j%�� ����[uW����=��a���������h���������O {Wle��_�����:�^�� ��k8)5��x����\���w��a��fRL��������d����8x�-�� �\h�FA�W�|�����K��(�Z����9s���� ����?�<~[�.�~�Lv1��^�7���P>�A�"U����TE�f�^>�����_^a��f�S�Yw��� P�v/_t��P�V �,������'��:�5=��Se����P�t% ���`H��Xb-Co�� � q�����)�)��T�|���*�W�t���7�v�t� �����#B�$��5� �p$�2l6����]����������y������.�6���<1� Y��l.i�Uy�� ;O��z��WV�\y��O���c�?� � ������y#zTI�Ht���O>y���1c�H_��)S���n�`0�D��#
1�gAz ,�3��7�uK(�����l�t���Qo s�Bu?1�s.'����c�[�2��p����2� x�����l�=�}�ox����t&�������q�4 � �)��I#�s�%�~�����a ��9�H�f� ���#�C����d���~;�u $����V��aMV�w�*2,N�o��%OTl]u�T6zfe�-�wU��?y����J2_�#TdA�D'��������g�Ka1�L~�p��S����"9� ��`'���H�8�zs &{NoD&�����z ���R6w����=[��9�����/��v���o:6���C�B^&�9-T@����C�Y��<�����
������y
/���}�>�(m��g<� ` ����d�To�.^82i�����Y%Ao:�)U4��j}iQ�e�h�Lil��8Rei��E�'@�`�7K�����R� � 9��fgg��W��o��F���������O� n��}����E�]tk���l�
*��]���;�
�6.������]]*�%����__�U%�R�-�]h���q4���=���9
����eTR��3������a���*��+����X( �����3�"3vU�O^J���������u���#���1����O��{����Bd��Q������z
�.���h����si
��`\x��sye���y'f=�����YOH��������=R��P �K@A��s���� �\s��~�3��$ ��������u �<���%K�|�M�N���� <8---��� Y����v_#��l������I+�
�� ���_�U6��_�� ��8(�/��|�lI@�%�����{
��� �R��e�$ �����E�F;'5���� �,�e �� �5�_� �0�^C���H�w(���r� `����G�d���)�_�R�Jj2���J���g|���_�������
��+�Y��:
AA$*�e��������?~�t��w��]�w#<���3�����p�<y���� ����� ?^���?���3g�L����F����/Jo�>� rP�j�Q�,�ft� ��Uf�&��)���8�}<� �/M`T!������}E�A|�q��.j� ���2����p�d�����U��t`��0g�������i �9��=7f��O�
��H
51K�PP���6�����r��H������q'����t�� _?B�����=�GZ��5>���T���:
AA$*�e������{l� �q%%%���y��w�yglgA����Xw R� ��E'+��^� }�����e)5�E C 6Y@/gN�=!32�eP������f}��,A����G�j �
� �r]�@�p��U��������-`�����$�
(�����0������ i�6�[� ?��5����TU������yF�j �� A{
�AA.��2e������W�.X� ~�A�����:" yaM.)V���������3��T���~=��Bb����!8Fo�$�Ao����uT<.�OI�����k ��L�?\-�h������@��e��9�������^�qYm�4�k�^o����,tF2�D��;�p�TC����f07v8�d� �D���s6�����s �
� �JM�2� � �F�@2����b��BA����2�=���a�>���Yf������ H %^=�Q�9��%W��#a��Ps��-{�/��K%K]��| 4�}�� ��] ib�_!?E�F_�6��� �����������lM
+���q���P�jw��� �� O��<�~�������������=�@6�D��!*X[�F�~����������N�%t �Z�o�y:���Kq���m�3���>���2� � H��N����{�/_.;�b���mA�o��=�jD>3E��p�������n n�q�-���~Id� G_�P�OQ�� ���_o������c�N�����
���O���k�M�� ����b��(��$�l�:�
�Y�K�p�l�))�8�xxQC M��+���/�'�s�C��T���5f3W����>+�"�����o��(
��p{?��lAA�n���~?��A~�'1���p�p �v1p���Tv� /����@��TPg����P?����nw�yd��������"#�%��-�i����b���T��![��ub.��.? �������O����<W+n�}>����n�!$a�a��TapI���u��:1 Xo�&�O�d�gD�����B�$ \�7E�EZu�TB�#�W�3�������L2��+ .����56�q��Y p����a�&��3����FA���3��G��AA���z�c�����J��S4.���065������n��z
���[�V>�0ML!�B�EdE2d~ ��}���j���bw�3'���+
��J� Gn~c�������������.'���^�E{���\g� N�u�' ��Pd�����)f�� �:;{���?I�K�7��P� � �
�2� ����K��� n�p��n��L�]�^�H Y��R����l��D
��$C�� 8jd���"v��2� ;y���� r:�!��$��3����B�������}�`,
�]J��A��m0��G���v�?�>���d�"�Z�o�a(AA�nEA��6 7H(�p��� ��� ��t�8^`M���Dhc���/2Z����$���d�v�J$ �pzY���L����jXl��l��Az+'�V�n4��o�����D/����" M���`��\�lOt@�N�z��!m��i���<���` �C���Y��U�`� � � � eA.�)!��^�[x v�z8
�:�Q3� ��^�~@�{�E���qW���]+��a�X�D���r�qA|i�}��_# )
8jZ��' ����������^�7�-�4v.�Br���6�9��`���l@x��5>�"�"�[���sG\����� ��b{�b���dv
(]�H�xEeG�@�K�*�s�
GO���-��/�� � � �%���� ���T� �Y#mf�j�b���-� ���}!#K}���@e����"��Bb��\ z� � 5�� �wU����u���Y���Qd����4�����Xr�������7��+6.�-� '�Y�p���������M��������Vn�"�"�[j �~>X�h�t�������={b��y���p�5^9 �+A}I���� ��i,��`����2�����DEAA���/��_��(���{=3�A�>��z��� m>�R �l����%��Z8h7q��gl/1�D] �~� ��E���r{��
�i|wD��Jz
gL�j�3]h�5�x��u��o(0��� ����P��
�K �
=��D�)��K�f>&~h��?��� � G� k��k�d��,{����������;����������c�<� � �=��;�/!�\&LJ3o�t�1� �a�����dsL���(�MO�%���� va�,���_6���4@? h�5��gZ��|)Ll��������(2�"1��[j]-�������Q�A�R
A� �L�B��) �)�M�_�I���,Lx���E���
�@�t� w������ � ���%A��6�� ,�&a��\����xgS�+��S �b��6 g���������l26�e����/��*$:S�!��H[�fD��I�~��&c
�m]�����#~YGd-N��a�'��� GO����6�q����m���P����������� �� |i}���
�:b�5�d��-�����JKK'O� ��AA�eA. Xt�#�6��`��:������`� D1_5�������blM/B1�E�zKN>1�
��qpY��A3�9Uv
� �^I�����I8�v�Leq�������7eUVw�Or��&�%d9I�4N.J��YI�+����'��B�~��F=�6��"+����+?3��O�����q \� ��#V����6��s��/|�'�� �t ���-�7���Ci� � Hw���C���?��uA��[f�����*�p��2� �aD0j`��j���?��.ds=l� �k� �~�����~8�;G(�!��L��E������!���H+����4n�(�W:*� �$ T���\T p� )HJ��<+�X��j����>����:S��!�����=-A�3��3��-������a���Qb��buAv��+��X�&�f�!�.���������q
�G�{���AA� F�����g���+��+�p��U_~�e\w� ���9�eb����9�� ���
8t�#�$*��W����U)�(4^����K[����( �������J,9�l��L�YI�x����KE��U� �=E;�]� kKN����� ������|���=(�3�m\���#��w�`�x�O� ��$D����p������l�_�?�7���4$@�����w�]
����2� ��@�ev���r���o��n��}>&UA� ["���{#��u�P��%qF)*�G�'6)*,������[������L�u��&2����_���.s��Ze `�� /�*�r�)�I��_vVZ��R�����
���h�L���M ��"71�&e�������ZA b���
#� ���e�~��>� 11Qz��+����A�:'�E�v�e8���������b��R���@/l��*��i���K��EE�d��d�n�s��]a�7!G`�a[�X �!�Mn~�t�I�K�t~������MY�j�K2�$Z�"������uK���<1� �5����{�����x���
�m��~�����M�6���y��.Y�0� � H!:Q�f�� �4iR��� �����Et��s�3�B�M������\���l3�/��s`�^��u��������bU���j�#1��) �sa������X�$�1
�N�k|�4��'q mYZ�M V�������L8� �m�d]���C�M�:u�����c��.po� � �At�K��� �Gv��p�o?� �i3��Rw#TC�6� O+��:���������w6���LJ3��<@�|+�9�� �IH��� I��4��#��9WO/l�d��<1x�/e� v�H���E�*��hk���&�vi�gPHj����[�z&Y�6p��]"�0%~���(.[\]���)��G�6?U\]��t�����#���#l����?��OW�^=c� �.'�G#� �\�s��\��������?o4v[��;��w����Az��[bK�$�"
cN�3J��s���^]��fc 7��KZ� pWK�:I����wD�KeI��5cN���3��k��z&���X������o�cP�e���QjJ�yy�;v���8j�G� �6m�������.�` ��� � ��D�������o��)fs���_}�U�w� � �������I4��
%� Pi�o\����P
f ���MR���>-X}Z1�F1�7���uKIc�?{o U�����3�7������VV�T�m���f~�]�]���$���m����}���j�L�5M�Vw�����6op�����QDf�a��\�? �\��0��(�� =�k�s�� =�y�~�^�I��G��Zd��+g'������lb��������5mje��(�"�%'�������)E��h�����P��#D>v$ ��������� /������ � �� �F��`y������?3�3f���eK�Vu)AHl�!� �
�|�P�`���z��>��G?k`^RL�W%���Y��(�"���T����f_m>�� �bfXL�6}�]�����*�!c��zG�U�i(��arZ/)
�siG�����@S�t�M����.���,>��o�~V�� ��l�Khg�JG � k����n�����=
��R �4�5cf�G�������������ap�o� �5L��s^�\��H��?�+g�����R�+ �A��t���P����N3�4���A� �\��p�irrW�7�����w��!.0r�D�A0�%t���n1���D�b����� �bM��BCfv���8�� dq�R�GZ���n-\}����{�y�% y��U�S ��O�s�mb
G t&(>S;��E�_#OV��>����C.���?�&;Z P ����7�ZF���Z�C�����7��������?M�3O�k'�i|���F,^� [������{��W��v)�8���-RK�������AA�nH��� ��������[��{�"� HO������Dp�m��W� Ld?���|W�^��C�&q:�&�aV(����_�r�o��?��/�D����y���=9g��j�NvW��PT�5��Fm��twE���&�/�7��o��<������7&Us*�1�:kNA��� ��
����\Z��g-����8*}��
F3���G Xr8�� ���� ��8EE���������W�:� G��Og>
�� ��w�c�U�%'a�q��G������wX��X �AA���<e �����g�����q�����-;vlWT� ���1�t>�� #��od'��Vks����!��qx�^E\�}r��(�):Y�9��k�� @�RO�m�@ �8�2�J}j$���,b��c_V��$��}�=�W �B����|�FC�V{�F3r��S}J�tE�M ���j�]�-��G���Q� .��RN47*Z� �v IDAT ;v��[o]�t����#��( �AA���:e����{����7���;���o�����?_V�N��ei%??�b�� ���|�;L�n��2��&'�6�����eoG@Q3:�c�i<��9���p���A�Ql{i!����e�3�Q�|5 ��^M8M����* 0��e�
}��M�!��?�� �j��4YI
��
0���:o�-L���W�M�P�� xg�*4kE��������'N\�dInnn�+�!���m���A����?�_�Z��M�)((X�n]bb��0!!a����>�lv)A[AQAz��q���3S���� �� ��O+a:)gg����=5 �������X����)�0df��^��%!Qx/�0�2(��pU2r���r!e�%�� 5[���o���^���$�D���y��d����z@�`�x�
2~�qL�z����W��.���� n�/��#�S)L'�o���� ��� �|~4��h��MaG��A����?�_�Z��M��z���! ����>_�ipA�D>:�����P
^�o��� Qpf��h#pQ|v�y��#��.hJ)P�So����|� ���� �Q���^m�� �"��<8Z]�Ub<����qO�k8��E �{�� �o�i��?Yr���;��q���I/LJ��S�w���lJ�g��oJ[fn,�X����� ��$BV�w - 1��FD� $�L���j��7�d����c�-\���S�";3k������NYcBA�
b��AA��-� |
�-�� �������RoK�l�p0G��'���$h�`";��E���u ���
���/��|��/��#�M_<f��6d�(&7 ���������?�������tks���z����,� ��?Z�Z:l�_����������(�� U�u��� N}q
�w7���U[��L�� i\��U?Kz:��KD�3�(���?�m�8����h_������jy[(��� �\m�dH�yt;G�BNF�`#� r��M���O~�����~�������� ��U!��\�>���jY�QD>pD�;F��L'����^��.o��pbJ8xO]�+
w����� �a%�AFeLux�|�`L�� B"���|��)B�5 �M��5)b�U�S�����D�2���:Wp�=��]*5���u2n��vU���CR��<���O�/h/.Z��LI����Dj�(2���������%;3��/>�u9���@�}���� �\ �&�,_�|��i555w�}7 |��6lx��w��6A��H��G��]AGN �� `��1�f�tR����<��~`�8��8���(��$G����B�/ �C�i#���� t�|4�*(2�f~�c��N^��OR�_�OS�dka���L}��u���u�fdA���g<g ������� ������0:�������D�_%��.6���
�A�l�M������c�o�1o�< 7n����u:]��� �iW����t�gw�P�k������`.�tR�t` �A�
�fT%��l�:�O ��E� ^$������>��a��k���t�b[i�u�~E�Yr��s.�e��]u)����K<�����Xf�K/���E3:��H�eT�)g���W�����F�
$!� r���/ h��G yd��-[�ly��GQ�A�\~�W��4yy��o3����K��|�Az�R�EI� CFN����f��� �i.I �U0�.BT�^7�� X���o!M4�_�P��V�����G� ����#"�'a���N�:��w����\���4�(���h������f�F�Wo�� W[�����|[>�J�] HBA�r"fQA�R��1�� �� E`�X6���=<�W�R��O�`(X;��oM8O�7������ ������)���u 7oj�i�N��3� E�7J�}� ��^3BHP�}�1�j-jiW�� U �R�!�U��5 [j��
3�������%�7���q��K�^<MN���:'����.�r�4���s�/A��SWWw���>4�g�%�A�,!� ���s�=��S�R��!�� "����Y�����T���G�:B�R��K�#=yx��p��+�r���Pe���CZ�y���w�Fs8���G2�#����,>���A��|��E�z
eo�U �"nS�� �����U�#�.����T�W��.Y����;�J�"p��>7���� R����On7�hsy������I��;���UG��0x�s'��t%3��������[o��W����e�"�AA$����N�������m[�Ts)�=���\�<�Y��y 4 l��|���W�)���� zy����v������|��*��Q�
I�����lb 5�vSrH��������]�?D;�G���V�1 DD��n�{�%;^<�^.������&���bM�:��������~���kI�6���53cD}}�m����_�2///��AA"�=?�Ge� 9�#$|X� �I0f�"����#\���n� p�i��&�IZ\4O�p!J�
��@(Q�N� �����T���f[o���0+���������R�_9A/0�V\�v� <H�� ���+� �2�dd��P#ppYmVyz�G��Sf��J8g�U���sG466N�<y��I�� � r��(���u�m����K�?�����G�vvU� =�q�zJ`�Q���K�Q:�s�����U��w�~� �������a�� y���v3�.$�B��{����=���B�u� ����P�
�������i�
��)!.6^G?f�y� ,��@���U��CLR{`s�F����f�6�LN�j��|����=v���Q����"OC�f�M�8q�� /��B�"� �\rD%���5k��a PZZ�z�j���v~]� =�������� �ma:\��f]��Y�uK�pOW����2rO_
d���{�� kL��� ��a(�1��?����j���f�-�z�@��j'v��M��gS��-9 � ��,�
�������W�ls�tCq����� @��q�w��{j=���e�.n|I�nN����o��/�/5$97�����W�s��/��b��AA.�e~��� _���]s�5���z����A�Vi�<�w�]-:_�^��*�S�c]<]�B��lnHmLS���A�h���?�?�XgH�u�_ �] ��%��o��+[b�
���)��Zd;���Y�Bh�@��)@`����{�y�%W�[��� �@l��5W�M��Q����kk���4�* hm����_S�eR�G���� �Ld�}���n�Iq}dA�c���� x5�s����C����AA.�eD����.�A��Y!J���%B��b��0T.}�c]<]�\p��
�����F3,��M�Z<�[Mj�&��m�o��!*�~� qZ�@����9M�&]���� ���X��%�5�-0m�:U�S�J� �����%'A�$���]R����� ������/*8�Q�"��w�o������3�4��zG����%������]b� 4�}�X���|�>�<�����������L� M1� � �
��2 ,�/� � H��F�'J���p�Z�� Gh{a;w�u��u)������Y�����2���x���M�F���CF���0.���y��T������l kRt�9��6�M/!���������#ki�5�`��^ �rNCn]�*�o�|5��[t��zT�]�E�
;�E[;� �@c< �r��]�X��� ��F� ��G!� �\Np���7�x�� � "r#����t+������Q�
x*�6{�=�G�K���..��Rs$���w]��T=� ���t ��R�����Y��|$�� 4]�H� �V���^�8m ���������%M?n���OK��Z�Y'��F����$���������o�^� �>r��� �����}��t�� �1��J{dc��ZV�P�C�n��*���Nh�S �I�}6k �Y=HY��<PN"���D8
AA�r"�NA� 7|�{�� u�
*�)&�}��i��i:���[�X��(fQ�N��/~� ~����W�@�`�T�KT��(��S�IW~x�/��_cRu���Pu`
)<w�k�A)JI���� ���� D1��d3��0}(�j#��hT�����$h�@���.�33F @����K���"K�������7�6�� �q>�1�S���9�BA���M���������������_�q���#G�������Az&��/A�6��������&�Zq��&�[��T�L0��n3�����[y0��DI�Q�8 �����L ���o��T�N�[MdF������2�t�)�� ����2���<{o�E>p ��#� QF����@�F��Ts����W;��]- �L,������p �����E H��D�AA��!����k���� ;v������ �:u��]���_�E�]*�V���/v-�\� f$4���y�v�P�����&_ �E��9��rW�oM?�Q[�v�}�g%���#��|��M��C��pa���
/��wP^��&��LX=�k �{��Q�����@��� `����z@�`-�!q�����U�*� �ee��PA�#��Ci� vQ ���>�g,���O��f
@���zo��V:W}�O1�I���'�+Wdr2���X���|��"� ��#??_��~�kQ&6Q��/����O���������{�-��Z�j���]T��mEA:�q��x}����CMzU����J}�0/`��.)�;�yxA��N�`/���SO�^�)�#����R��Mc��� �Zm L8�J���������r��R�m+�����s����O^�J������������0��e�o����B�edW �d�M�� ��>�;R��2IX ���,}�
��?$�����f�O��G�� `�������J� �t������]�2��/�\���8 ��w�K/� z�^��uIi� =��W��� ��A�Tw�P���!3U 8R��������������8�$ :��S�!4_ILk��,�k��kT��`�������k ���� V=<|V�t����>g L� (
4�|6��S\q(H��I"W�XW�]���<��������N:�$���/����-��B�]�CQ��� AA� Ml�Lcc�/��f8�e0>==���B�����9w��m�bS��g��)H���%~���)c�"�m����0������
�7���B[�b�h�����^��F�� X���>6�l����Dr��\
��.J����<�2�5�Y|�}>�/E ��. 1���� Z[k��4<�5�G��e�A~� ��"� � �(3t��={����0@X�y���]P� HOG�]��x�)&2�#o��� ������dD�y(_�# u�1df����XC)��By��/spY��v��Y���'�j���W�N
Ys �~���p��E�y���%'����?
��j�!��@�'���v�@9 b�K8�}�Y�o^R�s���Y3���4��Cn�~��I�9�'�j��YC���ZAA���`UMM�����nwaaaff��S� {��#F������9��� "�t �|�)�4�sfW�#�]Ql/}�u����G2�c;�|sH �J�R���6�M x�Mh���w������ Wi��IIr�3�#�a��&�Iz[6
�
zM������U���s.�����*��i`���(_���"��}T�hP�i�h�*m��U�i��7�d�h�*]����� Xp�����5c� �V[\\�V��7!AA���{~x?���N���~k0~���vbM����� Hd_��PbC�W����z^:�����(R��!��r�����z�<�}"�^������A���t�e������E���7e�$>S;��$�5��"�� ��E�|1i��`���m�!@��Oox�h~f��\S�_�J���ok4��W� � ]J���~^�2���F���RA���� ����Bx�y� �sfW���< �e���7&��4L 6�7S���S������7-��{'�}_�����n/RN�R�@�{��5�s�%;�L�������V����Fi��v � ���x���8AA� F�L8�f]cd+����G�,$���FI���p�O� ������Y� @�L � �X�l{�
�a�yJ!_I����0�5�s�%;����i{�&����cui�� �, �Bi���/..FEAA���iwuu������� �������~��;�tMa� Hl�1�Y6�e+������]QT�5=�\I�J !���?Jh<qT�x?���[�1������l���PQ�kW}*#H�4�����^�f~��N���(F2 _��� ]<'JEr��Y�g^E|���s�)q������N��7�T
�Q�^b����
����W��u�AA����D��k���� ;v������ �:u��]���_�E�!� �3c�I�j��,�g����������J��x:��.�6��������
� �(����t�o��4���k��;���+���^�:��5����
y��Z���]� ����!*=�]T f���Vi�lL��M4��uV� 1r��<P���)����d�F�l������3���<_7�`��_)�-�b
hT�/&t ;�`�� �R�����_����x � � H'6��������8..��W_5�����x<S�N���O���nN��
B�g��v��������IV�X���jWq����p����D�����H����JG � S��:nH���O�Y����bN9)]i>����w(�������qj��C��� ���������b^�W�����1})'#�����:�8+����y��Tc�!�}?�)P^M8�Zk���z�2�����~t���8w`�������z=��#� r��� �c��q�\qqq �w���^z �z=6$#�t�����(
w�U{��j7����o ���T����6GSOd���X7�����%���r�F����~
���^�nh����5�npU ,PH_��v,#��~=V����>h�
)ju ���`�+`��JCLpf� |�V����'�15��
FL�_�U�,T� �=���[*����E�
P�M��6
����yCG�.o�Q��C5��� *2� � �m|���Q��f�
�b������E!� �G�����M�3c� %�A�mOu�<���_n�B8v��]�_q�p~cRu����Q�!T��Hm�h��?k ��|���H�g|�����k9
����z �d�I�}���7l�q��53� Emj����8�dgf� 3]>|�S����������`��jk�[
����?^��z��������-+����eEQEY4�BA����Sf���{��IMM 0`����|�����0A�s�cP�iQa
wc
K��d%i� C�n������%Dc2d�uW�)]16�M��{�����t����n;3Mh< w#��g�i�'�pD�������P@��d���#k�?�&��K��9P�"���|g���m��~�8�MS,��,1�dd��}|�q�-����FH�f�r�r2���G,X�����;v�0���*�8����I�����+����N�� � ���2/�������nwaa! �:u��� 1bD��� r��1w�Na�P��POE��(��l���e�� ��v1df����6et��La����E�M<����>ni�F3G�X� !���������vU � |���(2 �%��~�� NyV�"_�>��+( �`���um��S�E����g}^L��������������K�@c<���� >r����;�f�b�ee��cS�~_���Q�AA�Gq^>7N���o�5 ?��O;��K���� H�"z��."Iho���n�_76L�1m���o�!����)~�Q[���b{���V��Q����f�{��2�~qL�.�? k��i�J�d���?{}��|�m�|�-7
z
�6��0�g jT�� �U�y�i/WdBv�[6�M���TKv|��C��_��!�j |�`�I�- =]Q�P��{>� V � IDATg��C)��?���<o�Z��������_����;��v�o|Rz �Tj��?Es-� � H�t���u�0���Q�FuV)� �+QZ�v ���K�SR�fl�Kb�
j#�c $~���Td@��C���=�w�;�$b-���F h�~���������w�3�������l�'R`�o X�l��9O��]&j������A��� ��r���%V1��(EeK���"�b0�� ������6@_�i�����������v��R��4c�4��u!� �\���(� �DC�;�t1�%u Y)�����B�P��WF�<��qW�*d�
!��5���j�f�$ZI��I���)�6`�l�d�[r�#%J����9 U ;{��8�BF�IE�N�+ U.�����%2 ��#�������(��>�hYY���0%)L�,����>�5��2AA.'bK_���s�=K���#�<r��� �c�S�YT����~QI������>2����%��n�!3�hy�H�K�oZ��` �~ �0�4~C��<}I�J�il �k1 f� �P����Lj1�Z%�Jg��
^�w�ZL���� z�n��E��k���Xp�X�M���'���o����{���X��r�X%!�i�9Q���W���x>� � �eLl���_~��3��_�^e��������]P� ����j��o ��`������o {�= 0c�I�j�p,��
w����?�o�7��6&�Fm ��KJ���N.�Z��g_��EQ��d� �����I��{�J��UI:����:�Z�F���
!� Pn�Uc�������us�k������ ���'�|��'����J�;�q�����u|������X��AA�2&�����{n��5�/^ p� w������A.I�y�Dp`�0v3{j<��i
JZ4.��M42r�^IJ���^Nu��(m}F�H��D��� ���wm�x���"����$Xr6
�F�c"7H+����Lkv���Fq�h�����6K�M��n���%r���>���QxY(y���v������w��
%�� L!� � "��2N�����B������� r��� ���<�f��U�W�Y�?G��0���6� ���S�?�p�sM�R����Z>a��K����[��q/
P�k@&��t����� N`�ePi ��
R�����
n./��~�����"��>k��� �M-�^���t���;w~���RE��AA$b_�hF��|`��AZ������ r1�� �Y��D9��4��51��u����������^��.� n�v�D����� ����7m����- |�f �Xr���8�!q��V��S�$5��x?uT�Jrk @x�� �~�J�dD
FL�Bw�~����� 0������ ��������y�
H����B�f��e��m���O���#/Ijj���� �\D������eb���W�^'N���|��g��L����� ��p"x�t�������O�[��q�~����D�<��y8����(���~z��e]EE�B s�G��4�x������c����"�����-��O��W�1�j-�Y�l������S�>m��p�����sf �,����4�����[��9���,a��� �]0R}$�� � (8�`��� ��7//��>PTd �{A�N���� �/v-��6���3������L0��m�k����{�uMm� ��c:p�0v$�.B# � �|���S ��L�����nh+�.r����J�PhiQ����W�q����NZ�l� )�b{In�0�����~�����[~����H��-��@���g����@In
b��d� 9[`�9q���E}����`�����,SPP�������o�����f��20AA�'�(3t��7�x��'�<y��=��s�
7|���z�%� =� H��;���a�:���(��[0G�f 5�iw.[�� �M����g�������Fz~����s�Pi����R�D#��q7��<���0��Ao��a�,9 ��<����n���1��I�5�t� �t���W ;3K���v5�~�ld�6�/��bqqqIII�~�:��H
��N BA*� � ��(be `������]Q
� "E>vd��|rnM�i!�����s��
3���\D��/ge�om�K���FH\b��|�� � �����= � �R;a�����������HQ%��_G���w���F@*�����4�^��e#���,���"t��64��_��]��g��������b`6�2� ��Xbel6���HKK��jAz����cGR F����^���������#m�����; �|���2�f�b7
�����7L�����EF�A����c4�� �����g�����w��O���K>��o�� J*��<�Z�1E�2p����0�4�v���-�,�����
�@�0��M���|��A�Y|���B~6q�Z��y���
KJJ��k+�� � "%6Qf��Y��/GQA��Q���:�0cGz�����,���z��UE:`y� ���"����� @(�R���+f-)��{� �
�9}�� ?����;����H 0� ��sO�&2rT:n���r!F�������� :��z/��� ��F�]e��^���7�@��;
�s���!E��k�@ T hLlh^~��7�x���d����k+�� � "%6Q��� >�Y\�d��+:�$A� �bv�T��k(��y��t����#7s�"���}d� a�(r��M��g�-��Ga���X�l���5�����L�� x �"W��A�i\�*hHr�={���z�o�H/QlW�9��I�� ��K/w�Uox�P B�%��#�Q`�������W^y��W^����RSS�
�V�8�� � HO&�H�� =z�Y����;� A����2Fn�;~��� }�&'�6� �mm�Q�q�"�dT��.n������� @������� p�d�)����BT���������"a1( � ^��9b�BT7X;a� (��qT�Ro�6v��&Or����E��*�Q��9� �~�����7a�.���o��'�vy�Op0+ ����bs��Q~ ����%%%�
�>[0b�I� F��``6� � ���D�_|��G �����jAz}d&�lw�U{�~��wy x 4 ������\1df-��M �g%�l�4�`V����������^��.�l-�S?+m�A�7E�}FqE����Q��Q+R,9 B�R�� Ys
�I���O���47��{�_�`�,Z�l����GN��i �|P��M� h�W��fm_���i�o���}{���Y��s����{��� PTQ��e�v� ����:���dd ���7���3�` 6� � =��D��G��~���&M�����R[��N� �����&����p�Y$�Y�h�������!gg����=5��)�"?�o ��R�N���T��VQA�2+��b�Z�
3�D����y����1��T�a����}S=�T�i(\q��|�����VIEH�z�k*�[z���^o�u1&��b��t��i�Y��|�b��yxAPe]�y6x���EY����! �&���&��+v�������T��F? �k�_?9���������Ee��^AA�!6O���~z������?q����r�:�*A�� J@�& �
f 7�js��hf����%| a���"���1�A��H�����h����G��FCf��"���;�����[�5�@*� PcR��1 ����' ���5��9�rARQ���7����}B��4��������<}����p�x���\^]K��|�< {f��+�x���G��� �{ ��+0dH�u����~����Q�_�<� � \�)��N����w�����!l������ oR�AA��D����E�1��v���zAzR����D���� a�Id����^��0��� ��fQ� ��e�;� �a��l2P�:0�t���k@�s_=�u�'����7*D ��� wC?sJ����\
P
����BL�<J���>T��$��1�n�?���������%'#K>��������Ds����*2(����O9R���w�����)>%���u;B���N�FAADb_Z�n�|177���A�)�:������j���3����\p�df���G��B�_�� ���i{��~�������Va�K�� �1�s��M���) ��{ �����6
�k�3YKNC&� �D)�}1 ��������� �^�{� Pi�K*
K�,��k�T� s&K���� � �Hl�L�����S�N��bAz
��#�K���..��Rs$���w�T�h�k. z���tm�U�`��<��; �z� �!3;~�j��Fm |��<�&�q�����1�s��o6�[|ZxTu`J��<W}* ���S�6� ����,HQ��~��M�m�����Y�J=r�� ��Ty����O?�����j��d} �'�TlAAD$��%�3g�|��&�)+++!��?�!� G?@ S��\�! L���;?47z�D=���\JqN��l ���b�9K ������?�oi����4��Y�U�r����n�w -o��� ����� s��~L��$��\m 4U�S�J�����`��J,�k���R5��'K{d�� � g�+4����A�5�_�1s��p ��34���`K���9������LBa~�V�'�TlAAD$6Q�RZPP��W_M�8��r-_������eA����+b1�8R��u�^:N�_s�m��E���AW���j�Z.u���TD8tQ�@!������ S����>�����M�+��v���[ w�P'�_m�� # RJ�W+L0���
�d��N?_t����3�1 #&J5 p�|Ee �f�f ��
��f�J��35z]����
S~?��(� %�aAA$� �PXXXSS���G�`0����� ;v���]Vaw����CAz8��=��R ��h8����mW�[�Y#��^E�_2���g
���M?���[�t�������Dj�1��*r�"<��
4�KW�qC=_T��������HDc� �F�WD6
������Q���{F�u���Zd;���Q��-����S ��!?8
Y����Ee��_�m�Lm���������� }u������#EeRe$'#KX�r� t�1a���Z ��C�~DO���� |�5��[$� � �%J��� [M'N��s�J��c����N�������A�n�\
!�~ ��<xm����H'��'k���M'��Y�m��p�i2k��)(
(I��1�S�9�� �^�
�����|������U���kO����{ d�}C$�
��{�� 7=�J��q�������F�Z;\Vi������'!�3_���vO���+->^PL"_=d���;�+)�8[��
�y �T������sT\q���[�B� G�g~. ��p�~����R <��������v^� � ��L��� ���qRE 4
�� �("�%�.��E��!Bju4�/�D��*O��c1� #�|�]i�y��Td����a�Hj��6q?'k��+��Q��we�Y��]�9�|���V�N����4� ��KT�} ������ !��x ��l.�������o��/d*U:s�o%"�F��~�<�wo0 �O/+�% J�((�Y5�|�>���l���'��k����GIAA��[��F�q�B �(��_����B���W�Y��C����w�4���_T�����\ ������Z!�9����lw�1�]i�y��Td��k�1 ����m����u?pVR� pV�Ks�����'��T�������9��C�9rWij�����I/LJ� ����Zk�mc����D~��l��L�r���� �8����2�Wr��V���{+�O�E e�Va��T �*(@��AA�."6Q��� ~��G|��?�9 �y��-\�Px������C�\`�����K�jKLI� ���0rE6��Td��k}��R����^��1Q���B�m��J� �����No��?�W��&���� �c�O�=���� �^�������9��r�o��PQ�h�i�����4��<8T�ah��>����*��� � �\Hb_���/� 9r�u�M�4��rm��m���_~���_~ ?��� � H�D.�(B ����4S�jK�I��lw!�:4bif�O��$���!V�f��q�L6 �� ��2T:�U��^u`
@��A�����e L�)M�QL�2���0� D b���TK��g����Q���Dt5�%��0AA���M���m��/�,>����������s�B�fD���\"'N���|0�UB_�T[f 51�K]7�+��"�_����,[*�_K���<�J-��k�i���r�}�(?�����!�* A/;y4����s�{�-L&�VS�� "��T G@�.�8���K����>D�\��a��n>Ma� � H��(�p��Q�F1�'O����+ `���V� H��q�]���P�5 ��K4 ~��8
g �l/�/Q�-�R���T?{Aa @P[�f�J{����B��`������Oh�|s��W��B��h� ���0kn�4!��1�]��� ���'8�rv��5�4@��\uB��Fyf�
2�����1�ZS�����k =f\~�)2zN�x��mK_�h�iG( AA�:bN����<s&�/�O<������U]Jt�T-A:y�u?���[��z�n ��nb�bt�'@����k[�^��[��pW�K e�1��*�����OP|m���4 !*?��x���������7���#��
���T�qW�����:��d����I����_u�N�SD*}[����M(L�d�oje���3���Y�y7���/���� ���y������u�S.�!�����?���n87� � �eF��� [MK�.����G� Ia�;��K���}E�����ii�p�irr����^�:�^���rs�63�K7Tg�j �4 �t��~KAo���pj�1��yw�����(������8������'��M��5�i7mcm���%���|����V��������k#G�)?D;�G�xf�K0q��ZfFk�RTQ�����\6�����������?@���Z�#�0���}�����{���+�� 9�AA�2�{~x�m|��O>9p� �� �������QkA����=�<O�(�T�nuxyfZ�Sf�:�@�)fE0maf�������QH�V8�U%|��(
4U0���Zh�g�������Q���n�! ��dg���Uu�>I}�/^���<��l��'>S��Q�8����Y� Kv�pfs�����<9zE r2��
,�
��@Xk�I�W@q�!��-�\�R�����w�B|l~����'*2���� � � �=�����H�������W_1��6m��z.Ih+�� �e���&���3q'��*� 9�:�l� ����)�ov ��f���9�4Zmh�&<GD�;m�q�%'�yx��AF,���0jy�)US�k���4���=a�7��� ������q�����
�NX= n����y`��%'a�q��0���=Ee�[Vh7�!}�����v� dJl�R�$XT� ,+��2���T�����i�q}��� }9]�;$
0AA��A~~����b��Ll�2����u�]w�]w�t:qq��m�f����A.�Nu�{�����a??K�����������g�+��H��) �,�h�O FO Q��q'� ��g[
�5�%Y��%�5�����W�[�Xr,9 �"[H�KN�&=�0cJ#�
c �Jq����-�1�����[ �WE\mw�~�pq������������;Rr������>/0~Du�%�3�AA���2��w�����1c�H'� ����
A��4���[�D}�'G����Y��A�� JC�^���������� *e
�W�(�) $~�j���w6PeA� 1d�1d�l)@66%��wq�(� @��?�Wk�� A� W[�����f�o�~�Z� IDAT�����\~�d�\�!�����yZ�����{�� ���0� ��0f`#� �tbe�������SA���i�N�h��L���(��/<���.�,���v�H��\��,
zJ� CF���85�E
�l��������K(�Zlw��� �0�Y������R�V �����5~_���)P�$&QO)1QQg9���\r���Y� f`#� �t7b����o~�r�Jfq��y�W� �����:`3.U?���$�J��$�j�u-���<� 0df��^�6&�Fm ?z���n�O��/4�064���<�L��������f{��.S�~�&��c1z�� !�0�~_���l�r�X �Q���@vf��1���&"�s hq���,~I�����:>c�o���g,�<lAA�nBl�2/���W\���[�,s�����
A����@ }"OKEx���e�]3Qn����S � ��E:��f=���B��K��# �rW9������,[�0`���MY���u�]�D���L���^�����h�'@��$ � ��f����
�Ds�{���>�(��TBA���&���;W 0��3�tZ9� ���: �'V��{�eGC�y�Y�<�y�u�T����� �G!]�p@[ b�^��wA�8��$c������S�g�m:��+p�F���E$Yor}� !P���[k���-�J�����O��� ���p'�AA�-��i�������[:��~���� A�����@ ���W�Y���� �TPX�V{"_�=���AnTF�z�6A�F���`Vp����5����
yU�IR5���T���x]~r8[���j���_V��H `�hW�t��1��� ! �B���=� �K�PH����1m[[r8�6p� � ���:e �����g�����q�����-;vlWT� ra8�d��}.
����Oc����q�L����}��O��S[W|�w1+����_|�fc��j���1���� r{��#�y�k�w��$0��(z��dd
������W ��A�LWh�W��Q�D�AA�H�&����=��s�7oNLL ��6{������,� DiC:I��R��,�..�I�(,�3-;��A�xlN������LQ�>�
�>���B�����/����V��m�� TH/��x��K�W��q���4�c!qH��E&�Z�8+�O&�v���`�������---+���� ��Dk����
Sb�E"� � ����
��['(2 ����v��g�U�:E����$Q �������Q���yx������}��i���\���o�I�}���7l�q p�$��Q����Q�;�>H�� �OZ���j��/M�gm��/M����O��� ��n�����,HQ��~hF�o�W �P�%�-�E`N�
r����& ��1����c�������&:����3L��Y��FAA�'��2^�799Y���������G����$Q���� Cf6gP8<�R�i ��A�Q|(8�2�
�S��;�~;����;���O������-�'Vs*�>�:y� ��D ���� ?a����ZNC�k'� `���o�#/ � W[��
FLd$��������3�@�6��>fp\o!'{�9Q����&�"AA� O��2� ��n�-E>Id�r�J�/r\�����h�h�� �����l����W\�����_8�����+uAm�.��s.����=���&� �o�.0� M.�5Y���� h���W�f����Wjgc�I�)[D>�T��U�-�jg��)��k���s2��3�(fqt� ��(���8�t���$6��dd �
� � ��!T�_1 �>����So��vqe��� |���/���]�{� H����S�i������t{�=��m-�����M��(�+��eK��j�1���\�u��(j:��z�R*m���V����E6�AF���>/X�����V�F��k�W�C��� �J�Z�#y������
�a��?���R *B��J�x��7��������Jw����hJ3'*�AA�� �c���pL�6m���w�}7 |��6lx��w�������N���"H����jC{R��T����b�#���������)�8-����tCw(����(��>(�R�)~T�!3;�kE�E*��<Ul/y0��wBa�%;���.5�Ud����SNJW���>�����L��c���W zp�i�J�����!����2iWK8t*������6 'N��0a�O<��CE� AA���=?���)�c���6o��y���;wn���=Y�A���
s��U{�~�pxy
������E/���<�LTd ���������Zl,{So�6v��&Or����E���w�0vy���w�- z
�C��F�:+����9���- ���[��]�$���,}���{�1r����� <v�C���-+����eEQE��~��� &����GEAA���;
E��SlC�����50�-)&��� E���� *�tJ���LG�t^)��>��E���G�R�F���T�nH>�����������Dcr�+�>�3G%���� ��?����V �Fo 6����-�4U��80���8r� )?�s���5�\S���'������b�����w�[�pa,g#� ��h�����2��y����O� ��� H�a�P��PO���-o��4\������/r_�gm� �w�~��6���D� ��l"X����b{i�� pVa[E����B/������Y���#\
��(���T�1� m?G����_������I�t
��h[�lTi�I��@
A/_U:E�J<8.];��v��@��$��q�Q �48�>���~_����k�n������"� � r����/�x��W.^�Xx�v��M�V[[��!�DE7���cP1+f-����K����,JQ � �O*���`��������V�$>\��co�'�Q�tpVR� pV�Ks�����@ ��P^����<�Y�k�R�� ��O�$�*}�����d�)0&Us*�1�:kNA���@ m�Y� h=�_��z����;�����������l� !�"���4��b�hA�D�V��o�Q�����ZQ�VEBT"���AB���"6@
��c��9�?f3;;���Ml��|��#������G��y^oY �*?dl�m���H<S��Bo���i E:Gdw�I��?������3�,\���'@!�B/���U�VY,�E����������+V��8��"�
!t F/)h"�����A��9��� �S�~[ft[�8;N��H��$<p�'� K]q��G�(J*Q%�[r,u�m����Z[�}?����T gB,[1. ���#�������e�n R�?0^���$ �$��
���g�Rh��%-��4U67��_��wH��K�L� ��=�\XO�B!� R?���)�}��g�y��� ��w�}����^B�a�� ���A��g� �Z����������~����]N��������1�����7���D38aZ}�Y��i���|K]I{y���V��q ���Udx-(.�R|�2� ��~���=/|�.��EI����8���6" (1^-��hh�j��������h����/R@>8���? ��r8 ��cE!�B�J^� !��i��"�%B�2;P�oH)�5 ����)���c# �!���X]���1�;�$��������������a�����xt 0&�����br�r 2�M~W0�������[�4)RA �+�d�i� 2}�����%����o#j4�@^FC��/7y�f<
H��y�� E��v�-g���n$��'SkbI����� E�6�&�dQH��B!�P_vQ�a~� BE�H E��4bq|������g�7�wD���i+w�� ��
��IG3�H_�� �N�� �Ud �6g�2}&���I������������ �?� ���E�<&�@k� ��|/��Pr���^"4�A�����3� P[�?��6�|W ���B2����U�`� �������5�=��� �E�e����=�6���,��
I�d�:���� q8��>�}Q��B!��S�+��3���?~����#���YY~�[!�.%��$c.w�/gL��_@�q�zt85#�8'��"������� >�����%��7�(3��0���fBw �};I���C��M:Bn���iG7,���]VL�Zj�����m���<�X=�s!�0.�������i{��=d\�DMe����\/i �v��� ���F�_PU���Y�5�;���3�G
1�u5���s�����
P�YT ��&��� V{�B!�P�^�����7o^���'N�HQ��}�� ?�i���ySdf!t�y�@K�����'�F�g�v?%�������� n����~� �Mp�H���u�Z�����C ��`r;d���e �5{)� �+�q����$jj���ck���() `"Rr�{���1 �ZqLP��� `�[����%�� ��z��?Vt���s���F�����7S !�B��"��{w�TYYY]]m��F� y��w^���"�����x����/�rzY +�xc������B������d,u�\c�l�D����}���(��Hd~Fw3n�n��G�� �������MP'K��'f���-����$���L��k�z�;�AY�dD����Bo�B!t5�������%2�^������O��� ~�E�� "H���X�J�U�}N�^����8���/ q6K0��oy M�.. ���'2k7������y�&�����d6q&��
2%uG� ���ro� ���bd�v�5HB
_mq{����&W���#�B]A"��{$��o���W��6�~����RWl��"��a�(�E�|�WXD 5>Q2�-C� ��G����'��5��p~����x;kj7��49Mn�S(����mk��(���/:k�� E?��7�wY�(�;2�r3?Jfm��� o���GE�C�� I�Q�R��cq:���K �R��������H��D_6��&p
A]�!�B��� �� �E����$]
K]���1�e ����+ �d1�X_�a2 -Lx��8��#���47�����683N�m�{�_��� #QQN� ��I�9-.K���C�AR�����p�I�M����\��*2r������9� ����c��aO����S�{br� ��R�9���_�r�|B�B!�PD�� B�
!E��l8
R@�3 ]�|]6����4��1�}��$�A��mv���}��[ {����� �F�0Ag+�`twm������CD��Y�8�jj����K�����`l.�q?���E����/';A��~%� ��P��KoE v7 �N`����?���Fs{��O �����B!�PD�� B�m�VX,u%�����il�W C�y]>����/a���fx� tF3|[J�F\+��`��v|�l��� an��me�yi�ptw�M�T4C �Y��D~jLmq�� Wn {�=� ���^>��A��c$��� )�,�������*> � #����>����i(��m@�� ������9i���� U������B!�P���-U}KdnKCE������6Phn���t�r��J @)��1P�o�����8�W�!{`�W���v}��5�����r����Y��K�i�����N�[[�}M��������M2��Q�V�{�Z��������f��{����^A0
���iFT��#�YPU��Y7j:��F ���B!t��������?���p����o-��V�d]�K#�R��Rf�Y��l�*h2���1��nTX�~� �.$���@,�6�+���`��1��p���Q0
RJ�kdZ"��
zX��^���� ��j���=���e�}R`�� _��}� "�����,�y�P&+��}��>~^ P@������K�1Ob8} !�B��t����"��,�`Q��"����@O hi����Q��W�_����.�������#�q��$:#a���/��� ������_�*%�n�1��� �{�� ������C�������)�y�%N�� �����/�������x \;$�_���+�������^A!�B!����)�B!���#V��E �.��F���sd� W�_�bqA��������KP����
�2h �����,^C���� ��K�@���V�����)~x�1�yZ`����
?9���6��!.�&�E�K��?�����*�n��#:��SL�~��eY� �Xp���>H}���?8X {r� ~��B!��<��^ B�M���tf��������5u��G ^Iy�-����+����S� �^D�j�`(eF�6g�D�J(�D���Y����EU����-I����l��Y �?i��V[[��� 3�>�� O� �-n��V�JvL_�����}�����
�Mr�6O����� =Z��1 ��D�u����O�P3�G�����de�|o���� ���eY��R�B!t��� Bu��&+��3��">���uaYK� �����3�����j 0z��<��b�*h2c��[aQ��'L��?��0�>���p� F�� =�� S�gMx}BJ�. ����� cK*�k-u����sd��4K]�&���A��� k��� ����C��lC���`:�[�����6���ihA3g���G��*��d��#6��> /nd�Eg�FEK�h�x��[.RK���Gosi������Rt�&v��iA|B!���'�-U����!%E���W������]�9���d�R�qb"�H�,^�9� (��X n��~3e�5�wy��O����p���
�5��=Zjf r��5�6V{*��_� �f��?{b�8/F@�$�snX���6�����W9u;�����cZU�n���*���"����/a!6��������*�c�0��Y\\,��>b�B�K!2?����{��w������x �s"��!���9�r�7g7IM���x��@Y����/� ���p_\��K�������~��v����~�d������G<���?���:����
� �1������ ,������_N�O��_R�l5�+ C�����<���n���J��M�Ud� ��B!��� ���:��&���������J�- !�"����u�]0��!�@Y��� �;P���1�M��Y�_���_c<���� �0��R���*�����3�E|�����W�*�_��2�Y����}���" ������4�����O �W-�ihy�%�F�gs��n�6m�B�R�)�;RPYjr����VPYJ���� B!��(�L�A�������~�m��!�"��d����T��" *z��>�X��x����1������|bq��in �B?���I�o����Ns���niV� U����&m�+��K��Jq]���F%I�T{��{(@����,<���a����.Z3j��F����k�J v�}���2����$���%5{�� ��� � ��!�B��
�(��o<��3g���H�A��5v�b����� +����7 ��DE���&�I �_����"����ps��� ���_ � w?�� 5rY��LZ���TeH��]i�kkAei����v5� �rS��Bqi&3O;n����.�vX��v��$.��SzT|~zV^F��Q���^�����3� ��3$I�� �6��ke�!�B�� �(��?�u�]&L��c��N��!
4E�+[(�4 �0�7���_�pwT���!���F�{Y�v(�`PT��5�������Re�� ��u�ynw4����~��w�/L�"������ ��TPY����.� 3m�M���A*�`p�?�:?=k�-�U�.�;������y���g��e��7m�$�v1IJ<<�m�A!�BW��rn����i�� �Z����c����$2��B=w � ��:��������(�|�.�g-qG(i C�����0q��8l�o�3��t^Y }��AK|n�r����&��\"%w}0�f��~�$�sv�K_=eh ����{�g�L �&��6�C)*�����4 g�xsS���W�=G�$�FII6����Uw?���#�?# l:U���dG IDAT�����Z*[7j:��F!������ ^�oll��� ����{�A�K�o������3�Mc! �b�.+^I �&Z�[-�88K]����M�u�U�eF�2=�?���P���'�7%g�
����;okK�WO1��>��������'b�k�9�V����_��3%��$����=J�� �?sV�G��~ �W�w��x�����a�/�����,����#�+7sY�� ��{'9-i�[�]F ���=��6�D�edo^FC�?} +2!�BW��
E���qqq��;v�����zuU}Id�B!�? M��5&Y�|��9�dM�����������t��Nyh��Z�I{J����Ki�%���$=�Q�������]Y����]6�����;���� %%���� �:����J�L�NQ�'�� 88�t�I��7l!�B`l�u3?�~�z�d�S���' aC ���8*8Ukq9�-F�A�=;!�B�����e��+2 p5WdB}]�(����"���pu�O \�YK� bg]�� ���\
��:���B�.����u���,��~ u��!G\E n�Nh�(M��������M�a �3��w���?���n0���n'�v�OE����2��ON}��I8*#{B!�_x��B�
��"�X������*c*.FZ
��$��$�������YK��,�V������f�(������a�H�'}2Y�r�r@����K��EIe ��*�DM�����UjBk�����#w�_��h ���!���f�3�Om�&w���(�$�L���.�������;W~K�9�g�F�"�B!��:e ����S�L���d�}���O��c)B���R [|��|k�*����3��.��a�i5Y�����uaYK� �����3]� ��460n�b9�)�;��$x�eW}������ 7��D8'��eS�W��ddQ-�m8!@�>���Q���M��H %{w��E��&�v��nX������#���V?�E~ �m~��L ��s9Gu���m~E�� ��?{��!7�J �&gZ�k���W���o� ���B!��Z�e�����_~������2�&Mz���/��B�RT|���E~�G��t������6���8L���a�A0�Z��F�>S��4%��0� )9� �ir ��[ �e,�%9���kf<�?��������J�������������U��������R*p<����)��I����/12%�_��f��N%Q��/�T���3�e��c�v��=���5���jb����]3jF�"�B!��rn�N���;� 0��{���s' 0 3c��O>����0�EfVB(tu��x�5M@JA�0C�[�����>�t;c���
p�|�^>�������[�d�r��J,�Y��d��i�0n�i���� 6�7u�\~������`�wl�����rU[���&Q�O��t�� ���
�� �
j�L#� zCA�Fiq
���f�F�F� B!���� ����1� � !�X�=��{������A�}��������4@@�;��q���p���7 �1 <2&H"L� �d;\�SB��Ll�F�ic\B���H��s��^|�/��Td�wg=�������M��������/�,DA�T�U�w��l� l�c��K�`�@� H(��v9���u���6O���k�B!����$�J� ��!� �W��\�g ����QDT�����2�X���.#c�$�t���X�Q��(AEF��N�y���v������U�����H�l��L3H_gFC�T]��%�T����\M�J����R�p� ` no���t � |�!�BA�E�����~�z��._!�Bh
u z��o(����VX��~�;H�p��1Aa�e�+>_��~�a����� �~��/if������c��!���/S/Mt ��h�e��� `B��@��edI{��)���T� 3G<E�/ }�B!�B����_~y������� ��r���s�����m�8kC]�M��C��
����k�g�(h�n��7��S2�p{�F��4 ���2=?�1�,Ap�����
2��N���2� �=���&�g�%�l�R���[��){ {Q�O����?�%��Y�3}p�k���T�i���Y~�������k��'c�����Kbd}G B!����)3t����W?���?����)S� =�c���;cbB���1��]�)�}�{rCq��#�k[� �N�r\|���������HT)�#!G�X��u�i�6�t�i��b��������p���[�+`H}����������r|�o]6%����4����
�6������b2�)�`��[��������Y��_�x���k�rK)�I�����F!�BW�H �["3�]y*��k}�+��5�[C�/�CMVA�����9p�r=����R ����n�� �����a���c��M��\��H���u�lP�������\��� ������? L�����P�!��[�4:���DI;�"1����~w������8����2x� vq�r�������>u�����M�� �Z�B�� � �������i2w�Q�~{dX��LrZb��Bp{�FCm��c��� � l
2� !�B]z������c������WV�E��+��<}���w�O��$G�������H� l���z/A�,���@t[��"��! ����� 0�#�����!��S�B�����x�����e���u���m����_���\t����-n�^|���P'K��'
�jo:U�p��&l=��������� �T�6gZ^F�+���;Q�'�� ~�,Y�c��������!�rB!���� ��^S{{���G�c�_z�����������{EW����w�6.H)�81�2-���8�Y;�E��x��Q �w�R$D�| �%���c.��OEZN ���}��t�`���m����� I���5{)7��eS������QN�g
5%n��_R�l5�����!���2����m, ���{��O [��U��
��Q��1���^zi������������t�rB!��e� ����9p�����{��� y�Fc�W"�z���� N�i>��_���0���a4�G�P�k���8���1���l�p������ ���& M�Tp���jk�=����!�%�����`06-����\E �&��B������O0�zY���fx��%uGV�X���L L{����`& E$�����fwq�O@���K7o��o�>�"�fl�O�B!�PaO_��cGll,��R� t>B��L �����zWE�u}�;�3��g v$�+�s:e����������Hc-��<� _���^��� ��;��+Vr������3e�+���?1��l LcU. ��eg���<�2#���,:��X��N$��0 0~c���
�����2���Hl�K�*�������hy�[�/%?���6��d���*hB��N`���VPYJ`w<��� }��������S#�B!Dx�;&L��w��[M_� P��! ��6���r9l�0 �nHs�rd��@�iI�������?�X�����H9����I��� P���_��L�<��)k�/l�������\�������o���P� �_�+�- m�����������r������H
����`w<��/Y�~}YY����B!�� ���=�5M�<���D�R�Z,���Y&2�^B�%�4 ��!����!��'<����e����2F8 �;�o 0� ��/ 26�����B��R�����������������+kv?S�����_��l�L�u��3� P[��Wv����dv������I1l�O��n�����?_H\�fMYY����{�B!tyD�����/�X�b�������h4��G y������BA�5�o�����@�K��.h�=/��K�]*��d<ZJQ�i:-<bn���������dzM\���,�� �Y��2� �[��q�`s| ��\T MY������\ �M_����%���9�����G-�1��� ��%�\E�������x�n������O��H��\����� ���� 8��B!�[B*��|����^�p�����j�/��F8�!��~�|��J��c�+�W���A_U ������D�"�y���*G�.?5 �]W���Z.�� �C��$��:K%� �����eE7 �U�l������s9$��ec���i�f���,���������?�U���� �h�d ����f��% ��B� ����Z�%� �d�W�#�B!�zWHE�A����;��<�@��!�"��dC�_���E�/_��������!����F;I �d��P�U�a���2}���,��24��y
�C�����s()q; ?9�lO�DM
� ��.��';W��9��!��j�\��#��`��WU�6���*}����O#L�L����F�����Z��O � �d���Wz���>-UX\�('��WV�cCrr��7!�B�n i$�����<�q��^Z BE��+��OLX9.~�@���eu1�O�������wsT���P$AE��9jt����"�'�����������Zks�(�g�������A[�I��H�� 0���>m�Z��?�.C�v�[g�{/w�DM]� � }mi�/���-)*U6n��_� 0�>�����8�x%?�Z *K�m�Kc���������)�%�k����v��D�UgG �D�� ���e ~����� � B!�B��97}Kdf!�.�����t���%:#a���A�&�ty��!��S��q���n�@W#�h��G�!.C0)3?�=��m��E��SC!R�6g W$��Ck�'=+)zu�.�Cpg)E9|�_�Z��N�%v�� � .��� �Hr���������/F!�B�$2?���+����6l�����B(�p�x���]�':}�����.�L�q 6��\�f�$�Y��� �&��q;�G���_ 3?����y�YK]�oMg&�V���]H����=��xK0rZ"nziO0�k������ 0�I�
�" �j���/�"�B!�.���/q���'8"��������zB($�g� �Z���������~aE�u�w���t3l"���0.��C�
����� w���>��V�]7�' ��or�� M��������lJ�8���C��-��z����p�� N�[kh�� L����s7�}_�+M�����RW�+M;�A�+M��i��g����kL�%�/ k�wxK0��'���
5 r�?�<x�:�s� �d� �dr�i 4E�jbc��@��BA �7FMIOO�� B!�
.��� &�����a����_�#�#�
�+[E�u�ot�����D��=}��������~����V����
���T�����~� ���P��#N�&}��&v�tJ�� x[^��lJ��MI�e����&���������;k�Rn�����k�� K]��j>�3������&sG�*z�-����s/��Uh�� >��?��j�;?Vz�rZ�������OQ� ��K B@���2YW ���1�B!��"D����n_"��l���C���7j7�� 7���":����oz����F���o��������g���G�����_0�����n)qJ1;>I0��?RJ�����_�� �`l+�,% �edq���:~ ���c�Wms9g��d�� �DE�R"��A ��?�/XM zsy�_�_w�uA�!�B���iQ��v755��RB(D=�_�V"L������������m�FN��>E�^Y�8��`y��� �ch���B��/~��7|�����T�6�X[����kn����� ���&�������,M�����rA�&}f��;p���>�/����"���a97s�����'�,Z����>�� B!��B-� 9r�f����WWWs�n����������6���'���C��|7u��C����&] ��n)�\��Z�2#�������&o��ir .l��;�-���[7"1%M����O��E n��4M��\��s���6��`dQ�s�\6��
�f�LQ��4\�|���fK6���A*?�w����-�����_ ��!�B��P�2����X,.\�g�B� >u�����>���.]������BO
+c��<��aN_
�x�
����} 7h���&��0��������Cw���xl�S�U��F?;��/i ��k���D���Z�)�m����;���� `v��2}���,��c!������a %�y� ���z���XsL���mY�U��`�E#����# �(-�LePKe��&�*--}��g����
7t}O�B!�,]�����r�"��rnrssw��u�V�EfVBW� ����+ �(;z����0�/�p�l����kg�Z�JZ�=�+��l��� ;t�?|23�� `�����7 }2`�O��}R��'Av�$j�f�Rn=�%��|��/��5}f����#�+7s��6��F�Z�����YRwd���m.�O#��� ���/��g�7{���-[�,\�p��}7�tS��B!���"��{xk���� |�V�E��+B���r>T���o;\��2Ij� = ��O;�f�5~��`j�I7�=������<������w�i!>?� g�� 6E��o�%>=5n���7�?�1�� ��9t������cM�-W(R��)C {k����|}��q�������9-yo����#��u�����={��r�-��!�B�E���=��_�"c4].�V�uB]���5v�B�/���6�iG���N�0mW��<�OE�D_U�^�n>��,�����Z�R���SE�-��)��pKK���Z���:' ��K��P�DM�[����Y�7��x���Z�|��I��:o��O��_\� @���{��y��'v����B!t��=}������_����(���m��%�G���2����2 �b���� 9����]���m�q�0�AqAG_1;e�{5�.I���Tg ��)s���0:�����t|�o�fq��]6�[�W9u;����y�bI�> ��\�pc����c�2M���cs9��N�S���;g��'7��^����'� � U�xB!�BCxE����?����6m�������Y�f-]�4+ ��!��B�5v��"��X����-���i~�3! ��� F���
:���^^�~��o��|��o+��[�� /d���4�iO ocu.���&
WO�:���Y�6� Z� ��$ �Z���-1>��&�/�����W; N�8"v����}�f�����+_;Wcr�=�5n��?~�8!�B����TM�<y��u��� ����l���"��o��mi���P�U�5i�MQ������W,H�a�Z��N�>� � ��3�H5$�� ��t����el ��yo�;����K�����l Z�?�b�t��
�# ���/L���/)j�v����K�
�RFT��i/�������Y��o�o��o6���~'�= �S!�B�� ������l�� $&&�����%B]���X����� ��(9%�H���w���LE�u�w���t3:�k�w�CM�^Y�`� J��Y�L� ����i��g�"��47�_h����Mb���Y^���� SVp�v�7�edQ�D �7K{�Yp�-�D���eI��y�Z*�\:�O���'H���v�(�� rZ����>x.m�+�u5 ��g���5k��m���/e��m�A!�B�w��)�B(��&�����Q����}�.�u�m~6 �����T�d1nK��y.����k�k������0�O+
�� ����G�����\A"Lf��z�yn����
�f����uI �����k-yY �9� 9������Umd��~�� 0*)� � #Ut8m:� ��/�����=��O?�4;;D� � &�#�B��.�����^�����u�]��/��b�����*��tzqQXu�P���G��+Q�L%����3.�L�q�\ ��Y �ph���)�6V� ? 2�c��8����t�{��L ��$�n�r���o��#=�d�HVE�!���Y� � �� x77q�� 0��U���~�����}�y����� )1���%n��E`Y�g�( IDAT���!�B� o�����W�X�a��^���?���?�����-�H�C���w��Ug�E��
���#"Uk�/��&��Hchy���k@����Y#��J��A�I��B ��3�cV������_��.*^�yh��[�������=�m��mf�t� RT�����������Y����+)��IT���;������z|X��8B��a�g���E!�B!��e���v�����>o��y��������+**�"-!�.6��-��[Xu��C�
�[ P�d�P?1.��/�H4�\��x[���N���;p�\ �$*Cp�� 5�� S�g����������!����g�}m�[�g=7���t �9���� ��^xd? ����QKe 2s<G|�fX1R���"�� ��
���3����R5�R�N��~x{����Cy-!�B�P��22�����~���/�jB���=D���C���Y�$+��S�*�7�E����� �2=_|��mM�J
pK]��?�\� � ���)�� �� P�k�>w���\@
�p� F%�-hr�<[����l�.�,�F�p��r��a�8�.A�o��"~���T�r#��R��}I�v'�B!�.��:eB�
��{��$+�� ���� ]�;P�r\|�������z*2%�����q;��}U����n�G���&�*�����-n�����~1�e�� ��# C1.���)�w� -�dN|�pa��[�d�i���*��$*U�,���tnJ�����������/�7v �)�8m�+��s%��Q~�y�2���� ��(!$U�f�4���B!�"G$���["s�9B(D������7uQI�E��a��!��S�s$QC 8�&�����#u�4{y���]�
G6h����e��Os���O���K����)���.�l�����7yv��������z�N� j�lm�4B����\�K j�l�����KMM��I�������@q]���&��"�B��.2?��Hl��U-� �K�??��& d�69����Wm���� ���`/+8��C�mk����+�F/q� �4�q~;_�OD:-Q��jiT�-�J�7� ,�[b�rt�Qb �������p^�/�@�\�f�dqE���#������c+2%uG
*=[��Da�@^�eB!��e�E���n�@EO�`Cw�j��^ `�u ��4s�, �4�6��b<�$PE b�)x���_��s�� 3H_cF�b !V:�6+;�j G<��t���)����G�N�4i��5��w {dI�^~�
�(�E�B!ta� B�H��j���� c�������Cp$f�6]i��
2]iZ� ��z(����m�u;e>Q����>���
�ov�RW�w1�e����!�� �w����'�^��7��
wP< [|!B!�B�v� �P��mxB*M���P>� QN����t ^��z�^\6��� ����D�"���v�d� �H��h 6'8���IL|��$ h������,� (E?�5�
��G�-�kS�Nr��T&�Z����3�x1�e�n ���������\^L�Da.���U�&O��?-Es�pA|!B!�B� v� ��|�g� �Z������� ��%t� �}�F#m�eh ��� ��|J�?�q��G ��ddQ�D��I~�+i�w�4�0� Z�!�v�������boc�fx���W�� �.d�l������=��0`�;�����w��]f�%3n�q�����O������Q���K����U����!Q�E� HEB�-4�� ;����i0�9��3v���1�� �P4b�Z�m� 4 !�B�K;eBW8�y��t��(^I�3�TaB ���n �s5<�v���VW����JO��xoQ����<-0 7}I���� �iC� D1�����'���� �+��X=��r��Jv���t��y�����,d�eB�����V� �6��{G�^���=e�A��ah��V�.u�������Q*JbZ.c ��'�FNK���V��0`s9���n������1nq^L^FC�?} �c#�B��+'B�-�9U !�y�@K�o����~��� ������������ �6C���L �}:T���05�n#��B���}���>���'|vY��
�Y��_ � 1��<�X���55nm��1�b���v��f �iU��-����Y6����3|B[LR��q7 |�S�sjMG�9Y�eGS+6<os9=���� ���)��69���M|rZb��"��B!��U$2?�c� B�
���p;k�=T{dQR��&n�DM�\��~�^��^���!R�f��P�)��eSr�X�Fa007$���X)9�n��mE�ykk��UO���� ���������V|j�:�
f��3x���P���B �:���Y�6�x��cd�f��SX Pg�f � V���A!�B�3eBW�x�f������^��e�i���*��$*U6n�����:�2#O��F�I%�T�I���Q�� ��� ��]&���$
r���$Z.�����;��"UBE;T MY/�wD ���z�I@��8O��V��Y��FF� �4����qv1;�����a���v��-��
Wga� ��� ��A!�B}v� ��p�����f U9�bw��e��d���}��������u��o����<-0L���� . P���������% ���U�r��Da�M������hIMM���.��}h \��c�*QY�W? h�3�>-4[ � /a��& ����B!�W`� B�
7&Y1���-�H���ws�/�Y�
A�q^� �����t�x����j�z���k�#Q�~X���n���K�����r"-'��$E]#��>\n�S�7h-_g�@lK���_j�L+���� o����/��guy2B!�B� v� ��|c*����]���K,H3N�%���3l�����&�g������3S�X6E�@�4�6�,J2-p2`��C.�-����Go�N��>�dN%�; j�LC����w�T�r#��RYVT�����F���B!�� ��A! �������J��m��J�,u�����%������~��)��\6�����,�xZ�llt��9w� 6��1����� ���a����TM���S5�kFM�����R��|~�K^F��Q����|�����5����nYB!�B}K$N��["s�B�(���W�n���u��}�|��&m�ZeF wBmq{����&�:Y��<1H�j�1�����?}�������/��� 2�����"�@��9J��G-_6}��d�.����E��u6w2�)2H��������x� w������'�{�B!���*2?�G������{E��T4Y��V ��b���#B�V�-C� ��G,�����G�%�����_ p�q�zg��[��
�����vLMJN����DM �-�N��7 �m���o���Q��A ����c���Z*[�3-/�S�ill 7n�3�<��SO��B!�Bb���=���D��+B���-�&����j��;�/�z�RWl�Y�27��dM�rv������O�]1�0[��8�� @��? ���'��v��l�����@+(�q H�[�������u5����d�`sa ��-���������~���wb�� p���;��c���>���SB!��'2?�c� B(�O��:Y�J�UNc�v8�
����M Q��<Q�G��A3�^�q��Rrvy v3� ������� ����2f��QB� h: �9%��P@
*K�m����VPY��T
w�����*2 �hn���������7+2!�B�����B��+�s��2�1�6B�7+]��K��K�� ` &��BeF�fx����*����A�:3z��R_��0Q4��� -�s=�j� ��W/4��]` ��m�^�ih�r��DM�� |�S�sJM{�9�Uu������������� )���]bK0���L `r� �����bk7
�� /y�J���|��w>��#�?�|��7�B!�"
v� ���>T�� �m����xM���tf������������d�����_(?ce�/?c]X���G����;�CN�i�s# (3������h @52�g�����gI%��C�-�My��V3H�~Q[���73 �Td<��9k�Re��D����H(�:���Y�- F�nI0~4����: ����� �������!|���v�v�"����0d��w������ /t�!�B!�G`� B(�1�
� �
d��=}��&.� �.��F��wCO����;a� ���If+8����$KT)N�I�#�A�cn{���t�����
�����:��F.K6����_�;�"�4��e vN��������k��[���/���y����� Ri��s�4!o�8~��O��9��_���!�B�!X�Aua�@����Cv��"V�.f� ��l>a�yQF3�H_��?�Z3|�u^FC|�teG6 *8vCJJ<S�gj�b�Y������ ��B��� �{Y�e � g��o����_�= xS�c^U5�u!��&�9���!�"��I $�z��?�{������� �B!�W��%�
C�(R�
�sTgv�0&���=�A�+M��s��y��5M*�� ��������!���� n�J/zv�[�g=;����o����wW���+���0�>33?����yWr��� KK���.��q��U#N2�n�kbc~��@���14��fb�p��_����I f��HQ���&@��P6��t���^z)�� !�B�>�2��\�l��H �sW�������� � h�K���0��C���p�� �� ��RS������^|��W�����u�~�����]l��iF���������/�:���V��������lJ��I���������
��h�� �M2~�/�z��%��l�[�� �`T�
�&�������J
���zkFj��"�_L&��B!�BW&����ZE��bd���K6�H �3v�bh�t��t�00}�z]����$[����s�;���?G����o���& ��TcU���v&\J�- �AK�Z'Y��-p���]^Bft\asf� ��j�?icu.����q�m �&�V��m ���46��E=�a�
U���8�O��r��*�0�x�!-����7GN�����?i��?y�J���B!�P��E�P��������[.q�H �3v����$� �OEw/&����WdX��%�9��w������VdX������
0 ������?14'E5P <[[�g\������p����\ ����E7�,��-h��_|�d a;�ir�}sE��_�F�bZU� ��
���� ����� ���.�e����������,��XP�����|�T�,kB�oB!�B}eB]�a6�� 5�@M1��I6 ]� b��F,��a]��z� ������T� �AD?��"��A������;��"�����/mx����[������L �����/:g�9 �E�K����iU�_|�]_rO�G��`,��G���s�����g�-��Vn���������($l� W�A!�B�
�E�P�����
�o6�%�C$ 5���bo���O�5�]�&Z���Z�4��{�l��aD|�2�����:�g�A�!A.�<��`+2,Zj6 -4����`%��FI��G �4���fU��Qs` d�� .<o� �n ^�h%��In�\lP�k��qf �E ��b �m �`l+�,% �ex�2 ��s�}99o�5��!�B!��� �?�Q ��{��|����6�7E��mE�h���=g�MQ��]��-��Q������,�. ��_:��0� ����&��m�3�Z5$�����O�
][�C��*j����!ue3���Y4�q�OvS�-: �L>��
�4�w �p>��s��G��\6�j��������l���&sJ�N����!�������M&��I��
�z�jB�e'#�B!�]��� GZ ��6&Y1���-�H���w��"�]���'"��'[��Agv9� �2s��31�����QwVp ` &��q9T� ��2��ED�� &R���+�V�u0�����2���
n���cc��(�q�H{�wI�)���!s �*�=�� '
�6�R\Wc2�rss���Z�� �B��eB!;P�r\|�������v�\�=D���� v���&k�b�%#x�b�g�g8�k�{�q ���3e�y��5M*��M�6g
��
���6k�v��@|�>��~�|V����[p1��uf4�� I4w|dQ�D��e�3�)4rZ� WK):U���- v S�����5t��5k�`E!�B]=0S!�;B���Hy�k���p�����s���v�$���i�� �0��C�$%x�����\��,�I�U��@��".��H������)��������-C�����:ALN��� ��M�Q�c ix���9��O���f��� @���7�;����^|���h�7���#%�����r������k.����-�~�������[�Y��_�"�B!��*X�A��@��. q�p��A��v��3�
?
67�v����Q�k��a������)�� �s� ;�Z�M�d-W�� r�@��Ml,=B��q ;\�a \���� ��}��Sj�8sL������qL��W�U����� �[i�I� �t� j�l����II�P\W��?� �r�#_n�'W?�>��S_��k��(
�7B!������w��Q�� �����^���A��hH P A���@�* B= �j�� *R�r�����r�
5DJ�W �r(,��@B0,!� ��������������
~���x��>��L�j|���<�o$���]"�VB��o��R(Ue�.��������UezU���o{0[��>+|�j��I�wJq���oM��w�x��9CK� 5� ��m-Dg��t^���Y�T ��:�d��Ry��i���%�H�\]�h�����|�`�I�-��1 �GJ����> � W�q��C�C�� �B!t�"��{$�S���W�� |b��% O��
-Z���|����tq
��;��^����l�Z�n����f��������������\���������H��w�"����QS�� ]���h��m����G�=�������P}�����mu�}����Xm�'hXxN����o�8��AN�B!��|��� �/!��Mr������( sQ� ��g�t�et�r,��M��;Q����nC_���# +�Kd�����h�O�;=��)�/+[j���. S�j����.�|��2��"?��LN����
�� D�
�`��59���r"#��x�2!�B����B�����V5���Y2?�t���u �\�2: QW�Q6��LhM��i�DI wS_m��a��.z��c[� ���k�@�� b��O�h�������O*�����*P
��������>9������.�jZe�B!��5 C�P��R���
��:��c����.� M�B# � K��v�]ST�G� �3c��HhSm �l�8+�������n�G� �c���X������T- L�����\����q���`�$���ob�E� !���b(���_��3w�(�Xh��L�B!��&E����%2��!t
����2�W���/���c�~j� ���j�3n, h�:D����R��k
�z����~� ���l�X(w���3���4f��>��.����8��w�� y�Ko�6��-�����5�M�l�uYX=�'�>���naM��w���-��X���~�i�e<�G�S gD�����8����� ���5e�z9!�B!ty"��;��AE���-����rl9�TvY��`������?@����Fv��0��5� ����V�p�R� p��������� �9��k5�d�pSr�]���� 7���%A8OK��YK�R#C�YSbG8��c�^;��� �i�o��Y�{��=N� �:Z �p���K `������9�/��@�'2 ����XA� �A�8U" 9�k3g%�b8�M6������ B!���a�_�P��3����_F��tQ��#����.{�RaM�� �����������$�� �w����8��)cE �+&�6�<���sN�_���-S�� I�w�8� ���V<���mbUa+ T,=����n���i�� 0�x������� IDATh�C��������2S�M���+��fw�<8!���k��dL;87%�`#�B���H\���D�
(�������Cz�,�I]�E�{}5vu�� �D������6�����_)w!u����n/�t����p�}�����>�������O��OM�+���;zl_ x
�o/����� ��k^4<��.( �~-�� aD"���@h�^�7�� ���[ �M�M�z�sC!�B�D���H���%2�E�K�N�>QU�� 0b
��JgT��J?�����\�Z��)�����Y��5�� =��^��Z����I���W���� �^C�����������D\���&���~�C\����_[o�c��'<�h����N���X�g���iv�F!�BWUd~y��K������p!���(
=��3q���xEU����/�� zJ@J�� q�E���)W��ugM�K =��&��p'�1 ���I�:�(�:w��?�cT������bN����q5n����_X#��f����6 ���*-�|_Nd ���� ��� B!���P!tu���+U�:��r/�L�T�v�.{1N�1���� 8�J��Yj~K{wQ�����I��5d��=�� ��ke@#1��u7�7��Zy�����]�������,|�@@�i���_A7WW��:m[.���,�B!�B��C���u��(�V���P�z��%�S����o��Kz�Dl/�������O�������5�M,�v5�����9�V����_�p��S\��F���_F(�Rq!�! ��4<�-���DFY#FY)&X��=R��<�! !D� #5T
#��A!�B��[b_��������P�� i�t��(� ������7����k����&'2Vp����^�@�3� (ZG�:B��e�� 0TzM(�wt��5�Zs h�~`���Ez���J� �_U���M�����DC����������$��p�\�#'����_���n���������j�#'��V #��A!�B��Z�|��������p������ ����&J�_�E"�|iv "����_����5�������L+ )��y�����m���$��Y�� �������(Z��5�Zc]�-�{�g ,O� �:|����g�����F��� ��mf����)����2B���C� O x�����,�hi�����&\9��� J@�}IZA�B!����|�ry�Dd�2�X|�w���E������/�j��g����a+��k�l� � �p�e�:Cj�%_ ���!L]�!�f����D�3g��E��!� �x���.� G� �����b5>�H �$�3]�R|C���F�{����&RQP�e �q���a| (ouY��a[Juq]�B!����� W� ���+^�Ev9qO�+�V�P���Y��\ro)�1I� ��7��6+C��D^��p�@t@�� �k�r�S�k�u���#� -���������~��} ���z ����K"C} 0�0,$�W�����P��E|B!�� ��A�V�V��#�F��� �v��'T�+���G\u����NO�nQ@��=V>{L�[/�}"stE���/�uX�5E�������~��(��-V8WcB�����) �����f�~�c;�T������>�)3���
L\��lS�� �� �-Q`��%�"�:�{5cr����<� ���`_�B!�:�+eB��e6�m��Yy��U-M0��}M��_ro)Cj��\�o���� ��t]�T ?�oH��2T�t��w�v>��r6m4S�;�v]�T 0|���1�<�}�n��b�h �
v��O���2m����kNl�c�.�T��/v����� ���u[JKK���<{d�r��E!�B�3� �z��
DT{�z^��4��V�PNF|���/���� ��
�,&���W��$�!%����'�Q����Q���������V��o�1�a�!��j*_��k
z���k���[�s�������� �h��-Z0��� �>��I��� �. #��u��d���{��_��Wll���c_�B!�zC�Po���.��������lV�4Y���1�������2�$�������J� �T�!z����X�����' ���<�D. �)�����kJ���l����" �����x���'I���<���N�;�L* #wV��" ,)��'��ai�!��.�9���-~*�����(�M�KJd 7%C�~�B!���2E�o�EQ���G��f�a�:���8�E;��R�g]�TO���i�~1�{Q�Y�W!LI�.�tLA�} �R� ��J�O�5�G��x2����-�U�U��y��V�,���v>��-)�#
k*� �]�h��� �:^��^' �9��� ����� ��}��� �B!�zC�"�%7���.��'$�)��,u� �u��~T��R��U�����U5 ����I�r,@��S�O�����6���t�[ � �|
�C�|e��[��"�^���4'k�� PXS�l������H*�>�g�J�������A�|�@�8n���'v5B!�B( e��Y��7�"���f�]�u�����6Ae(�'4����� - ���}������h�Y�h�������|��� ��[}���� �G��^��6�"u�>������[�� {���D�;�&�-�9�WV�^���D�BX�l�t�eS��r���o�d
�B!�PW0�A("\r����ydy���# � ��2O�V�!US��
t ��>x����X�T.��* L8�4����^���r��q�n~���� �o�x�>}l��R�l�r:��W`��!]5[h �e���
����'��nT%-���r�)hV�J�����B!������ @\H��.j���3���5��n|j_��3�.FF��}�KPj�l8bot ��yE 0�
C����7��.�U�lw�|?���z�#'���'#��������V�p�R��^��2~���LNcJ��_kH�s O7��Y}��
���'i��a��� ������-��H��� p �l��1���r��M���A��B���� ���8��O��k}���k}wl��H8� �x�B!�����2E���Y{m���G�4���/B��l���jX��I����x�*�����<�j�}�OZS���������xk������\u p�V v�u�9���%��"[���?�!T�1������D�/��f||�K/��5��l�wG��S:O�=� ��`�`�k�B!�.�2E���Y{m���G�4=�!�;�e������ [�&4���.�T�#�~jOs����qw���-
���>������BZ�RXS9d���hPrT����[v,,/�����0 ���I�68>V���vM���z�N��������- ���&cl�B!�.�2E�����f
����/M��}e�+Y�d�IB��o�0wM����K��������wT����Rst�����W��)����7�E8e��u40�����X)%�/u-���J����u������ �}��(����@@���������G��u���W(y������ ��m��ZGK^Y1����� B!�B�P��+�!B�g��i�|�������n���W�K�J�;�jzr� �z��-��]����7���Z��7�� SSd+_4f�:Cj 4� �U+3�~ k8N]��������_f6�R^S��-(����f��EP��8����������X ���6���4����?'C�r�Z��gZt|�b������[�X`��&t)
�@;��4��G�!�B�H�_�#��z�����k[���KVj�� ������n�t��\��)��#���Ec ?�8 �o���.J�*�^^��`� �DnY���i�rX|CT�������� ���U�,�s@#L�ck�^y5� ���� 04�#�9�\]���� >/Q�E
:V������ �B!�
��/��} �����iek.*CQ
~#[���
VJ�+�Z�t��.z���i)�����3��"��tF����������
�Z����wK� u�����?��o��� � �EP����R�*��C!x�����
4�%x���5��n{�O��ng@!�B)a(�P�t���DT����P� ,�5���g�������/�L�)����/���n3�g�eb.Z�N����|@�� D��_|_~�4~��������4��u��hIV�n6�)��_�)�����<z �K ��C��8����{rR3~t��?|U�������A{�8�Y�9�P[!{c#�B!t �o�B���g<O�k��������g< R7l��oF�e;�0����p��Jd$�9}��AJd$���P5� ��` �FB�����O~�4nW��c��a��xk�i��U��:��� ��J�)c
���|p�[Lv���Q�{A�� �?���a�@ ����>��8��#��9���srS2rR3�f�J6�p ����<���������76B!�B W� �]viIhC�����o�E-��b�*�Q����qZ]&�q��o����m�#�v_>FI������v��7X]G�l������'1c�����w{F������6�}QZ�(5,��y.�����M�l�u��-�I4Z
k*� �m���4���d�.O ����)�������&��KnJv�F!�B���P����KK~{[��.[�s=�l�����F����=�p�I���T����Nm���
0��
:S�9*��.
���jL�(%#�dl�jL �tr�K�o~f������� O���=e^] t �i''�+k����xw^�pKE
P���C����m����o�B!�����K}�u��d�@��q�������v�a� ����>�]g(] �� @���� =�I�X<�]S:���p^Yq���/
R}���� 0������+����T�Y����h�<��6� �-�E2h����OU t7�W
s���4������5�`{"s!bD"�9�}�+e��.������Z�?��{�B!��" e�����1aMH�?~�9��jod �����] V�5�� O��-�k�����J�������U��L.��� ��c�^;���e8bN�f� �2�q���S2������� ����<}p�k���qW �e~��b�@�� ���'����������U�m�����A[_�n\4h���5��Ai�R�� ���VW��C��v��� ]�$r/'�B!����m�{��lu�P� �zT�c��U�����~�=��D ���c���1������G��{���?:V���2t~� 6�
K TT-�q4\�g�N�����`����G�r�`FG�Q��| hP��������Xk� O+�9��l�2<�����}`�?wY����a�������y����3�j��H��\�B!�z�����K���������1�.� v��V�sw����h�^5&��S I2F���W I4ZB'w��������4���� �2|����;�����O���
8;��Ku��F/����,�gQ-��F�3��x�C��t�t$2�����wG{_e/'�B!����� B�#��@a�_r(#���2�'� i�D���������i_��'tN�g�:Kd��F��8�r�~"�8���@�����D��tp|�{�*[��@s�x�� ����9 8���~����s]��f�[����"��cKt�}�~��������3_�{9!�B!���)��h�����2��/�HGi^?���~�jpNj���Y���>�� Z)�< r�����-��R��F�� �:���of� (% �$ > w��yeM���"KE"6�;��W������z�$ct�'m�s��2Z���S��
����do�q d[� ����?��!�B!t��J�PD��~�='o��3��!w��%���������SX=��8�)R�.h�I��0"IN���Rs���� V�"���r�#�1��-��7�2���W�9����C�?|�O��[��1m���;�� ���<���q���`�C�8���r�����")5BGY_#�,�_��/��!�B!���H�s��Df� ���P��b ��������?��J�/����zs���@�w0 ^�gEi ��mj�6D�� !�zS�
CJ. T�w�`�,����&<�,��s�u�g��f @aM��e���^�@���������X_O�_2�%��qYE8t�������� ��!�B!�z�������B���a�T����%D*>`�t}�� R�]��N ���/��D�s5&Tn�?���O�T�L#
�71Q������ ��FKh� �Wj�
�#@G"3
��t�M_�?�m]M�N�c�� !���:� ��k52��C!�B!���H �z�� ���
������uq�3 4�vH� b����v�d�O���7��#?� W �*�U�[{��U��f��r7�����{�xN�>s�#���s����"|���?����U��<)K���K��I k��&��7������j�a�/��w0�@[;t� ��t��!�B�"��� k� ��)�Px�^�C�/R ��6���B�k� ���6�6%q�AI�� j����/�F�c[���� >4�q�k�����_������ ��@�!IF��Q����|���m���pl���H�Jv��$�j�����@��������^�4�� ��������m;O��ox�NLdB!���"1(�]"3lC�;+t�K?��]�+e � Mq�s`�����k/��~�i�!�����Tm��[h
8���hx&{}B� KM������Z�t\������/����[:���*?+HE_t�_�~���QXS�l���N�d�4��Wd�bm ������@�� +;^|�6�� ����9]�XB!��fD���H���%2�}g]Ba���D�K� �*�g����xgr�.��*l�Xz�i�k�Y �k�.����q_+���F�P ?�Sv�P�2" JR������_Y� ���U[�hHW� 0>`��M��G���H�9��^1jr�-�B!��6D��w,����\la��&��U��3����i���N���1��&x�O��>���j!hG�B��MG��� �Jd ��c����J����p�^�w�Q� �:j)`�>rN#����:Z~�usue�� �B!���)���L ��I��q|��qu+3@�������T�l�����w�YK�r !"���F�����.&lp;����� ��Y�R��F��\�:�XB�J (Vh� ��,������N!�B!���P!�����Lm����%���0Q�����������%��l�
��k
�K�U|Q������ �L��]Z>xU %r" �m�( ���a�m�AH �Ho��D ���5�_Fh_>C8�W������R��dVcu��au�w��g�
�%f��K��"�B!�.W$n��]"s[BA�QC��!���d�Mf��R���NQ�Fi��d�w%�;�L+ )�a���)��/����F�*�����*�{��o]���v������{wd
�<z�^���:�/���C�|
K���Z�wz���Jws�sU�l ���[YQQq�}������|q��o�r@|�����]�B!�B������+eB��R�g��������#����R(���3l" ��1����$q|�LNd ������Q��c���z0k�`�,V��/�a6����� ��)�N;B
��9�%���#I�+O��+�M4Z*++�M���[oM�:u�z��{>�����W3�lt���B!��J��/B���E� M�����n��1�a]8O���������w=XM���� ���������UU�M4ZT
FN�9��V�j�]�[*��U���
��� +wKV\����FS���g ���B!�BW ��A]��N[�&Jq�������Z �Q]���Oz�pD8����.V�Q�@~K�w?=>^_W>�� l[����:�
$
.��s�W2�Q����A ,=���j���m>w� �i=ye��BNN�":��KopZ�|7nE��9�d!�B!����-U�KdnKC=�ASgi��% ojo�}��9������Yh�� ��/>��: �|U%f���m �k�,�C��� IDAT������� is,U��&o<a!7�&�����^��t8���U��99t��:ju�%�V�����U���6/��:�G�� ����{@!�B���_�#��z����"�X=�|9 ��SU�%wMQs�e����� ���5��a�j���-��Rs.|�����%7Z��m�j5
5���Aa����Q���f���K��u?�[����##>~�j���HE{8B!�B����� ����D��!]���5� �U+�h���g W
�nLa�I������� ;g��V�<�~��)G�:�'�3<�����}C���a����n��>}�2��}��Ww�y��b�Y�U O5�����'w�B!���'2��cM�PD�I����������&�#vu���"����^��3��� � ���!5���E�/����4�=�����) �n�o���r$a�S�����
�Jh:
f�^ �K�x� 8 c/�-��������g����0GJdV�^
c�.,/v�;�6������MO!�B�o�2���lk��6t�tV�E%l��p�7O)�9\ ����Mnkh��Vf���v����j�~_u���'3~X������,@����`;�����}oZ�W����� �|��[*R�� ��������?;��w�Z�*'' (�g+���u ��q>������?����F!�B]m�!�Tm��D5�]Sd+� 8j��8jm�y���!c
��M �7��������|���Sj���r�j�� � z��xP�FM����Z��U�_|���/�p=��5�����n�6r�@Ua��AU�j ����Z�1���
�*%7��R �6�-�Zc]�s[��~�������Znn{&����� \X D)��������w�B!����J�P�.g��mk �8>[&�j �w:>��K����F�w@,��A� pH?s�q'X ��j�����n ��� D�PB�&M�����-�� 8� D� �������G��:���5���c��������bjVm� v�W3��� ��7�kS~�lTM�����<RgoU��� ������B!���a(��F�]
qV��ZAd�i�WPQUj�����5��3���-��N7�� �/ F�E����=<��$�rn��?���x��@�ka�� �+����� >���� e��~:����
�q�KM��M
T� 4�Qm���b���5n� ��H ��j���V,��)1�|Cl��`�K�� �B!���} !����!�Jf�9�_)�����1~]��H �����}M��x \j-� �%���@���hD��������E�aH�� ���G�Z�h�� ������L��]� ��V����7.xY\���i��r��D��u�2��@[��'��o���2��+���� [1��Y��Y�)U���5 ���1��S]}��!�p �h��B!�B� eB��� ��$4 Q% �V�6�y �Hj|������H��1c�j�Vf����O`����v&����J��te���� '��� ���v��� �wvJ=1��������Fg����3�,H ��P ( �>�zo���p�+���CwV��u���xw^��~��v`� u��}v�����2 @����qOgw�B!��&a(��F\����=DW���N���_�"�-���[Lk����53�m1������E=�~&(v��/�g=���<�=�W�a��X�C;�!���,$|�e�^��9����[C���
�����0$���5 ��(��)�Ie�<ve�k�.���G�r�Co`LA����yn�k4�"�dz�x����a�!�B!�
��2�n� ���PS �, �Ct�t�HGpH?��~f����!5����n�J ��k�*7�������'[�u ��\�G?�3
��e�}@]HuT.=N����
����5d�������U�� �xO���_+�� ���\���Yn�M+q$r�������v[]m ���&��d �����KS�n���a��B!���C�P7&$�)e���o��Rh�_���z�����T�~�
Jm���HD���q��'�
i�sw���?>r�)���U���S��<�wj ��m5�}���"�k1���Y0���H�������=�>Z/�:h�O)|� �!�PX����,5��� 0�P���d���b������F�5��V Ieb�j��K�hj -ye��BNn1!�BE&Bi�&#����!�\�V+�+�����H�O��p ���(����G���c� ���,Yx�9+A_��y;��:�&}?Q��iOGc
w��� VG~�>v��DM���.m �G8��+�pT�+�P�z!z�G�r� 98������}+�����:�.�x�����>J*����G^o�X(w�" �E���Qi����}u��-����������3g�I9���j{�r���>�g-���!�B�kWd~y��2]���V�W����c��#�����O�\�c� �|"� ��YiQOU��i���1���y,�m�5ig�h �m����.pMB�_���y�9M� �kFj- �6����-��H��n�D ���2���� �S�<74{�&F��IB=��D�:@�HUx
)#��Q���L+�L�=9 ���!;;����%�6I�MI*#���a�k�B!�"�2]�����]� �������LUa�����[���G��� I��%�����e
��o�G�]���0+v�@�������}O[�������L{^���T�J ��7��}������D���:w��?�~_�d3���������� �C�����x�������-�F���\ hhh��������x�b �M�PM�d�>a?�<���B!��d� t��[Y_A�Uc��OUE�}yg�<���z�b��2�r��j ���#i9��b�9���^
w� LZ����E�LZ<������\d ?�������*n�8J��:hV~�Y��v f�
���E �`���O T�����x� w������?�l��������xN���B!��d����w��mi�3 4���~<��������4��1r� xgp���=�8�=^��j�p �3�/ �3��v"h��%U;�DZ��}�?��^W�����u�� �`m���H�z��� �� g���0�������t�*������v��@�j� �?*��1��w�� ,[����Vw_
YM�B!��wSd~y��{�]"���tg�.��+�����m�����X�YrU���oa����;o��{C�9�)������ U���J��i���M
H�-�u6� \sP�VG @���q;3~X���������4I����P�P
�B
/C��Wd����\��@�����A\~&���_�� "B!�B�m���=��w���+B�^�}����\s�6�9o�d�* ���{�K�����@I�KL��G�_�r.� �x��uu�L���p����U=%::4-��<!A v�
�&��9|�����#�.���i�o��r3l ���xp�}�Z����� ��AK�����!�B!�:�_�#��z����"�{�Z=���H�H���%0�v�|�@�i;D���\gH�Q�o, h�V ���s�Ni�'��<�km���oo�S^ 4<��>(�������L�D'���Q'����Q�*��A�� �q"l���
Kp�B!�B+2��3��
������<��)ww�S����i� �iT���r,�k��� Gn�� �D ���,_5>�<�:�q��]jn������������AU@�|s�6{���9����a�0|��.%�j����uF E����Q&2E5����k -~Q�� o����� �B!�"A$E�Kd�mE��V���}N�[.������A� bu �(4n�je}�M^�1��C��v*���J���,h��L�����nzK�*���7]�N����9F#�Ck�(>��5q&L��o~��hp�k���
6�9>kq���B!��w\d~y��2�o���0n�
�6$��4z�g(V��X����#�4�$�>.,���������y�p�<�G�]�" ����>�/ v��a�t�; 8�O�T7�X<��m����1q��AU��k�b!� ��6�f��������i�����$&@�fjh�������fee�&2 p�e zO���������z�B!�z eB�4wM��<�����X�� /H�L�[ ��Cx=�QtE������iQ@��a��M#^)�z6 �7��������b�[�E \3s���J� i�� � ����)�(��;��v�6 8�:V�������
8j��������o?���g�5
��������;`�vH��-�.��������|���z#<y}��U������z�'M2& �� !�B��(W��.��
�HZp�<;hEl������F�k
�K��5�����M
���G�<��:WZh����_�9�p���)�}�����K �o��_2C�����{"
�j�|t�u�t�����}�������v����q2;����� P�N��!{�
M.�V�����|7WW.,/v�}� �>&�B!�:�_�5��
��sB �F�g�+b&�_B�nCJ�!%Wy�����dxK t���N H"��]K�����f#v���g���b�D Ny�l��(-���1���^/�h�����0#r�w:�����PF�j�4�V�Gj�4�����k��@���C:��J��Yj^Y%���e��BN�:��I��� �]�hQ}T�R��A!�BE2 eB��V���}
�:��6v��7��.>��Z��=� `�s���H����& ��_?�x �g�����Ny� �E���N��O�����6���o}��^c����\ �
������?����<�SJ ���=��~_��=�� ��d��d )^y�~^y<��M5b�B!�PD�P!t�J��
:(5��
G�����n��5b��P�����f��W:���C����=��;�����5R( F6
V�:e��4yLaM��C���6����j��G���3B� m�<���?�l��U��-4��������xN�J�==�!�B! ��/B��m9��{Z�G��������AUU��.'y���"���Z���Q�����|�_������f��Qh%�-��_3�N������|TTs8��������C����@�<�� ��'l���XW�_��t��u|sR3�f�J6�p �l�Y�9kN�� ��B!���R!t���-2w4#�����/� ��fCM�1�� �['5x�Mh���IB�MU�V,=���(7 &��z�~�O^� X�;�S�2�oy��\����}eU�pl1�P
�@�:aR3�pl������M~�|����4��%����n&i S�S!�B!�"��A]�8�:�=���(�?�o{V��I<������H
��Z*��Z[y��z�j��u@�����]�9{� Ko��������;S��&
S�����*7��D�s�� ��S���q�������>��r��Q�m ���p�v��A[_���QOw|���M$6
P������y��s�n XM�&b��z�_�z���i��,!�z^Z����g���B!��5&;B�.��U �o��g���2 ��i�m� z�����<������ X����(gL�8��� � �0[c ?������5 dxK j{N ������������ o� �\D��7��w�=��}I$t��-m?�G�-�5����� "P ��~K�g/����B�=�Oy�~�. ,#VV=BaM�� n� @H�0 ���[)����j�;1kI��'B!�B�k��� �/!�.��=% m#�� ���� 0%ra��*����]�l�����*�&ot
m� p���49��MMq�� `u��i�w������hi7 �����+���-c�D.��-��/��2��O��&{U� �����������X���y
��]5���Z�vv��y���)n��u|B!�����%�P7�
[7
���|����7��
'���5$|�Q{E�3c^�v�O�5� ��j�������UGt��B���J����H���r��W����KC:��gIL����-�������y�-�q� ����<z ��e��g����� �+����\������={��V ��4e��u|B!����J�PW��l���HJ��{�r,@iP��9a����:�p�u 1�(�U,�~���p�i�+�������o�Y0��3i%���� =������AIZur��`����������������Vf C4���S�/�:>z��� .�V�,�� �)����2Baty�9fQ��e����3�8�''5���?��������V���?�u@ =���T�����*u� !�B!�E����%2��!t��3��V�S � ���Z.q�!U��s�j����S�}�R7s��r
!p����[�0��e�f���E�c������g�]����.~������`s����_�z�2��]��*u3D�������3g��>��) ����b�Ay0�!*� ���/yN��?����U�U�@�0Z�o��*^K��(6 �o�RQ��e[�[��������B!��Bd~y��2����Z��� >����s�����Ua�T����~�Z� L�� )���U����> {-)��3u�*}c m�Y�`R�W���k� ����}\9�iW{'����n2M42f�h_��W�G CjZ��39+����� �`O6��5yN���-~2C�T[��K����~��8' D]�U��c�i����/�~�����m����JO�"��r�m��B!��u0�Au����N������� I�Xm�y@����-E�;�D��%-'�?�����F1�6q�^�6;�F')����:N |��Y��_���zB�~�����{^����&I�������G�&i- ����������#���������-Q������_/Io�>�
�������� Z���P](��(eL�2�J�@P����*��C%Yg�� k*� ����B!��0�A��PE��V���U-���!/ �w6 zyC�]���F�����P��&�q������AG��<�����y��� �;�y�X=a�� �y�=�U}D�����w�K����y�*}�"��� R"#a �;����a�z���izJ�������:S���JR��=���D����W<����[,Q ^��x�9m��4���|$
�U��H��
;)TR Q�i�)K��:Z��� \P�B!�P���K�R�p�R�/E�����u�r,�k�T����G5x����]S�X<��&mc� wM�����er� �~'��/B�
{u)����A:���!�~����b��QXR'##�<��-y��2��� i�?�2��]�ku�*�>����O�T��)��C�x+������=����Of���a�)����a ��[�yO>�%�LnJ���/�����T���TLI������ FN���8����aw9u�cA!�B}�0�A�w�6 ��_�+� K�n���Q}jc��$�l
��N�F��3�#Mnul!��� x��r�>�1�Q=I rS2�4(���\��.��|���� � ��'6���m�VK 9>[�s�O>��j�7��8��m5�; �I�*oF�G }����
����SF*c
�i��?�a[���=v�w9!�B!�"�2��:�������+\T s����nA��Kc��&��Z 66�4�W��u�kP�N�����7�7�n�7�$�(2���Cu����t��� ~�v,$���� � % 2�
6yN��8������H��V�B(�O>����'7g�a����@ ����GU�����H�d-�s�6{�@����[�q�������/���:9���*�$��t��rB!�BE e� 4�$�>��� vc�za��e��S2a8�)�2~�����lGB�gIt��)6�r���� �������8 ���7e`1����jD���@) ��Dm8b?`U�2U���U��=�iP����/��mD�h; �� �������o��S v��R�'j6�����_4��v����?�}\d) �,�}���� IDAT�V�����N�=�>�d�\�X*S�h���T ��-Z��]N!�B��m�{��lu��=������rPB8�2~�!e����X28�V�<�1��@��cu �<���(�/
-�)`?4OTj��� ����.�-���%�gKFy�Mu���z �������h��� AKg ������LUE�} ���F?]����}��t8�������{�N�C��X�g��'� z�[� �. \����I S����y�x���5�U�?un��]?������MAG�}��Z��Y�} !�B��"��{$�S���WtM�J������_4�$U��h������zss�BFh��D���ZoH�S���oaG2r`{�� ���<d^K1��u��� ������Q��f�t��F
�d�KH���3��V����;�T�,��d2K ]?�-����a�:
E<g�yd�R�q'f-����^�b��'vi �6���k�j�8�lP� N`}������{����RB���B!�BH)2��cKl�z����KG;�~>���oZJ���Dh���M�+P.� �T�����N*�3>1�M��f��62`�i�h�� i9��b�9��cX� (���U���y(s��@] X{^��S�$�!-�"5oJ�+c~���{ ���V����W>����c����
�p�w�V��$b�5� �M\�7��b�����H�7(��%�4�����U�6���(gk9���4Qa+��
6B!�B���P������/�����^���wI����M#
T�L#^1��B �K�S�����N� w6hgT����� wNH�@Zn�\*%��Y���#�h�^5�E�.nm�D��w �KG� LI\T��[ [J��%'&����E0���� �0�������Y:�z��)����=��Q����) #6�-���'!&~�����w|e� ����^R ! �$`$`��aA� ]�(b�zgAP�$�'E���"���@�;�� �H X( i����2�����dvv���x�p��S��E>�y�)�>�^��Fnmts�uF����|�x��%�@���Q�z��}��,X/���1!�B]y:b����cf@�+�X�#
����D�%9y�����M��2����s�>[� |>2@�\����������B�v������Z}�2V�����"lZ��a���~�/�O��f��P�&.�(��r�zX��w�!�����g�� TB�[����hb9��!;����)��7O�������kX��z�/B�RL���H~��]@��z������� !�B�K�c>�c� B���gK�|��&e�w�)�i*�w.��h}�3��mM}����(�8��?�04VS��K������.�s�������I�����{JK���( X N <�|�&�#2�2� ���b���G�r��l<00^��ms�Wjj�8��M�7?l Ru���s� �V��h�� >��X���a�� k�m ��7��F���v��� �%1�^Y��{�HJ�@ B!�BW0 � ty0�+M'}�9 ]�����j�� O��m�-�M�EI4,]/>||OyH�#�5��,{f��'�vO���%��+�T�#�j� �t{W8$N�K�{{������x��5|E���w"�09C �:����o���J2M��$�����?J�����*�����{>9 �.9s ��h����~��7���p#<C�C^�t��������$B!�B���A�.���3�zf�k��������U^8� ��6��2D(n
<�{H���� ���t��0�% ���4����<�}j�TK��tjty�h� �:p�O[U<�W9A�~��M�ae����j0&@��������vzg���,RQ��pmP��a��0���@]����0 ��|�x���[�|��;��~���_/B!�B�-�A��&tT#Cc5�9J6�h����S~�g{i�Ty����������;�QCc5��Y{����� ��� �m�'����' 7�w<O��,�4��6 `r������0��0�V��F���� Kh�tp�X+U������q������n ��� ����L����]���u1B��_0$�?%<�N��zB!�Bm�� B���J����Z��G3� �(�U�F��*��������cC�T�������r���[X�n��~sf�"�&� v�* ��]i�mp� 1���E����B�� � � �[��i�p`$�x��������]�Y\;FY��5&�%W����3������-��K�
S��3�;��� ��8��Et/���z������U{
_i���!��TS��R��wJR� �&B!�BW���|���18���]�5��p����)��Z �.94#�4���A���.���<�rs�g$Y�����)I}W� z��/l-i,:P.�e�m���6��tK�@�"���W��V� VF���<�� �� (�����{P3A���]-��:r�$�� � j���t/98&�e�f��Z�C)��
|�m��������g!�B� �1 ��?z�+���~��)S���??�6l6l�m����SO�\��� t1�3G�}��4}�X��������9�GG��E_� ��]���'O
�,���b�����,���L�O�V�a[����}KWNH�_��M��;��v������ ���9���c��zoe�uA�� ��a�;�?3�2��c��]��z���'" �\�n� ����%���k�r�e;T�Q��^ �sVU��`98!�B!WU�����k�^��O��m�6� �������}�v�R�v����~��>�����lA�F�d� �K];gU!�S���6y�@��j�_]�_�����q�� nK�6�0��Qo�3 |�{y��u�m��,#��a���*0l�[&��n�����R=<��,��8����'=V~_f�o��!�B!�.wWQ� !d���>�l�n���t��w�yG�T����������^A�"�2G���q��@x������G^�? �= �ywXd���w�EJ� �u��JG�
rQ�xbjODF�W@�3w�ev��� )(���k�Z�����U�)Mz������Zf���9��6�hefy�( ���7��B!��j]E�2���t����o�������Q�F��KBW��s��O-��a����V��%|fe���������{���K[����FI�4 � �hk,�������-����2����C ��.j��6PZA>�r��@��=�P�uo�~����>��}P9�Og]VI������_���!�B��
e��N�J��n���k����s,!����������_�����5������� ������Ql�=/��}���4]:��: ����u1C��K�`�j4 ���n�9E�u�=����A��+M��:�M���WOu��'
�m�l:|wma k���)�A������`�����IdG��"x-���B!�B�R��������Wq 0(��v�eA�F�t�+I�6pF ���� ;����1��8-)���[�p����S���[�T�����U�� ]w����#2�R�����A�qj������ �� �$P����}w.�%@���f���) @(��<��m���|g�D�������z������sogjq6D���L��G�3�����{��v�q ��1��k��9M���Y+��8���qfB!�B#++������)�X��R*�������i:_�K���V��Zg��'����T�)�Y��l^���kg R��Q�S#�R�����&����6
&]���&m�x�W��^���V���&<H~����Yx� )��"~ j�Jw;0���� ���
�]G�@@�����_�����g��00�c����ZO{�4�!}�B!�B e��j���� ZQQ�������G�i�A�2Af3��;3�[��[�'�i
�ua��n }�1�v� ��6aH�d����Y/
�,z" ���v�<��$N�x�MLl'����<(�P���d1�Nz��Q�����t�X8~/90���5
O��M�g�$��c��J�l��� ��0��1���|��c1�b/����l��.���@��m�I�� B!�BW��h���R(��������������]�L}(�l&Y|����;Z����|%�f�^��T��-r@v�B��O� RNIQ���7�����#T���U:V�h��q �
A������EO�?;f����� J ����J}�� �$������w��T����\D/��I���O^u6�% �� �hO} ��G������W��+e`��{� 9vE��G���aIZ�(�N�'��� G�����Y K)�� K��`���*hw�B!�B�J��2��}����~z��5,����v��%66�\^�~U�.���N�%��� ����;TR�c���98A�P�*A�d�4z(�Q� ��z�����0UmD^������jm ������i���BM�k��$����m��-R��"�x0�����{�q�����`4������7�=3�?�1 PNC�`a� �P�DtK����- �]Te����Q+C��� >a5!����d�W~u|�������w����rx>u��v�n��p�����i��B!��\EA��������f��RVVv��I x������� n����g� >�e� =z���;�z������A�5��?�#<'8�)�G� �3��g G�jB2_ �Nzkm������?�����7���z����Mq��X��������_��;� � 0��t��B ��QgV����m[�RK�G|6��? �V�Vm�����c�:]6�IH��� =�Y����Ty �>������� � ��V������lB!�B��we�Y�jU�}����{����Y��5����;�J|B�sp.��Nc�7P"������Fd<�����S6m��aT��sl�\�i���������o8W.�P>�qG1��nq�`�c�6���e����[�
c���f��V�+����>UK����!�������;W��!{;3��� $����E_9�����*��=��` P��=����)�]��r�B!�BW<���/!� ���8}�3o��'�p� �����[����O��_(��a9r��C�$ sf[��J[��!���o�%iX���L�� ����I����b.�����H�2����R ����^i?p�~ |?�*:P:���|�����T��E08R�������N Q;SV��&� T"T*E��K�������t�����f�i �~j��p�X�D���A��IS�, !�B!tiu�������K��]��8�i���uC�]����V�*�Y��.�s � 1���E0x������=p�I�&�\��qp� .�Rh����*��JZ�& �u��}���8zN�O �(V'@?����O�F&�B�-�]u)�C� 2U��XuRk����p= 0�I���D� �&���������p��Tc N5xo����p��LTmH����v���<NHo��^u7F��������/!�B!�~7 ���#����1W��9M>�1m4� ����' ��6R�M��/��!�t� �N'&�^��t,�I�SF ��p��7���>mn=D�|�3�-o����'�� ��T���R0��sS >��2!. �8K
5V��FH����M�6���oDf, �lo� � �zpq(1�F��W ��J�������I�E�B!�B�t������ B �%O�(^m�9�J�#k.u��U��y���QB�i��MU {$�ru��M�7���H;
��H588�'�:6�����J��� �@���RN
���z��Fd @2�I��z��9�2<�
��� )@��9cV��4���D �~#�6�Yz�H��+�����E ��f�1l��d��wI�!�B!���A��n]����f�d�s*���Rj*��h�O��3��_#N� ����Y0(#�{
���Y�� q�����0����O���.T���nF���_w����l\�&����=e��� ��3m��P��YB��0��#��| ��ul J �QxW�v*���f� ��;���I���7R�����~A�ge$��FEd!�B�+e��I#�GC���p���EBDF@����>]Lz^�C���-��
k���)u`Q��{J�^����<�] ���%� ��Q��m�J���[�J��t�^Z���]G,{���a�h��PZm"1[�s���@E��hz �a6�Ba.�Es Dl��rD39�]��f �X>��U�X������?��8��+�t@<1-� ���}�=f�:��x��v��UV�K���z���B!����A�~W���}](�a ���(�4�7c�-{� b�����Cc5���F���/Lh��������R ��?���ki-��O���GU�]x�"� ���.���kY�X�C�L���������������a�PP+=y�����`z� =��(������ �5L.��� 9�� ���5B�Gxm���~��u������Y� T�F��:p�s �^�}�>.� D����x\�VUB!�B]$ � �����"�c���X���j0�2V��t�hIB��q+~4
C�4 �y�!��K�S��:�0�(�����d����Z��6����W���SZ~�L��l44�~W�G*Yg��'bC�3������ �R��. 8���i�����+���c�h�B���<l�5��� ���$�~���\�9ki�2�1�����[�q�������+R
����������s���1����&�n�d�B� �� �B!�.%��^�����������~'{� O����V����=U� �s�>M,~_u��������K�[���������"zq�
���`��X h� �d�88����KK
#o�r�i�@���+[[���*i���E_���x5r�s�*^��/�N�z�}��XJ)��XJ��fX�-���p,/d� Xb��\{��������o�u}��� ��Q���+�*��;�2;�?�Rwf�ev?�������Qx�.I��s�d� �3�0��$|$k"��� � � ��EUvK�����6=Z���� S�,�M���V9 B!�B����� �����A�K�����U"����Gs����THN�[��j���.�M��*����A�x�t�n9�A��f�Z����(�������'�d���%�{]�rN���sJ��gh��4����-�:� li��������'&�� �`_�>Q�9����? 2��H+h�r��)��o�j��q����q�,�r��[!�2��H�� *j i["��H(LXs���:b�����(�5��Dv����n���oI+L?��w x�2���2�=���Z���|�n=9u[-�3�;�QB!�B AVV����G�%0,_B��LN*�A�
���9�R�4v��k1��F� sP3A���p�P�\�X >)R��EhY�Ev��3 �� |�������� ���G�#��~��� j�
��Mpd��LQ��)�#? y��������� ����{[�P`(�u��bt��G)� Coq�e}#��f���!��pM_ �n�� �17����w4��V#�Q1
��� ��u��[�<%��3��)��\���QB!�B�=0S��$9E �Q K�wK���a�b� *FI� �a�bg�#��tl���p4��v��z
����a��k�eP�_�hB����sB4a��`Y ����)-s =`�n�Mx+oj� ����=��gK�1 $qQ8_u����]ML���F&���6�x�� �i�5�^��� ;( 8�8��{f��o�|�8��� ������w��������J�u�C������d P��'g��B/�H��*�Z��(!�B!� �)��y ��"0�+M'}�2 ][���3���~��Z�$D��������G#
��P��K�c[sv����d��� �M����b/a���� : +�))*1� �:�8*��fx������K`���&�G�2���*��rH���g�t#���q�l�>��u��z�;�� f�>��~���� =��x�/d���C��/�<���IIF�B�:e�Q�+�������5���O��p�X�D��b[���QB!�B�=0(��y�t��+[ IDAT~�d��49E �Qz�9az�{_��%��
G�C�Z�& � Cc5C%���d��`�VZ�%g�M -#� @��A6�{��* ����Y���L�k4�n0 �1Fo���g��C(���Xb
��5������ �{��<[}GF�D>WK"/����� ��.N��.��fa�R�����-I K���i����HJ~�u1�2o��c9#��0��2�(�-�&M�j�Ou���~y(
�~����tm���� !�B!�~e
,`�A�� H������p9Cu3?Z�J������) ���
���]�:����>=�O�j
������X�nn��e{=�� khi���Fk�N7u��SF�����\�@���:��xbZJ�Y@���zVa\�9�qw
�FPf<�o �5M$�_��Zw����bf�W�)4�Q (@)[��wv5 pN
���{� `/Y
�W�b��mR�t���+���@!(s�?
B!�B�H�mA|� ��+���C3��I �? 4��m\�* �������M
�?vS����#D�y�=���U7}����s�j���5�)5@�������?� ��9���<���W�S1eezZa����Ui=t�7)������6\���k�Z�<�|�V��������"K�Y�|NvH �� ��� mI������]�B>!� !~�} `����ze������=<���{Py[_�e��M������4\�� _��<���$l�B!��h:��{G\���c��(�_}�����S-�T�[��NL����?�����IC��< �)�k{��W�v��VE xz��<��#�G�*B�&�#N�-�Lk/�~�� mG�����jB���I� �Z:h��?d$>}sx���{�4y[
m&��B���b��l��OT���=D�|'o����*@)�����,�k�od���������� :�3@<%('� ��
T���rc��=����!���� p<B!�B�c���8} ]u�3~���k��!q����s���S`\QNI����Ed��o� �I����Gd ��yj�V��^���X�����������-��}�]]���~��k��)l�M\��so�����.511-/i��������X^ngD��Xy�����?�xrg�[���=9�����!�
T+���^]��+��RC�l i���6�!�B!t�0(��:�f � �m�( $�# �X�C|��i����� (�i�4��t�u�M��{J����z�U�p�(���A��ISfsw(=�7W�/L�^��� tl�0�Z���]G,{` Mw][ ������_�I�P �T���#=����pn���o�- 0�!��J#�B��[�>����{3��8���u�� �2���������>��[m �)m�qW�R^8���A!�B�_��~�U�=��em\���] #-n�z�X���� ��]��C ����9��B��K��+Wp�JVgH]�M���F���k�=`� )�|
D��K!% :+jj����5E��V�C��(|�l��e�cl\�0�Z����P�HR�����\��� (��.���Q�������( �7;��*��^��~�
�� #�rh7�����'qB��9+��$���>�W�� j����kL�F��j
�4U�$O
���X��I���!�B!�D�AW�� 3��=���T��y�Q>�8�,�# a� ��n�Y4�N����;<��!�����"2 0��>{ o���%q
|��@����w_
�
u�SI��% �?�����p,��G����3���K ��!�2��&��z�&������� �����Db��{��n 9M$�)*j i[ �� Ci� B5`V��jI�P�d�l �2��� �B�� b ����J[� Z8��*S�s�
�l��� p��0� !�B!��KG�ssy����P�N��/O�4%�o�C�]�P��n�<9�� ;�X�J;�n���:���<�O�:���0���� 26qN��1�D��j9������e{]�)�Z��"� K�>�t�'Z������O�I6/��p��B�T�;�
L7� ���o1��}a�D�<�?����� Qp�VQ+�9�H��)d}����b"1������J5 ����������ms��@��K�_[,��B���Z<J6�Ia�5�D�W�B!��t��w��AW Y��~#�Gd���I�T��:"�j�b|����D P��� �AR�3��vb@�z�c+7�+�C�<�Y[�,Q����O��E�w9�`}��o�4 @ X ~iEH����T!YF�gb�e�A����3��@���~H���-/�<i�^-u�&��`��M�'" ����G�;
�P��g���J!,����z��%4} ��o��(����k��\�r�����3+�4%���Z��Z!;��v eV!�B!$�F��j������H�.���/;�� �gwj��e� �J�M0�/K�(��Hw�j2��2]�� ������ik%�}����54g��-�B�u@v�B�H�0^�n��QWY\�l�o�|h��}t��8���>������^�2��
���>t# a����d���ED���L1`^B���~]D�O�1�����D���7��
@ko��pl.������* ���,E/��TU�>�^� �SC�-�5&�%1&��-�M������%6�A!�B����w./ 3
������u�f.�R��������Y��&q��n�))SuF���/���h
k����e���X�]��(����k���4�;g�t���tN������V����k�Z� (��'C����\�3�rJ��~���t �1*��I�%/F� �PBx��m�DA�:�RK�� �|&�G��� ��!' �@ �!�,��H�4@ U��Q7�nMu�c�
]��R~���\H? ��$��92�N���oh��R :����b��V�
g
=e@���x�B!�PG�1 �;��./���dfffee�qkA��c����iA����NL���]���L��U������b6"��d��%YHH�TM^60u���)~������D�s4v>������� T"-�.������&���������.s �Fk�g�d��Y�\���3��)��wI�� 4o/s��<(\�mGC��bMo��|"�8l�|��=` ��I?��3m�e4��Hke�9�v��a={�
g W�t����C`/�CEo�S N_B!�B��������-��� �\��lCmb+����vP�
������K=��6�� iZ�4��I�� �6"�<.yJk��ck&?d�� 1��F!��� k���)U��$(r:��<������'K�����1��'�� x��� k��3 �����N3d3��)6 $d#����M#�����G�_HoJL���l~��Fk�H�������%Z�)6y�z�a��3�#�\6�Jk�
+��F�H�4����� �g �W �U�<����!�B!�. �� ������C����W-I���������jd�nW���S��JO[ji�Tn�~T�+�[��${��� u�
���vP
� ������O(�&� T�ylQS�
�r,��( �8�G=\ O}�o*okK ���]�wM�*������5�-�
C�P (Uo��2�Q 0 ��!y]��,�V���w����n���@" ��)���^c�<���e{YK��+4�������� ��}C ��}��Y������r�b�����sFRjme��|2�E��S�4*�V
B!�B�`P]E�}<"�?VIP�i��� �r[��i�n���'N�� ���� (:p�&�����S��2�0 Y3-����Qf�NJg�����64b���vT�5�\jm� jIr�q��L�i+d3�/n�B�'�i���DP3A�M����4���)@�G?��qt< {v) X��V��Tg� ����LXu��J����TY���'[�T�Ul�� ��r���
��!�B!t5��K���SR���u�X�
��g*��S�q��A����9�����������i[�5�T��'j��n [�VD 8��:U��@���Wx ��NI3 �3����A{[7M/�a-��X�<|k����X��o���H�4������%�;��z&4�0\�n�(��� ��33QE���,��sf��.����v�:h���@����K� Q�b���O�j���r�����@�B!��U3e��I�����/���K���� Ug�|a ���sV�<���hbA��#v�%bZM�.L�J6`
�@����T�Q�6�A�,Z�S+��3��3��A� %]� ��[34`Z���2�Q
�����=0 ���3 �6�B6������W����J�m"q[�s���u&���
�A��hxu�e���w�mc��OT�$��i�2�
zU���=Q��*��Zg �$s^Q9�B!�a� �2������a����_&�#,���� T +�/�Fz���;8c���Z�3 G����e��$��������PH[5PC�l���om�\L�o�N���R��s|�������zz��a�0�v�D 8�8��n���7��-;�8*=y����v+o��[y��va��,r �"�3 ���'[f�sl�]7`b��GP :(%7M�����3���g/�� �s}7BO�_�Wi�2(#���c+o���:�� $*�B!�B"��AW&���V��)J������'�#yj� :��y�Y�]@ �*�/� I �Z�
�AqNS��s�M3�FZV���]���������C�a���7��&������K��]s[���Z����n�/G�~_��Q�1�Kh�t�5B@��\E�#mK�d�,$%L��&F��KN��j|���F[v�
u$b�~�Q�/7��- ������7�i�9���1�����Objg�����e��D�B!�BH�Ate�/)j��q_���V�d������ �9�R�um.��r�o%+(^m�9��c������.���\�o�������� ��8�{jqNSa�9��=�l��u�j��<����<u@|�' e���u�?������uC�5�����E���tm����D^|t�6B
�0����1��( ��Z�+��H
�d! �������z VO.s��Ms�PEJ��/U���mN
��a1d�F����6F�d.Zo� ���u����D�7�y�=��!�B!�P[:����K� u�V�*�&/�����������I�X >�Ql:�s�� ��O$9�p����W?�'��a��e��BNSf-��
^�{��R��tp.� ��c)� �wl��(����O';�}�� ��� @��y�' ��� @ ��b���,{� ��"�.X�
����!>3�y��i�k�2-���5 ���� K�<AGm����(��t��|B� {I��h g�duq���V22�{�%:�+�B!�B�������ty���+��>�{����0������<�sc����>{�9R�zj����[X���M�����, ���I�n�� ����^��p�Dq����C3������M�]�+��r6�s�_� z;�Ji���F��7�3��R]��=O�o��c�x���N'?,J ��5,�m�sl�l��:z�����s�a p����r-'���o��qb�\{�jS� i�K������D���( P��a �� �B!���:��;�/�+VtVT��������m�H��5����s�V��^��O#����^�4�(E�C��t�|���0MY ���e��u��V s���� ��&�������i��b�'RI_����b�( �U4d �=�� �|��I�F�
����A"2 �����O=`~\���D�SO������p��]o{wyQ9�� ���i�
��v������ ��L% �7�F��@Kh � �B!�. ��+���5.0~m��z�)b� �������An����f*�4� �_:���l w`�Pa$���
���9�2�^_��p���I$��$�+��U�W���4�p �x�U�SR4��-g9sx�>�.���j�����,����o �rgr!%R���( ��n�TF�
�L��� �� ��/TO�:W�����5���, ����4Yo'�F�/��n x���Kg*������w���F!�B��`P]��,�uZa%�eBQ]�%@�� P�R��Q��3v��!�f]���.��mX*Ox9�4�.94+�[.Tji����PS]������}2z�S��F n�M>a���5�n�K�)��$�+��ibb @��z��A-� ������l��;Y,F������ ����f.��L���5�^>�� G�L ����b���-8ju����c~�4��Z����_ �H�c������j��Il OL������t�R��|�
y��u�F!�B� �K�./ �,
�[��e5n� @�PB���F��� bo�[�� ��t���F�,���1 N����V&������U��%�� <�3����5$�+U��'��{�O�������~���"�x ��[_� n?����5��D��PB� m�}���X�6����n�����,��w9�9`�*���]���t0
�~���>uH����77T�[��7�U{DM8a?�J6S)t�rm� ������]�*����=eu�F!�Bu| �������o�����=- J(�����jp
[�Ih�"���������\F�ZK|�����HP�����*I3� �R��T������'&�� �`_�>Q:MY���!�ErW5 �����f�a�� ��=!� �V�NR
Q������������xb���� �� �%~�������� �Dq�a�FH��}� �� ������K
k����C�2%m�����) �Q* ��� h����v�SC�-�5&�%1&��-��� B!�B���%tZ]r���y>1���GxB %� -�5 e,B
���E��VJ(�h�����H�o~M� y�O���7#)5#)�8����s���O��.� -��
_/x"���`o���B���;<���e�x��Hk7]�Is��UO��i;F��WXh�`;������0: ����3�/ ���O�6J;����
�� �LI��p �P &����%g��l�B!��o3e�h~�Wm�� T�n G����U: ;Z�g �AR���u�FDC�T��:B��R���/@��!��1���O;��c�z���^���b��nj.u��UU�� C�4���vw3���X��9��g�Q�d� �#�MO�:�;%~[>���r"4��o����a��t������BX�- ��u�� G������#k�i �����K/�<y�a }^ ~�����J��6�o�B!�B� ��AW �����w�����f�yI�F6� Z ���B�R!u #�� �
|��GC�r��}V��#���2����^3�e�v�e�����~����$��������Wk���lk 0 H������������4���tM6�m�����N1��F4 ��
�����9���{�@���7��-��5��� !��~�x�;a������Wdf�^�|�V��H��E �����*�`��d-�r���[,+��L)#�)I��--K�(}���!�B!��o�2�
�_�"eh�^��~�Ut�7F��Px�]e��������F�@��^�i�[ ����o�U��<��sc��g_i�����3=�v�����b��4/�R��?�������H" �s��|B!�0�\�Zx�['����i@!#)����sR �v&:��G�$Ja����A�YL��������;0��#����� pP3��� ��>�p���b"2 `�k{[�S>���\8-9�D���D�� �u�u�J�g���2������U&�Ps4n��AOOz�bV�B!�B �2� !���������?t- ��p�H#��)
3a���7�Z��/+�a9&��!� �% }�r�������uV�k< �c���;����&��+ g
�r���m�G��xfp�����5�����E������F�����@�`���e����$ h �7�aY�1m�l�8�y��}>x�G�C��b�-� D ����.e��*� IDAT���f���=�����E0� T?�9��7 �@�:���-)�L���0��0�V �9?�&���������7�Uz� ��=�����YPf~�W��+h��,���B!���deeegg����2�@��u���=��RF��|���bt �]E �2��S��7�-��I 4s���[��%�"�c��i&4E(wb(�V�T��$�8u �����`�����?��s��?����I��z���q9�-��c����Db����� �7��? ;Z��V����B��#L���J��<��"�|��"�������k 2��� ���s��k���V sB� �,q m��� �H��_��s���G6>U^8�U�cL�:n !�B!t%���3'�t��#����t�Q�Hju���v�u�<�[�Sq���lfO��* j�b��t!3 �Y������G�v*��N ^w:g��;A�p[x��� ���#�EkF�Q �������3 sf���������KV?�m:����i��DAz����X ���X��y,����?�>���� ;�y -�t���Oh���6���FY_<�( �x��D����~��'�[>��X�jE���r������k�/�����c���bauq��'4!�B!�� �� �/�+���[�� �T����:������b�c ��0����a�zhc�����u� Tb?��Xy �5�]iD �+���������oODF�T���[_�V�t������c"k
�}��V� �C~�a��������Y K)�[^�wf�xkKh�
Z3�@)t����v=@~x�|�GEd �� YD ��
�qBV+�3�
|�-�����7]h#�� K��`������B!�B�
1Pty���6 �9MK�������E�=x� (<��3 ^ T����o��!e6HL3����� ��G[��&�#��������� �o ~�Q��! z���s�5�z �z�wES�-�A' (��P���EM6�M@�:w���8���[��@��K-������v�_(�����f6�\�v�@���!D:�YnKqNS��s�2���=T��H^�����
�w�0���p�bo !�B!�!u��w�)��L��M�������b���W���5���iI�h���Q:����4�{X��i p�Sr'������[1\>O� P
FJ�Z e*�7�k;�6��FL���{��e�t�4���c��7 �
��O[ Y�U(�2z�,�xy���Q �4%���x�B�[�5���2�#��W�g� �^dP��9K��B�}�������m*��h�O*�8[e�.n�� V��# �Y�[��d$�f$�@�g����c+��[@!�B����K���o~��#�Gd �,
�m���v��r!�����h �������)��ki�{�����P{}��c�E�@'�D6��zk���j��1�Go��v{��B]���`{���{�'q��� Dd�wL����b������EX����x8�=YQF�~+�K��|P�?C[��k>r� ��f���rh3C,w��6�=�P��r8S����z��J���k��,�.^�E�5��(q����s���SRtA��B!�B��Ate�T��"|HS ��ZC���9�V����B X�y��[�F-"u{���8o� �F�� �
J��K�o7B�s�����@���������HkZa��O�#� O�����W����UUn�w���[��N�}e��u�����8� �! 0��}�mt�S[,������8�nt������E�����������|�����~i�Q [���|���^�mM�����B� x-��XC�l���o�R�c�#����Z�<Wji���~�)�� �B!�~CX���L�x�|�4�N�� ��w�� ��f�����E0x�����yc�����c� F�j���]�������.a
��H�o*�o�\��K};�� 9.$TH��uO�K� ��� �,|��<����
���K��� ��� ��k#�
��wX�hT� �<��FHI����)�����j��Ag��Xb
@h
��&f�=*��xb�
GV����a���� �e�v4��D6H[�����;�2L��e��lr}�r����SB�O��>����K���H[/[���C��vO�?!�B!�. ��+�������Y=�g����R=6
�$=0`F��TjZSQ�����.��.�2��lM��b�({
� ��&������H/W���FU��O�Q�fZ*�h����f�b
PE ���=�|���^6����)� �
w
z
���F�<���\hm ����n���N4}+������p�bo��d(,���0 ��Y���Q ���� 0�o|^(@5��nD��_�j�:�Ro�����+ ��ll��k��0q v�� i����se;���G!�B�K����)yj��H{���� (^m�k�Mjgk\r.��Bk���������e��������:�qdQ�3 ��/�S[��a������b�T�q��Q�
�o0��L����*� �
U� �!^Yut���]4��Z����?�6�]��)&�� "���oK� O�������,������F x�� ��<���E+3m�q<�TB-�&M��B2�y������q���� �B!�����2��e������d���u�#��M�.�sQ�()��E�t��
�J�IW�����9- ggv�������Ngs�J[]���?�,}���O �M}d��V��o
�h"jd�R�����'�=u��SaB���h��'�G�5o��p�[���9v� ���%��Q@�7��Zp+��K[x�n�=;5� ��`ty������OzG\�W!i��\�����Y�^�`�+U����$��B!�Bu�1����9� @�*���� �7�p`����I����L�K �/m��S�T
aC�����;�<L�5��~�
�/ ��Y]\�?�R^0Z���F�O�N����?= ����%RX� �����R_ �������#��|�8�4J?U�t� ���!�s�9����_���f����� ��;&��gfwH[
h���q�5��!�H�g�-� �&�����;���� � Y�o�0<!�B!t����)��X�SC�������z `��"�������|tY�G�bO����;=K,&�2�@�#-\�R��Iu�N��VQ+�XJ{�|��G�~m� ���.ij���x�g��[��� ��� ��F�^bo��Q�5����M������"?h��u�V�S����^� �1^�KE v�e��7"�P2� �y�$@�QG�"zXm��( rL5,M5<Aoe�N}�R����:y���k������j �^�9M�����A!�B�n:b����1�mHJx�~���<
���WaWi ��9�?��:���#����F�$�E�aT� .���p���w��Q�����������@��@� 5��E�N�H��P�C=A��E������R�;�"GS`8�D�� �J %�@�f{�yL2���lBQ6�������3��� >�~�o�N)���m B���K7�@#c�^�7�@&�!����p��� ]����K����}7�:Jo�<�1 e
#�� {G [*�j��=t, \'VXwO��A ��ef����a�������0N�`�p�d�vB��%�*��S����������G�[���g����)��9���u|}.��g-N���>2!�B�@���=���D������4���v?��u�9���H� \�� �O����s��w�i�a� �zy[
�� ��>[������Y� y0q4~����O=��bY K���Xtlm��U���>�� �U5=��<��"�.�d )��ZJm��\c�O��|�T��Dj���K ������w/�8��� ���Mk�}��8����e �ZOx�+Q �R� !�B!��D��w,_B-����o��Yq�4����-�K�N����� �i�aS�>S;��V������]�l�.�RD ���x���j��7����@��������� �e �Z�D
�G�Wg�D;������ ���-7�_�~�y�����4���X �.u6M�K ����'����{�^5q�B!;���������Z�CC!�BE> ��Z��+�['�[����!0 �$��R������<�^�6�y�b���_�Z������pD���Z��y��4�_� ��y4]L��u3+z�Q�ffE�.�5��9�#{������>�u��o'� z�������I
��/+I���8�z+��n;�^u�rO������;�����s��5�r��6��B!���`P�p{g\�;D9l��j� ���OK��� P[�C ��L~�_� ��"��&�]@���lC�'�J�P?;_?b���XX��t�9�M�6?�q�L����Sc�3,���gF������|pFv�
c�gCj��w����uw��+��Fi1��W�����%�a�"!������= �������!R �U/=h�3�g�n6
�N� ����^V���L�9��B!�B�z�| �pR�JL/
�^H�/ ,���� �����w�����Ah�l�����#m��#�&���2s@� �
��3nI>�+8�YcJt���i9������;'ki}j��:��LvZ i9 ���� ��0/�:�]i|�'{fV��&cB�M�����L<��i����B 1���C��@�`FL�%�U����Mz1 E�����"��19E���m.:�W��im>���pc��z���7W�\=pou��5k6��8} !�B!��� j�L����w���W>������N����b^4�P#�.������ �|*�nd����W �sY��1Qek��fQKb���D� Kb��������A�q��3��6?Y�b�:�u�D��h���
��t�i�wE����Y����,��
�.�- �g�sL����[
z�@j��P "�j�7��� G!�c|����������_/>����>.��2Du�����vN��M�r~�����R3�b�6�
B!�B]X��Z8�D���i~�?���^�9M�� �%������������_���UW����Y& ���� ������?��IR�ID�����%���!6�
���i?J� b����� ��w����C�G� G�������H��/F���|��~~���w�T~�2�"�':$�>g#����Hn ��hO�Y����:����B��`/u,�?rLZ��\yF�&)"#ib3�B!��"0S�
��`e��Y�g���{A�.��=�4���.����������;{�WKC���(� �����V
�n��M*� L6��O�������������h����l �,o2�W� ����4�Gk�E�C�;�BFv} �4��f����_�����w����c�M �D ��j�J^q7��C'���f~c��M���������?9v�2�����z�L��Fj@��4�
B!�B�Y�={vnn���E8�8��y��Q���K����n� 7[�� ����C��U��M�wJQ�� �Y/
�����e�.���Pr��.������+w�����<$�k�s��f�[������'�]�������\��yw���BN�C ���7�2Z������q ����[�\�pQ�%u���8Q0a�9����3o {�*��-������!S��q�<e� ��q�G�tE6�B!�� ��� 3eP��w��]��?��|������� Q�7��\O;@U\m�]{Q�,S*W�kh����J7�������*�v�8��@��e�R&8i��Xe: }��d��o�������5{`i2����� @��'s�����������L�d�gR�����5)F���Cr�n��O��1X�ib3�B!��|�A-��������Yu}Jx�A�p8�:�-~��Y��V['� ��
� �C,P���� �,W���z:R1X^��������P`I} E5� NC���:iR�
�T��'@ ��2��~�I����/�Xz�F( H���e��]�^��!�&6�A!�B���A�bm�}�G����G �-/k�l< �����L���{z�����p}�&Jm�C/���j,��4�3�Ajgs���6�=/� h� dGS�� %�4)�>��j!����gF��@�~�{k�?h��2�0�`�!�B!�.J$�T5/�Y�� �[2������Mk��TaM���� `h�>v� �
�� � �b��b����r�D�f(������5�K�4_�.\'|��2�o�O�$o��D� ��%��Y��'.E�r���������� ��h��[B!�B5#��� 3eP�W�_��Y��w��A��+���+��j \�+��&R� ��S�]�Czv�7�*\~���G����<1���^�u{�J����j,���i[�K�,��9m�?\���KH$��\�h���O
�����n@�-!�B!�P�a����LM�]�� �:�t
�U�R ?�5t����dIj(c?0C�����s��j�N����~���$Z�6��]��� ����zI��B1:�&J���T9�Y>
i�K b��iZ�R�P�$�V(��c���l;����"'�l���m���������_�~4���W�\��"�B!�P`� j��8 >����Weq|���5�:��%�E,>b��%n���MS�<���'��S��qVuq�����i�����=�m��O� ����!3�b: �8�������e�(�
�T�:���������+�VgB���K�^'^����Jj�"�H��t(�h5�6>�%�&����*\�g�(v/���( �9�[p�������d��Uw�!���� B!�B�7�%U�Kd��! 8��Z��{�PS��S��v�~�� ���j��m^7��
��G6>sb����zs��lmG�����S��> :�f�G���u:�'��'=�k�:�E��K����!-g�9w�-�r|A�� �o�� ������M�S�~�����v�7�S����S�|�z���"V�v�j4����|��Q�r��m�g|Az�6���-��k��}q ����V<��[��_|�7 ���<��|� [�c�`�?6�����B!�P��� �����%2�WT�l)[����^[���������J�15L� �E,���V�*���z���g
T��jS�q~$g�8��B3@�!�e��;�����L��mS���0����o�C����No���Ghxf����&��p�h�G� �"oi�Z���i�A �-�y1��_/�
��� , ��a�kQ��n�.����� qY���y(���l8����#���:V� ;��� B!�B(E���H�S����5�U��r�� ��"��7t
>z�bCz�u�$�/�A�wJ8�������/9��A��u�Y1����
�����h���H�*���������q�u\3O��-���[�)�2���j�������/ /��G�oI�0��_�����]M���)�~v�<>��F���7�3eZ�l^���M7� ���� �B!�P��� ����@�3T � ����P����� P������*!��9C/� �!y�c�)CUD x�,V<��@��Ju���u�2�dFC ��1����W��5 �L8>��00�2@�Pn�S��Sk x�6���A��`&�k�8�# ��h 8�>�+vJ�
�U������p94��wTG^��?
=KFw�� �����D!�B�`�(j^"3�v�+�DB�F�L�� ���7Fd���$xAgK�����������.�+��S�\��
�L��+v��-s{�I������R�
�4���A��V�$���[�Tbi^�/o&��JZ�3>���0������+NL���N�Ae����i�� ��� xN���HU��m���~q�z0?����B!�B����� �{j^"�{������\��#?����Z����� -�:b����p|T�����������7E����w&��?E (�6�@k#,N���;4�Z�./,�~��)b@c�b&��0N��-{B!�B�"��;�/���=X��v�&�3fu ����LCJ@Df�����u���rD ��a?0�������F5_"a��G[�f� 0�o*�Y�����I�L!�� b�jE> !�<��F IDATB!�" eP dH�����h,WqDe��7e��� 7��^ ]^�q��[r�
Q {Au��Y�� ��oTh3)`�bR �����R�x����Ri����uG ��!�B!�0(�Z&CZN�����,����c4�_��
�x�
���YdH����H����.��" �lM��k5�o�NZ/EU\�U�j�6�~�����EeF`
k��gi���K�S��K~ h�Eg��Q\ e ^S���8�K`����=�t8 l��B!�jV0(�� �>�x����aX)�BN�!\T�EQ}����UoI�CZN�����zF�`��&��� |����GrS�n�#�=-x �����M��lM�� k���� �;�/~�����(
��:%.k��rv����3�/b/��}�L�U�������_fW=
����2&�����s��
��Uq ���e�����#��#�B!�P$��>7�Kd�
B �*\i�5��j���'�It�k��
* 2�0)��;���5{�=e�s|T�%��1 �:���{�����de�:�kN(W��n~q# ����e�;��RwU���-|R?=��E��������A!�:�R��S�������a������>*��>{<1C�^���_�� D����V[� ��o����1&��R ~�s���~�-#�B!���������@��b?0C�� �9�?<PFd�� �� W�J�������s�7���!=
k=Fuq���j�����#�7���o U��<�{�����)�kQ��J�6W�Dz���=W��`Bq�#�@��� �N<�w�u�0 ���2�2�h/��{����-q�[��b���~�xF���
5�B!�BE�H 5/�lC P������&4��)�.�HR����TT��xm1���jG �s
x"^b�����������x�DW�����������Q�v���X;���Xgt����2g?���RqQ��
������U��u���w��}��� z����s�4����-dL8B!�B���� �����%2�W�*\Y��!�b��6��%=��^�|���' ?���X$5�z!�D���m�!kN�� ��9G�V�v���CV ���l���+�M,�M�r���W�71����1�B!��D���H�S���+
�m�XK��{�T� �$�h(+I�l~�W�qR����
��Q���0
�������L�����sdd��%�u7�tW������ �B!��q���=���D����s[.
�xC����R5m y�H-��m3�{6/L�@���X�F����E��bi�g�@�������mWu��&� !�B!�PSD��w ��Z&�1�"� @XB4�h4�vQ�y���(�p��Du�`�P�o?���}��}~�)����+T �s��n�-oGe}(M}�j��4��?�Kx� ��/�,+�y ������m{q��eB!�B����@Q���6�:��z��M^�=b�L��+v� ��B@���L�Gz
m<'Lc�p)3�$=R����J}" @O����yQ���Ij����� *qj D�,7�=_? �t��;���e��4�h��sHNZ��,B!�B��� ��)s�:�g���{A��v�S��Ah� h��L�� ����w�9e����90���m ��4��a�3K)3�.!�-F���3��J/��#^���|��R i9��Z/��Y��"2 P��H���;=e���s�����!�B!t���=[��~��Z$���� �]����n _�w���6v��!�<a��^�P`��O5e��Zp���vt�08��l1Bm_y����������0�>�R� Pu���������S�Y]t��������\G|���? �����b]'V�oLN����Y*���AS�����(Gd$�y��[������W���r�5�
:�yM�NkW���F�xE��B!�j."��{$��y�������Q� �r%����yCU��lL��:��?������ 1C>`8 y��He������ ��@�P�~�>��� ��{ �# �Um4� ��C�^��������y_,�U
�>qd� � �M�}�z��(Zb%IR8��� c�_��yN���� ���vsc���V�� ;�������G�g�����L� ��/z��x��������]#�B!�������i����Y�z��g�����"��U�c�>��� ����N?� ���j�� )z"�d���������"�P}r|�W�ft�<��T�P� �� X �E �b{�t���iu���Z_�c��f�|)�!�>��~(�>vk�!-G9(�<i o����v�x �%?M�R�@�$�
�jH�Yu�!Gd �-��� M �������+m�QWd�!�B!t�0(�Z S*g= �)cj����P{�t��X�.cNubk���rNZ��9�����|������� ��e� o�v� ��4� (1�g���E�K=��nE���o*&9:����A����@7\^���m�*�b��� c)�t��:�"����c��]k �����w�B!�B�l��Z����4|���5<��wek�[�?
_���J��|��j&Yu����#�lg����)fQ��c�z4�q�<��l���b�%��� ���?u���Z����mw��/���{N+�-��,6p�U��lM�N'1Tw ��
��U����1_���������l���������!�B!�~�)�Z���(�t��RG��O�����<��; �o?u���j����g�*'e ��1 ^?�����[�E}\p��'�9���o 5���2a8l'{�~un�!�t��������V��z�lz�����J�U�:�K������q����H}��
?�Q`4u@�o��IR�D�%��:�-^[�v7t��N��I��1�!�B���2�e��� {2�)o��'3��<e{-u q,�|��U���L[�r�����f��G�j��<m�x|��Q�iL��iL���>���Pcj���pY %`�&7x��F���������b�3#�S��H:����!�� ���h����f�R�X� ���e
��&Le��[O�;��M���T�$ ,�>���`d5 I0��� ]^����||�p�5B!�B�9�L��� gU+��9�E�~D�~��D��B hxF�j�Q ���X�u-xYm��nj�){��a���+�%��B�G��q��f����{���GK ���L\Y���5w�~�O�&�R0�8�l�>���]������������F".��@���Z��N^ubU�����) L���` ]i�:����<���J�w�������B!�j0(�Z>�1U5 [U5�n���g�7�����.!��t����t���iu��D��H������^���*� C���l{�t���<5������-�`��UW��N�ZQ�Lz����?� @
X�d�����xh��k1F�H�|B�p�D����3��%� s�n8*��w��C��u�������fB!�B�
�� j���a ��Y���g�~�;��j�K��0���,��{���u�.����&��� Q�;������1!�.����E?�G ��q�<���R�@�
�9�0�;RD&�s ��^�e M�`����������P���T�e�j��8�!�B!�\`O��I��;��(r��*��{�����#7jm���^+����c�0��Q�I����4@�h�u/��?Z�.69�!��SU+�L�������� �y��j<�}���Lp4���7�U2@��.'�1W�T���#��!���R9 �yw��>���B!�B�eP�%M ��OQ�'��Xp�uY�j��N��U�Z�8*O����t�i�^Y����'��{.���IQ ) rr3gj��B�Fz�W��I���x%n�o����*���L t�C����j���wn���
��^�'?w_����]RHN�F!�B5#X��Z,U�sD���7��c��lbj��mh�r�1Z�3r
5&6F�Vl�&Cz6zr�_,b}����OnV^����UC\%/��
��������'���
� ���{ �t�0 S�Y����~�;n1{ ��� �:��%L�B!�BE��A-Vp�u�!���X�M��������,c�,���b�9��`����c�_���
%t��r�< ���}r����9��l�\��.� 6��R?��p�L���q�kwU�� � �����0��@�r(4K� "0�L[��"hD�V2mW�� ����u�:�3 D����� Xu��U������!�B!�P�!�F\����|��lmG���S=F���Z�j����7�a���)�J<'��n����Y2��Y�<})F����W�ln��b�� mn�� 1��9d2��ADL��40�8�=Q{��[���7�a���������i�cCl��i1v����e
P���Q����Gz���V9VI����x��C���*�5��}k��������K ���V����L ��eC�A!�B5k���=���D��� �ub����Et�L�I�M�QE
�# b��19E =����\)���W���r�r4G�"E^*\�]�9�9 2H��7� !�-����w��^�����,n
���ot����������Y��m�2���08.��,�!�I|r���J����~� �����r|Z�����7�B!�j"��{$��y���I\����g�O����R��.�k�_�
���}�x���#� J;�HA������X��McZ����>��M� �kL��d�J�#��b�"U���m?7vw��" ���,�p�z��d��V��?ZC��p�B!�B��� ������cH����T\�W����m��iD��� �Xzf�}* (�GHP�^�+���W �=���oQw�*s ��t��3�:�lrZM���N��dz�5��X�_��Zo�_����Y���O>8�%��cy+��<��V>�B!�B�2��P������ �Z* �@Q��b���V2eN���;��gV�%���� \��$��1,�Bd��|�6�S�)�F��i� ��Do�+I��e�.��i�G��{ �
�T+�+�2|K� �O!N�<����R9��������3�� ����"�B!��U�AtM0j��R?3��9�S����
?�nokx�����Zu�!7� � ���
@)��" 9m��C&� g��g�z��yF�o��G?��{�">B��%�P��
/�W������m�(�� ��R��n�*|���q�o ��R �;��� �=e @�3�����}"�B!����#�Q������^tD��{���^o�������]�*s
~��}�E��_�r}����,w��� ���� ���s�Cm �#�D���)6M����2���W��5��\����
��2wFJ���a�A�SE��� �L�U��U����7~g�j%#;j��ds;-�s;��E�c�.g�!�B!ta� j�V s�.2&TJ3��z
�u�����,�~(�8U5�[�?BL�70�.�)rc�p=�������|��-��?�:IN���������� ������)]�� �Ay �J5� /�T�h�6��e�Dg�D7��B!��|�)�Z��
e�:��c4 � �Q���Y$*���zB�P��eJ�2`��������MJ�h��� =��$������ ��&Z������y5�h���)o�P:��W��/� $.�
*��zI�0r?����i�6B!�B���L����������R�
*,
.� �����j�UR-�H�6���J`Q),.�!��,7�� � ��!~���L�$���X�� t����|#8�%���A���&E��0��������%�b?����B!�B5[�A-�2 ��g/89&"��U�CR .�Q�z pg���*-=h#�rq�W�0��%z�T���XQ`��2�Y?� ��tCyx�� � � ��0�J ����=����i���^r��u�9�����F
e���8����T�5��@ t�B!�j��| �@��2�p��+Gd 0��A��� �lWEd�lU)M} ���D��/�n;�Ve�����C�2M�c��c4 �:�>���gC�����HM4�~�)����'�����7 �<�m���� ��S��k�E\�%8���T# �a�Bl��� j�����B!�j^0(�Z u@D�p��� �- �� ��Y�0R���Hk�b��*U����lTD
:� j��~�}z�105DV� �)++;��_�W8Fgbc���j��� ������5�"!�-�N�R2�1 � ��'��c�;�!�B!�"�/��)�F��G�G�q-��Q�q��0�xB��% �F6��K�k�B�S��TmeS�S�V�&"��`)! ���n����#G�9n�/�
� ��:b��{ ���%<���������S����Q�6���+�m"�B!�P�� j��;�^J ��a �A�H�H� ��,�z�P�%��5 ����P P���y��2��z4����!C� 2d��� �PX���-6�wL
��QN� mQu�������%�"��k������B!�B��
��AMC>����������F��4�:�X��}{W�@#cv��m�m��������� h`�&�q�E_]H��M>a �"x�Z�h)��^��y�%�3��������
D�:��0p���?�����|i����>��)��+�����z{V+�5d�a�Z��p�qq��2_�c��n����� ����oY���]�B �B!�B��� 3eP 4 E�K�o�i������V���� �F��- l,_$����l�{ >/@�I�O<�}��$�X�o-����� ^N �/�����Z����jA�R_�92T�H-:flg ��������h����������QR>KpY��h_�� '�����������Wt� �����HgSV�/!�+cP!�B!��`P�L?�{��K ���
e�N,=3�> ��#DJ�g�j�(���GDG�7����� p�eS^������������|V�^���Z���0�c?��� !�U���_Ch@�� R�M(!�=�� ����'����a&���V7���v��t���R����H
� ��gB!�B� N_�H���g_���Zaz��Y����&t����D6����_�h� :D��:�gim>��Ot
6���.F�!"5 #��X���������~�z��rxv�a� ���&����b�3��S��g���'�+�t��Y����63+z�t�m��]<�������0����z���>�B!����=[��~��e� Z�2�#�; �#�D73 �}�SMB$��'��3����.}}c�"-QgpY ��6QoV�U* P�PXM��9�l�S�\����N&�uMM��w���������odoI�_�U��&&�<�f��������;1�����8T�?��Hj��Q y����o�`�B!��5h����������| �L�:�� �Q-*������J!+w���{�sD��jLl@ � ���������k��W[?�s��)���#C��X���}�H���p����o�~t�n��� �!" p8 ��w_��]�}FT9�)?R�a�1���W%8� �q���^����t���Of�_^�R� ������U8���
� �2D &S�4 EO (�/5�}!�B!�"eP�4 E���v�;`��z�u�<�T�@���?��9�:��cGO�`
�U.����y2 ����#���;kR��I�[��btG����{�����S
l q�A��������-zxS���D7sV�� �BK��:��7���@�c�Y(����H92����Kx�s 9*[�����(�i�K�0%�+��5�A!�B���2��{���c�u #�{���p7�_�� u�8� G7}o���4�g,Tq����9i����G���]�Mm��B��:����h�"BH��= 8�:�g&�w��L���M��-�]
�pp�)�|�k�/L�~��=0.e
���M�3���"��x
�%�M�A~�2"� �'� IDATI!�B��
�2��R������}�G��aw��G����(�����]j����vI�Q�� ��l,?<2�y�9T7��>E�3��������-��/�q���G�KP���A�U� ��K���= 4��r^���!� ]�l��H��s��J!�������������jozb�l���F��X��HB!�B�`�A-���%'�� �]B�P�h+�g�<�, �����hkMO#c6j�6����zJ`z~�K�����*nL���x`� !����s3�D�}S���1�v�x�*r| ���mS�gK1 H� ��aR����W�i��Z�L��=pCHc n��#��� Q������ygr� �B!�B�e����TC@
��������������u�bH��$./����P�I��I��^�VIq ��4'�L�������j��b��6 ���AKR2�DI���������U�4��C:>��� � �eM�u��0��U������4�6yG� �m{��Ki�G*t
s mx���`�������~(B!�B5w8�|���'m������j��B/��:��3�,�?rLZ��W��"�j ����9��jb�2 P��"-s
K���3��-J6��2 1��Z�,Gd �@`b ���+V�����S�;&�N�����C���Q�������7�$� � �!�A��.e v �Xs�����k"�,�K�B!�B��L���(v��@~�,��< E�
`P/�~�l #�;M���FQ�$�n��3r�3r��~e9FU:�����m���L� ��Wf��.*j��Ep�-{ko�-���2� ZpK�_xZ1�>u������q, �6H�����fz�� �g1k�OB!�B��
�2�� ��2 0cya�<�zI*�tv��,����M>��n>���=�� (� 9�9|�&����V���Dy��{��]�,bI5I�`|1�����J/�-�k*lW!���q�Y����eoo���Fu�~���4���2 ���|��2-T�U"��95e �� ��dK�����> �����&%����� B!�B5;�A-\������'�\#
W�n����-~� �3���p���vm���$���d��Q���^B�n$^�w��Q���n{�- Mb���).b1�2�YV&)��[��5 ���<���}��@P���:n�}�J�>���a��H� X�����2����������7a�>���, � �B!�Z<�)�Z�� +�@���M���^ ;�P_���\�� ������!5� ��$L���7z�h�F���??��"]�>�e�DG�"�^R/�������������Y�`?����*��&�� bO������G����������b4=�kgV�z�����^����O�6����DE��~gQn!�B!��7��A-���Jp����*�N��tm��g�| ]i����� t_��
��EI��c�z��u��q 70��
�J>Q�H �z��o�9_�R6�_��c�� W�r{���V,�s
i��X6���C� p�}�t5
�f�m�g������k���NV�������P�P ���}[o�)dr��RI���`�`�B!�j�0(�Z�F%�����r�u ��A���s�$�bE�}�C;�8�����S���
�����A ~�a���~�z�N�|KcL���+W�l� cG��,�����
WZwMl� gh3����1;E���m�i��Br�>����`��#������\l�d�����P=���{���}��: ��! ��/�%��_�'X
pB!�B�%��%��
L��;(���� ���I[+��s�����sZ��B�� Z�-'p h�����g�� ��Rp�o?EE� �W�rd�����[��� ��{.���������\�, ����8U���� )"#�Z�H�w�{�w�L�g�n6
�4���k���J&U \%�v�ia�F0UL�JS&+n�V���{��[���#�c ��S�eY@x��OiL��Y5SN G!�B�3e�5A9[����I H�����L�/M_*�� �3� ���i����^�f�)(&88b�>U &��q����?��s��6����[��3��~������y�{��F���7�w�U���bYu��N^�0��� $_?2_?�h@�q��OV� &.(��� �@N?d����o��# �^�kQ�y���h���b��5�_S~���a&�+�B!�j�����P��g� L���B�\��<���qaN��f�T�t?��:�,W�����X9��= ����Ax��G��?��_�� �����n�w5p�F��X6$�����*w��$��r��xY����UK��F���*]���-��(���7��8�]|�?$xjw�����q�Y�K��p��Qo|M�)/��g� ��B!�B�FE��w,_B��Fc���^ ;�XV I4�S�������c����(sd `�1� ���R�F�E�T�N���h_(s
��Ed��l�]��!Z)� ��/�<Y)���W~�{��}=tz��YD�|�����\Um�|��������#2 @}�!��M�`�B!�B� �/�kB��= �)�P�C&�5%9�#�T�M�s��'(+�$^� ��>���'O����3 �B����f ?.JfH�B��K� /��a!�$� J�.|��K��K����;��2gk��"���� �u�i t�>)��%uER�����j��b�C }��u9��'���`�B!�B- e�5����!��e��e�����JH�����#��e!������>���� ����y�'� 5q�v�G����k���*?_.A� �V��j���E>����u�8&-���J ����$ ��5����u�N�t]�� zKCV����(�t����6�1DT���� L��l�B!��� �2��pQ���D��q.W��5��<�m�_�&��� ��?a�� �q��M&�)��
��.-*� �dz���z
�B
���3��S@������)��Y�b�Lr���gK�4�sK�u=�;5�j�:�E���������?��l,��t5
�go�q �� (]��q9X���<��z������{��%�?�� o�>�bo !�B!���H�s��Df� te�
���}Nt��+��~;�����m�hMl��y>d��sC�Ovo���l67t���K��d�l/v�!���}��S!+g����M ���"e� ]i]�h���{t~ h����4p���mSt�Y��n���a��j~�i��G�_���7�� a�>���{���y���5���/oy���!�5� >1�
�.j��T#�B!�B���� �����%2�W��7>�>��UGL
�b���? z���l]� �������������
+NL�����M �D������X&"s��!�y��� %����3,O�d��U��Vav,��bI�� �" ^��������z��z��]�������w��M�t��q�G ����}d�#s �j�WbuI�/� �B!�PH��� �����o�w������@y��e���<�pV�S����}�y�)��������(�1p����b��y���iDuOekSU}pd�L �8f�A�[���u�� Z� �Tb] [(���) ��i�C�6�1���
��o�c��� ����7Qt6�������8��� x��N=!�B!���Hlt-Zp�+�����G�_X������vW� �C�;�v�PNZ��Q/��I_�5��e111���VL�sBt2��K~��%@A*t��1_Nc #�fi��w�2)�s�L���o���Y���{mC }�c������O�`om��!~������� ����Vl;��2!�B�ke����� _]aQ������>� x ��O�^��|��_�����7o���*Fd �h����f:�RD���b���pUl�U������_:mu���������^���5M�Dg��$Z�R_�xf�}jCq�� �X��.��
�2-�d�
��d�����+s
���Y]y��B!�B��/�kQ\
!�&`����li���`����U�xh���v�#�ejS�����_�������bcc�m7���'�9�M��~�;6�����]�E��So4�j&9^8�<��Ib�� ��^ N p�X�0f�����:rl��n�S Z#W3� �=�)0�nX���#M����< ��#�{���fdG]�S@!�B����2�Z45�v��>"��j^huG�g�rF�������\�k������sZ��36n���W_����J� W���5���-[��U�����g} ��o�� ��i���~]'�+�����^ �3N
(#"����L��Q��&Z@(n�X����|#���>sW��� �� $�6#��p��v�P
��1rEB!�B�`�A��� �z�xo|����*���{������fdG
Z�ln�e8bn� �(9cL���3����F#2G�W�����C �?zty5 ,/,h��U�G���~uyaA��v����������o?e�5�ubE�#M�\j�/3 �5&3�/��(��{�f�h�^?�����l[9hR�~�c��n6 h�������8��
]z<������?=�s�^az���}{�� �o?b���l{�-�a!�B!�RE�D��%2�j!���i R��2������_�Sf���j���[��n�:�aGWZ��/�;j y4<���z&l��h��vq����
n2x,���!a����������)���scwK������H�;������%?����=�k ���0~`E�I�LJh�O(�^����� ��g�Yj�� �Q}�K��������p|������y���g� ��� ����yQ��Z|� |w�C�3z!�B!t�"��;��A-����w��B ��Uw�!��<.��� ?���o��&|D ��� Gd ���~����+�w���av<���,
ydFv������s��h��8a+?�9�D���> Z=����fS��/k����a]��j�� �v���Xu�pfc�" �y4�~��j�;y��Iw ���%��S��O ��w���( � �B!�Z8,_B-���Mr
��<��G���� ~�a�92���R��1 �������{`�p8R� W�<�[��i��X�z�� ,�k ��J�24ab[}g
�q{n���TM
������|�!(�K�R*�J���,����d�g
�d���B+0qe�?�����h��E;_���j�f���� !�B!�Z ��A-�Y�U�"�<�����oK�,��ukRRRS�7�r��^�J��p��> H%��%�3�I�1u����@}��� �����m���~qT��=3�z~`� �)- @�/ ����M���L��i��m\(�j~a�n8 T�mT���E���� K�V^O�y^�� @k�*x ��=���yU�h� ��B!�B- f��+��Zic�bS��|��>�`�����������m�����ix�e�.)"#1�o*�3L��!=;��"�� a8��]T�E��1;��K����_�eNa�A��b7 �^u�@C�<�0�I�&�6T%����jT��][ �"2� ������(�����sr�S����{�� �B!�j10S�X�=O��F�FwN��W��o������Q ���JwO/u�����D�XB�1o���o���*���?gf�aS�4s������[7\q�\�\H���4��p%��b�����L2s�J!�����M��������l����� ��a�a����f����939�������x7��l������J�^�� �x���}�����y�#��A�d0� �[�����];|g�Z�[���K�+� !~�B ��w ��+����[�=�yH�� �_������u)�,d!k|{� Ty��BYH���F��������hT�+�5 j6W,>�^\��3�X}������w�����i�j6���eQ�)�-�:���9�}�|cs��S�_-��1q���b[T�r������8�5���Z-��o�.k�0^�?��x�pL���'�����h�H�����RJ"�������
����;�!�������$�)T*� *��AwI���q�yv)�Ll����� ��s� ���{q���:{�|�J�������f��%K������{�)����.e�;�����]�l�[[ �Y�u���:����=�Ws����^1<�l��}��f��,TI�x�~�����N�0Z�N�
���!�S�/���M�������;���_m�v~������`@���N/���[7t�w�R� �����wj���8�!s�s��� ��s���|� ��B����X���U���Rk��j�qU��M�<T��~�����B�h��U�.�����Y�#�h ������A!�E7O������!����������#C�d�v�y��B��N�W��!����n���������?u����~�%���9I *{ �$��(r/��lCI�[��<]l���-<�$�����|�Itt�����5kV�S��_+>b�N_���+�x5���F�x\74Y�s�n����|[�H�TrI&!��&������L����o�{��|��'��= ��:��4�0� Y~����[����)���)�����zQ�F PY����B��-rR,��X��C��] ���?*�R�y���Kl)�2��(�(�P��Ij5� E��*1��"s��EB!���Y &�8Q���F��,��];L���.�<����AZ!DC!�����_�n������#���fE22��FC�1�XH�K?_������*VFI�� L^ @�GR��_c��S�F���xT��O?�t����n���B�'�|<��B�R[�<������R�[]���wM��(�r��.�V�����O��;}]0��-5��Rs(�rn���k���X������{���<T�TNX����o�Ur��\C=��r���i�p����_��7Q��o-�t,�����MKA�� �)9�� ���� j���7�3������ XtG�'n��q��9��������`Bd���7�JaDK�5��Bh�������:��:CxJj5��2��� ���Y���S�H���_�dYY [-Y,�T�xYabt\Z�V����O���d��l������J��.}�����d!���}S��`����R�/ x����o�������F����s@c�!��L%OT_�� �9�V�� 7BR�$���� ,\�����6S�� ��g5H+2��n�����VO�3�����1cFbb���zJF�T�4���ZY�9�n�FQ"��3�P�0���� �n^}��gr�Q ��_S� �*��];L17#�"�rSu�$��(���t��sy��M�����_VR'u
� ��W��{�Q�$� z���/�Z�F����fO
!~�w-����{��b���7����Q��>^�� �� FGG;;
k\���{q�ZA���~���u���?Y���+o+R�w��M/��RBB����;&.��X��w���/���E$��J�'���=��i��W�&���Q�L�^ ���[X_]G-=p�V�r�~����M ����:K��9����f<��p�|���E�a�J}�^s�U��J=Q�<����oZ���P|���Kzd����o��H����f7�/ �)����H�L�s�� o��h?SR���A�7o�6m���B��cR�4%+��S�������R�32������8k4-��*�c���#��u�Ts���Y�\���#'��2O% ��.������q��5~ML�K ��i\�}�|�� LF @-AR5S�����L�����u���S������ ��z1 *��Q��de�RG����s
�Kj�`������� ������m���R�f���}���
7�� ��n�x���e@�u�+R�t�^2�'���I��~�C��8F��U�W�@� � Xt�_������\�)
;-��] �� )���������e��m����>..����3m,Y/����b��l�<���V(�9�y�9!K��GY�DQ�;)k j������LudJ2�YR�����B��
I�����5���J������z�tJ�F I�9d��z��+ @���S���kNK��Kz���p� IDAT��,��,��v��t@�R����� ��w�u����J���(�����JY�HB�$Q�[m���7����
9�~@���7V)m#�J#�0l}$}���U�1
�J��.H>_�,�#���?o���]m����Fwy�b� ������1���\!�X�����q��}����;w��U�|��(Uk^��j����
����=oh���F�� �X���)�c��
H�Q��b�lUC�l���M[�j�o��� ��; �����wW�������������z��o���� ���i[ZZ����^���g09��'3O�N��&0�.��������f�=`���B����M���Y\��"�`���7<
.�Zj�,ygb��fb�
p� ���j������zj������!��G �x?��9I l\���[j��� ���_"OY�J�( E�\kI���+r�z��B���[�f������B��N��/�=W)�S�s�Rg ��")��.11q����6mz���p�w�QBR[l,�]�Eh�����RS�����(�+��*^����n����+e�����R�)^R���o��B �}�|�\�[�DUN�<�JL? �I�j?����1c�l���G�*7���������eKD���U !��i��I���������������������^��B���\j��G����� U����j��jX���w��+B���m*��*�M��Y ��X���}�F��q��� z�:�����<8�4�C���k��������o�z���������/� !���"��JY[Y�w�0!�����D������B�N�[ /�f����V�E�<�
!n�����[�D����E� �Jo �m1R������ ���_>����l�����]Vi��J*�_��]Vy7{�1�L���.O��ddFO��)���2?��;�C"S;e �����{�Z����B����%�b9�#����}�� Z 2:������5*i�+F���6LT����<*} �)W,>�^\��3������ �a���={:;�Zh���~���9��W�6W ��K>��4����M�����i���Tm %V_ `����cr/�����_~���� ���/z����X g���������W������u���K�L��
$�/�� ���������w ��5f ������%�. x��G��][�22B�-}�jI�1k�!���2/�����A9�~�z����� ��h�%# ����"����6����C ���� ��X�`�e�����t
4�+�`e��$�,D���Od�2��+%��+��K*2�H��
�1@����G� ���?�]1&����+J:|���V�^=x�`���J���'W>�
C��c��J�����Re��f<���<����������4��%��?���[}D������;�l 8�k�xg�j�����Z��"#��%}���4���(�� 1G��_���H��
��+�9' �r�e x.������4>�K�~N�/�d��~o �Z�K����g�R�)�!?���/�19���B�
���\+� @��qv ��%%%������C� ��Uj���q 9I���� �07'i��q"����8�A}�N�%j��R�5���ix�D^���(�yR 9M���O�~��<����[ �vb� j�#G� 0`���<�H���J��Xn�]�~�&��wfC���Jfd� �)���o������>j O���������Uv��� ��[���%�Ll��{���L�|O����� �?�2��� =��_�w�}��G -� +)��JN����T3Bs����a�%/aN��#H�aX`l�� �����r��k5h�iZ�I�UK#[���2 ��� j����0`��eO<�DY���2(�_�h���^�����"[E+�?GQ�s���%����*����6��=Xk> ��v��Z��@ ���� j�'N�����7�|��'� �=X+KBY}��a���{�p9��z.��G I.^��Z����i\�}������;+�E�\��i�{��w� ���nV����)7� A��kZ9 .�W�r/���V-w���^�z���� 3fLO�;�!�������o���6,�[�
�y���<8�X ]��*�cq���<��((������d����d�o@������e�N��CO�u< T�k�xg�j����^�z������;���`�����vY��k*�<4~M�vYe������)��O� Uj5�W��N 9��zFF)�[���\0 p}L_B������g�y��� ;�R'�u6M����n6���`�|I �G�� �-�4�Q=�Ll�-�L,�4���8�"]�$ Y������m��` ��H���8�|��}���;q����[�z.�.��24Z����% �Xpb�����z�,���
j`8[��!��sB������c���g4\�������{ ��uR�*���6��n� ��2�!��? ��s��6V<{��>���KY�iLcO�i:C����7
�X���+�Z��w�Lo���#s^��K��tk��$�r+E9�2~�h4J�����a��J]f��h +*�� �FM�.\�����Y�^x����/�c�f�� EFY�����/����s1�=Q�
r��<�Wqb�����k�m�{;GjP�aE���s� ��\h~L����
�$ I�mhzDe�r*���h�J}^ P�����x�bXX��3&M�d��d�dS������F7 ~�������Z�Vc{bb^�������oE�h��U����vXt�?7���Xb����^�d��,oc��eY��A'�[�j|�z|�tW��*��� ��1} �M�����K�'O��U2{�^bKz���Zk���T�~JE���{�V��iNP�Q��qX��x2g�F�7 &�,���9�0��/s_?��W���g~)
$����\N�� \�)Hv�� �e1Rn,%%%,,l���/��b����;&�Kl���'B��6����B�����0,0���uF!���ayR �cdC��� +��Q�e��z��F��u.����u����41Jj��%u���F��y
�TV�N� �,�2pW��]����3�<Y���Z����#+�{���v�
}����G��no{\E���v5�4%>��,�]�����Xu��~�����6H 7�Mr�
-����K! ���<�f�����������f4��Z�o�k��JfU��y ��b����k�z��9v����g�u������}����O����/ ���G�W���B�XH{DK�5d�
r��5�!%�X5��m��/��()�����Bt���Q�x>�T��Y�
~^ PcH�,��&I<CGKMM
5j��9s��T��
��5_ �n���.�\��/�W����B�������b+i[ZZ.�]�y����:����P�� ���a4��.k�� V�W���2��v��V��!�*�E�F����5 W��?�]1&����k
�����g�'�xb����o-mk�����[4�-K�~����I�V�6�����
�Y ��)��TJ��|,�|8�$D�������VQ �=���w�/����������� �IFF�g�PIe�+��%}��#h�t�����,L#h����\<oR���� J���,�{���:���,�"!nO,�������B���x������) V�����~�z�^��
6�|[���il9E�7�V���\E[���19�4�I�7����!]���Ke� R�L� ��hV���a+�d��0���7NnW�������� �V_�{�y�fxxxxx��� m��_��b�@{���_���wY?ew����)���]�W�*e�+T�{�H )�4P��?�� %�0�����*5������>�hv��Pd��|��K�z� P0Rn����}��y����z�-��\�!��>*��u��Ac%@��:���4��+��R6?+�������/VrL�� �Z�� \]fff��} z���K����R��<�`�����@����kHWJ��-�(�e���$D9��d����r��s���J�k�� �.��� �6c�\�������;�����:fbQ�`�3����5*�����v���6x�b
UIz���x��Z�v�-�Y%iU�,-!D`�@�` �Z��2p]YYY��������e� |i�M,*u�=��P�V_2���a2��������%s2*I�%�7��g(k��(1&H�$& � I�����������t|FF8vb��X$���I/9�I��4���I�{�����������������,����
��#)W���;p����[�\�R��\[�2�����x�wY�+*e���%)��V��h�R��1G��F���|2�a�� ��������t�
j����22
��l�<���oA����I�L��8R�Do���M'�/�\#� @����kQ22��5[�z�J�����z�g��M��$9�G�,���$�aX����#�Y��I�QE� ��F�+�� �I��N7x���M��Y���32�Q�3�I���
}���$%s8��X)�l>0';_������K �����V���2dH���cbb\<##J[������ �_��a��N��wY?ew��DR��O�F���]���j� 5����E-���?b��
|������Vs�����|I s4;Mg9��C%,N� �}��C �F%5�Q?�����ca�zT ����|�����]�~�Z� �M*�<���x�eU�)4���%a>�����Vf6 @m&����nQ>I�V��������a�FC�����K-4���C%�oW���0�/�-]�x��q4 � ����
���+((1b���'�T����B��Hu+3� �6�g0������� �h4dd ��.�2e�b�Q�{G��5 �RRE��X�. �Z��s ��c��T�/���������|��R�� 1G�%Y�f ��W���F�eI.eQlQ��0�� �����a��1YYY6l #����.u���Rv�Q����,�Se� �I8��`����y���m����� NmQr]����)���]���7=�P��K
/���)�: �a @Y���2 O=�Tzz��_MF��J�K-����ju�|���J��nP��� 6 ()�1 ���KKK�����Z��8����Z�Tr�2��b�V-�l�ke�%������)��G������u]���� � $el@���� � ��2�&L�v��#32V���V�`Z����T�������R �$L�V�h��Q�_��B��Rz&Mg(2�i:���>�3��uY ��-\��������N��2kF�"$�gX>Y�������+..���A%~��_m���V-={��{��c�[�j0�������/�h�Zz���pj�u�e�I�)����� P����wF���dY�4i���������aQ#����-X�+dq ]A���AN<��V"#�\�� 5 8 I��,��'O���?������ y����=n��Le]�JYa5��v� �� �H��)S� 9r��QZB�e��)�p�=n���n�����(�<p�\�Q �� ){�ey���� ������w| ����E������/�����,*������%���R������q�G
.�������t�,O�6���~���������0�]�o<�{Co ��F������� `���k��&����^�p`�[p@��Gf�dIKT�QIB%I�z� `�k�xw����k~�N7{�����]�v8;����Ic�R �'8<Tb}�F� ��,n��+�
��s� �L_������ ����9##*�v���%n��+�
.�� ll��9����v��_���cq�
���.�o%oJ! ��a � ")[�;w�� ;��(*8����3�������:�Z$[��+�
.H�� Ps��?��o���� � �K��wM��2e�%��u���7eB~ *�_R�� l��)!!�A����U���
RnJ]bA�@
� ��������{��G� X}��\�������[k����{��w���X��/�-M��&�� ����o����C�9���h���_����:{�����K��� ���]��W,�k�xg��k����|� �Z�{�V����7�� �jdk�nn2�
��������� E1'7d�~�rQ�Q�}�`�s��$�U��q��2��w�yg��5�w������ ���� ��]�vAAA�kY���������Me��+W^z������� ���;?������.\x��N�8 _����(��O�������������������m�v�����_�]S22��\�� ���1GRU�l��U�V�����W Pe!!!;w�T^�X��s�����b���/��r� =$I��������!C����4i�c��� �������l����c��]�jU�V�����/���n''��r�E�-� �~QE�����+v��MF lb��e������EGG����t!!!�$ !���W�\��Y��4�h��w�yG��!���s����4���5���b���#)��X�|�������� ��[�B���FFF�]�V�.������������3r��?��i��u���H�g�����������0��<}���v�A��z`� ��!)�J[�f�{���{���M�:; �Q����� �|����@�]��7�����.]���9x���G�V����\�-��3v�jT��UA�M=U �S��UA����L1��A����,Y��� �CTT���#���u:]FFFHH�"'''33S����q��'�j�j��uK�.UN�$�h4��fHHHRRR���M[����~'B!Z����e�] #eP ��������{����� @M�e��N��=|�pll�����c��O�eYy����n��N�:�����{������,������k��k��ES�L9}���v������s� �:����j$��<�O>�$::�� l����c ���`0���zzz*[222� �,_}����;w��|���3g����Q�FM�0�t��h���RJzyy�����9�q������O�������u��m����9s�4i��C����.W����wW������js�~��� ~���-Z8; *�5�SS���������� `C��(r/��l�����j������m��uv, T�k�x��/���i������ `[L_B�6o�<m�����{����� P�0R���e���S������ �� ���2(���['M�w�}�9; j&W�s�^\�VPu���s�� ;v������ 6��?����b���&L���w��� ��H�������������:urv, �p$epKBB��q�������;;; j>�2B����������?���� P���W���w��m��YUnd��y��7?x��
C������k��Q����� |��� J7q�����U�VU��K�.]�xq���6����3�������W���$��!�]�]�������7z����7?��C�� `G�/�;w��.�����~�|.�F��p.��s�l��4y�b4� �������o���_����c T]�v�������,���7))�M�6����oo�-Z8,�y��s Los N����p}$ej���y��a_~���?��X ��s�N���+:w�l��B,Y���>pdH)�L�-u7 ��#)SK���/C���aCXX��c T��e�� �����m��� ~��}��
6tdH�}����n�%���#p}���� 8��c���������� l�u��B��������k���j�.������O�6��!Ew����iz�����c��cpq���u~���� {��O>�����c �LQQ�����/_ h����cG� 1b��699���������6mj�xF5�(Kb��q�tY�>u����f �zE�CR�v9t���A�bbb ��X �5r�1��� IDAT� :!t:]FFFHH��]�vg��5 �x����{�;#�����fT�-��j��� 8p����
rv, [��e�N����P� >|866��!�\�,�����I�{<��������Z�j��!�� P9���[tt�"+++%%�m��B��]�FEE ! C@@@hh����.C� �={�y#����:u��'5j���K.�5��bL��5?W G� _�b���>��X p4���NM���������{����� �:�)S� ?~|����-{��'� ���LMv�����{���[O>���c ��������o�1j�(g� ,����N�<��w��_}��1�� ���L
�����w�%K�� ;��� �����iN�:��g�9s�DDD8; P&W\�����R���� ���|���� ���~��0R���p�B��=g��EF �GR���p�BXX��3&M���X @�H��/^�������'O���X 6�n�������W������_~y��������O����ax�2�2n/%%%,,l���S�Lqv, ����x����|�,�c��
��cG|||��} V�b�e���������N�wY�����2�����}��y��g###� ����>���z�R����_����4{_��K����i:C�QN�b�f��D^���@�]�v�g�������c ������l��������'�xb����$)���k���ey���IIIm��B�q� ����y;�������nL��n/x�7���s�k�}]7BR�]������s��1] P# >|8555&&������h���o�����������Sy�b����;+�R��������� pF��bKz�-� ������y��'���� �� =���o���
!4��E�~�����te��e�� ������.��s����9����r@���j�-�Kl��H������^�z� 9r����� � ������2��$�W�^III����[ !RSS###��]�V���8w����?�z���� <���V-��j�������a���Q22���7o��c 8��|��J�2nO***?~����K�~�����G����h������{�V����7�� �jdk�nA�)���;�q�F������,X��X ���o,#�rbb��1cLDEE� 9�C�B�N�����:~��3�<�����G�����1}�m��y�o��aaa��� p)~~~�&M��tB��0o�� =z4h�@��e��N��=|�pll�����c�'O��q�)#��7� 9r��w K�,�� ��I�#��������;��c�k \����W�X���N�:}���z����h���%�
C@@@hh����rJFF��!Cf��-��5k�/��R�~}S�g��y��w����r;N�� ����1�|����}���������k� P#�fR��K�.33�_�~]�v%# @MBR��eee���?���dd �a\q��{�������n�z��U%W> ��%T�N�4hP�V��� P#��qEJF�Y�fdd ��H�� �N7x���M��Y�F�� �f�7�k���2dH�&Mbbb�� P��b��b�ZA���C�������U��6i P�� 6���o���dd ��H��%#�����h� �;�2�WPP0b��V��_�� �� )�d����?��F�!# �a����m���:�������uv8.�������
R22�$m�������� �Dyg6d xV.�U�J �uCW{7 ��`\���+��^��� �1c�dffn�������� j���OGFF���h4���{��Wo���3f�B�k�.((H9R���{�&%%�i���!�mmQ�u�|���E�a�v�hY\3)�|�0 c���y���_MF P W�\y�����]��qc���d�����/]�t����!!!;w�T^�X��s�����!�rS,��.������2N`0�z���� �� �o��E���~����*�j�����53?`��e������EGG; *�Oc�-�!��!)�h�a��qiii_��V�uv8 �w����m���J����Z�n-�HMM���\�v�Z�v@T~��%��Qy���_�����/9��h ?~��k��o�NF `yyy[���-�� ?~�����������(!�9��t��>�~ {7{�1�v$e �h4N�0���+�|������� ���7?x�`hh���h4n����jLTT���#;t� ���t!!v�L��l�w������b�����<i��3g�l���� �����?s��s�� ! �+�����g~��-[t:]DD�����������\qE(�R�U�dY�<y� �g�1 ����������|�F3l�0Y������� C@@@hh����rpFF��!Cf����� �5��v���K���,�S�L9r�H\\����� �ZK�.���������@\�k&e��d_�,����IIIdd �����c��.]�8;��B�v$���i� : OF `W'N�|��B�����?u��qvD(�+��q/VF@���+�v���kW@@��� &L_�]^}�������D22@��p�Bg� ���@�A�\��(r/�&�������~������;%* ����u 6Ggj�;P{�fg�����;7..n��]dd @Y(�kc�������~�����@g� \#eli���|���]������f�:7`\����IkUn�
'�������������W����@w5n�x��n��^c���������p��W�r���T9Knnn\\��-[Z�h�h��
�e�� ���_�����;��g���5g�Y��� pu�^�&�U��*�X�S�� �\n��jv�y��o�:-������������� � ���p��U�;��/�v����Y��f��-�,O�6��� ���������o����
~��G22 �"�)s�$I111B����W��w�y'&&f��=w�u� B 5#el`��5dd @�0R�N�8��(��$I��r���n�K���*7R�+{��}���� W���6o��
V�{�w���:��=1������n��jvg��%�� ����� )S].X( �>�/ 8I ' ) �$e ��B����1c�dee����?���SB�����]�:;4 PI, �xL_ p�2 N@R � H� 8I ' ) �$e ��� ��� p�2 N@R�6rss7o����O��7��� ��r;��-[���z�����/88< �Q�N�\�];8� p"'�n��}�26 ���i�.\�0n����lg������������MHH��� �w�>u�T� �j������+�k��N���m��/�v����i��9;
vQV :thJJ�������\���� �@�:8� p ���t�8�w���3R �%???88���w��� rb< l�z��n�: ���2��n{�}�2 Pu�������[�4ir��yg����wp�?��������+�_��S�') UWXXh��Y�����;+ 6d�����V�N�\��z���>I �:�?�z����5 7e�����V�N�\��z���>I �://���B�-)))w�}���`C�;8� pk���t����w������j�h4W�\1�MHH�����`C�;8� pk���t����w������j�1c���S
�b���w�ygPP���`�;8� pk���t����w���k�y>�F�q��1YYY999���?u��"**�k���
@u��� z���W����W�V�h���>pj� *�� ���2�vp�?�DN������,��o ���% ' ) �$e ��� ��� p�2 N@R � H� 8I ' ) �$e ��� ��� p�2 N@R � H� 8I ' ) �$e ��� ��� p�2 7�n�������W���K��o��`0Xl_�zuxxx�n�f��U�KT���,�>���� &����|��m��i���\����w�}wxx�� ~X�s�����/ <x�_�~�>�h�?~�x�����W ��H� �LDD�{��W�v��m�RY��p�������V�*��7�|������.*��O��O>i����Z�����g��s������N �=�X||��)S*{b����s]_hh�� ;*~|�6m������ j6�� �9 0`�������W�_�U�.Jr��,,,���������[�[�n��}�������;����� �� ��_�u��e���EEEO<�����%I�p����?���r��Q!���7#""N�>�d��\�+Vl��=;;�K�.K�.���.\��� /�8q">>�^�z����������-X���Y������;w��7/33377w��9�{��`�EEEK�.��o��h4��=�XNN�i�DNN��y����/!�������;t� �X�p���~���_��S��O?}����=��O�E��[����>4hP���������<|��g�yF�$!DFF��O?}����^{�m�����/^�j�����|N�:5��������:u�DEE������\���K/%''���t��������^��`Y�[��l�e��N�:5l�Py[��B�>}:222''G������^�z��/����1cF��k��]PP��Z���{�&%%�i��t� /�0r�H&e�7�c���*r��O����j}AT�����y��u��m���=��1 P& ws����C�����999�,��3��7�0 ��_?����Y�m�6�F�����3g�u���.X����� ���CCC'O����'����7{�����Z�6�z����X��(�����u�������^�z%$$(o�]�6p��_~�Ey��G��K�������y�������h 4h��s�dY>p�@hh�� ��\TT4�� �_��>x��7{�������|��� .����� v��Y��K�.
8�������A����W�9++�s��� ��^+O���V�9�L�2����3�b�Y]�|�W�^��1 �����w��i�����,����7��������cq��`���NOO���(����i��Yl���(7����������r_��~7��� ~��'N(o�� ��#�T$�
>
@��~�x B =z�������Bh4�E����O�����������}�]�V+��[����C����U���q� �F�Re���3f���+{W�\��O �m�F���[g����?s�LI��v�� ��$�� �r��r��7V�X�<g�Z�p�����W�^5���?������m+�h��}�������_�vm��M�c�����[�d��Y:�.$$D���������5��^+�����s6w����-[Zl���-Z����+OF�R��;����X�l������qqq����T�-Z�8q��7bE��
����[�^U�/���E����;�Z�R�����|x��� ���K ���_?///�[I�z�������W/g���o����b���Wll���_��k�ZOOO�[??�
�9p�@�����H�d�����5k����_����NN����ir�$I��i�(�7�J��~�����o�6m|���M�kL� ;6y�d��'O�4�^�zutt���������G�Q�F��S��{��~��������CBB,6ZyV/^T�V�] �u�[�B���FFFn��M�V�<&$$����]�t���������X������������g;v�h~����+3 V�� ���?U*�s���={�E� ������~�,��yV:���e0�I�?�������������y���}������qqq�^�bE�v�*���j�o5i����K�XyVyyy�*� ***?~����K= %%�4~��Y������rss-��
K�3 V0} ������������h�_�����,������$�F[�f�;wNHH������8�w���7�������q�F��u+��� ;rrrLo
���o1d�T�:u��fpNN��K����7�8q��%K��[W���U�~+�M�6��~V�U��� <h�e4�o�^�+FEE� 9R�J���RRR������O�� ����� ��������BHHHRR������
� �$e n���o��I����0o�� =z4h�@�[�N�_�U��v����[]��{����S~������]�v�����������y�f��������7n!���O?�t���������c���;����h��������B���K�=����REFF����{��U�������O�:Uy{�������g���������SE�ZW���������Dc�Y��?������B �W^y�"3�L�l����"""��� ���5?����j��Y��/c������__X�h��)SN�>����������� j9��$ ����_�b�����w�����?���EEE#F�P��V�INN�<y�F��e�������Y�r�� ������~S��fee���(�>�v�%���Wa4���������BBBf���p��J��G���gO� =��[����o����'{zz*1���^�_�h��C�$I����;w�1c� w�������;w��{���c���J���^{��? 2dHLL��s� ~�����3f��� ����|���]�:u���G ���������G+�~����V�:{�����r������|sss,Xp��)�����u�=�L�2�I�&B�#G����S���W��9sF���5j�� �������u���&m��=x��)x��A+�J��� �����h�
&����������#��j*w���1d���r/������?n����������\��T{ET��q���*�#�{�����JT�/T��q���W_}5;;�n���Z��3gN�&M z���������z��c��}����]> P��� �S
6 fR3���[F������Z�t�}����_��G�����k�C������I�I @IL_ ��^x!99�j��������V+%)��*gd��o 4h�G }d��`������4i�� � F� �R(���_���e��C�����
>��'^�|Y!I��o�i�H6 ��H� 8�� ��� ��� p�2 N�� ��=�*3 IEND�B`�data-unsorted-exceeds-ram.pngimage/png; name=data-unsorted-exceeds-ram.pngDownload
�PNG
IHDR � � �W�3 bKGD � � ����� IDATx���w|U��?�H�V�����*�j� @H�� �V��,�,
"*�B���hA���`�o-���"VTd8@e�J �� �M� !@�s���q�9��O���/�'�yn\4� P�� �4R� @) � @ �2 P� @) � @ �2 P� @) � @ �2 P� @) � @ �2 P� @) � @ �2 P� @) � @ �2 P� @) � @ �2 �p{�����{^>h������YYYy<���{�
n����%o������^�)�������]w�O<Q {;��N�Z�n]�>}^�k����'�h���k(��� P�����O�>+W�,W�� '���y�O<���o^�~����������U+{�}���u�Yc���D" ;v\�v��%K������+O�8�l�������H$�w��~��-Y��l��������_�fM���#��M7�T�b�X�
6��'?�D" ���y�.���#�Dy�9���G ]�|y\\\�J����{�WD"�O>��u����m��� _~�� ��G�=���� ���������Nz��?���O>����N�D"���{���V�X��#�t����2���K��A.������2e����u�]����6k�,{IRRR����4iR�|�# �l�<G�Hd��%[�l9xm4���{JJJ���;��G�~��G~��_L�6-{��S��i������5k�Q�,�F� �����������+ �D���;h���?���s��9s� /���_�k����^�Y�f��=�x��SN9e��;v���w0�������i���E]4w��9����������W��
J��>���!C���o��� .���+��2���F�t�r����k�n�����< ��( E�m��'N����o��]�e�m��A���n�z��w�\���z�=������
������{������ '���� �q�.]b������5k������={�$''O�6����h��/��_�����ck,X0p���e^�xq�z��v���v��=M�6=�����_|������� ��}�n����K |���^z�v�� E����\�m������K�� �}$4h��s����g?�q���;w^�x�1�,�5jt�� =z�Z�*��\gp���3f��~���o ��������������-���w�1{���B� *��e�����l��������K'�t���[sm�������, ���� �N4]�f� w�{x�i�=���'�|���/W���g�yl�����>�l�V���J(_�|��=���{��[���~� w�u�YG|���<x��{���I�&qqq�H�����4iRzz��}���#��S��o�����^���S�L�U���~��� � z��7�����o{�~6n���e����7m��O�>�6x���}����G�������O:��� |�����G<��:uz���c�^�re���O9��#�zzz�������/��S'--��o�m���u�]�� ��k��l�255�^�zM�6]�dI�=|��Ww�y�UW]�����q�� |��M�\;a���-[6j���ol�����ks���O?m��i�
RSS ���/���?����?U *U��s��H$�����~����7n�V�Z~��K �����Q=�����-Z4j��~����0{���;�v�z��7�|����v��e�r>�p����w_��5k�LOO�-Y�j��7�|���O�>=�0���OMM ;v�C=t���5+�6�;Gs���]�������i����G"�?���u���Z�j�ok�g���O?�tjjj��
4h0l��A��3U ��������������{�}�����sO�z�7n�m��#��H$1b���?��������n����Q�*W���'�D��&M�T�Vm���\{<g!{-Z��_�~�������-[����?������]��Q����m��5i����.�:uj�s�8Gy ��y�c�vB ��/��-[� nm�_���������
�3R�����Wo�����������N���������r����ol��G����o��
M�4��0�H��g�����!C���?|���s-��eK��uW�^ {8{������;���U�b����.��=���o��S�L���o��y ���5k��W_E��������>��_���_4���c��=��3�������[�.��Y�&11���?�m�i������� �OOKK��_�;A;v��]���e���v�����}������7�����/�LNN^�~}4���z����9���_=��8S��K�.�A1-[�l��I,������A>G� ��=�s��x���o��S�N;w��F���������O? [������<{����o��&55u��E��y�������u����x�����t����y�� ;v�����x����c��x���m;n�������?h���/�8?��>V1/����RRR>���h4���������)S���)�O�����O?=�dK���c> _}��
7����+��������~���0��y�����~�s=��<�w @qd� @�z�������-Z����;v�����/�LKKKKK{��� ������=M�����w�=������O>x��+���_��W>w^�z����~��)� o���^xa��������>{�E� ��W������M�2e.�����W m����h4���_� 9r��! ;v���H>��C�����7o�|����3R�*U ~�����n�!--�v��qqq�\s�W_}�D }���_~��s��m\�j�W_}����O��{w���cg�J�*��
;��������.� �s��� 8p {U�~������>;��)S�w��9�x���P�Z�M�6���?##���o�3gN�
����� F��s�����?�pV�X1p��J�*E"�����L�}�]$6lX�6m������?�������_��������rJ���sNX3z��_��WG��p�8GU�T9���>�����'M��=�0���x��3�8�e���=�-[�[�n7�|s~"�}�����~��Wb�b��/��y� =�����_��{���f��
��r���\�r��%��r����C g��� 2dHl��H$�����m�|�����1�<����'���������_�~�������dee
><���j�����N$i��U~��F#�)A����-�������333����]����6j�(�y��/��^�z�����������K7n|��y���?����������}��������� ~��A�Y�f�&3���LLL|��'�����_�?Rv7�=%p���+W>�������'�d�{��}�����/O=��_���-[�<����v����F���[\�b�U�Ve���aC�^)..�a����3U �V��i������k��W���?��}���~���33d��\O?�������A�9�s���KNN^�|yrr��� G� �s�SO=�J�*;w��\����=�����XO�����+�8��n���9��o~3|��_|1��Y��Z�j��)r���x���o�H$��4 N��*{��O>Y�j�����Q�J�?��O X�t����>`����'�z�����g�q�g �����,�[��v��97���������g�G�
�9��,�w @8)e ����{��-���P�F�v���k����>��������\[������N�D"����5�\s� N�6m���w�uW��MHH���(�o��c�U�T��d�kH���Kc]FBBB�v&333!!!�~��/��[���z�W�^���F��6m:��-��kW�%���g����>��(��U���93������{��;wn���#�8�1qqq-[� 8p�� �C:��s�N�z�`�l5k�|��W"���
f����q��C�f �<�����������S�/�sg��gO��9x�8S��j�����?��Y��Y��U�~��7��m�g���!w~���� <��L�2��z?�m�s���U�V=��������]�O�SAM�9�9�����w��k��J�*� =:g�������8��6r�c�����v^�L�Z�j��Uk�����M�U��<��!E>w{����� ��{����� �������^{-��3�8��S���'?������ �c�{����w����i�?����6����cs� ������8q����v��1g��o�1�\r�%K�.�������e�]v��4j�h��U�}��1$���K� m �W��|���Kb�9 8p��O?�����!OL�9:���+���+O?�t�{#�#��}��+W�<������U��w�=���;wf'<���������������Y;a���;nV�f������ 8�� ��'U�b�/}����������y��c��7�����o��{H3g��YZE��9s�����x��o��V���o��u���7������{�A��]����O�5av����r���S~��]w�5a��}���\�2�p���~���g���k��������*�cu<����=ot��u��3wN[�n������=��p�y�����9��u��}�r��9}{��(?��]P )e �NVV����o���sg��������~������G�� c�;�F��'O������R
v�?���Q���5�������;�����+�DN;���N;m��q�����8p���t��SO����?��7�x����/������c���~�i���� ���������?>����a E��+Wn��A ;v�]��}$���� ���G�w�����?�}�B4�5kV���cW�\��������W_}�V�Z�Y{���f�.;v�x���?�����}��G ~��u��E"����G y$�� ��*oU�V���O;��X-R�~�1c� �W/�-��=N�+W�������#�HVVV�>}~��_�f~�������`����[�li��u���c��n��� ��-��C=��CG����.�1cF�'��� �'
��9j�������y��������>���� }��������3g��7��u��#���X ����%K��n�z���o��v����.]������}�.����.�,Wmt���|��� ~������}���#s��'�Z�����_~y��Q���>G�q��:��(�@�7 @~��7�B�
&L���ov���o��{���y���Hd��� ;v\�x���^��)�������#�H�.]>���\w4l���K�.IIIy?7�D���^zi��iYYY���7�tS����o;Z�h��O>�s��/�����.���+���p����={�����G ���U�X�o��9�ddd���o���e��9����� ��w��u�[���y�%�\���/���{���j�����|f������K������Q#��?����}�F"� ;vl��16w�
7���g���~�i� =~����N:���.���W�5���3n��X ��S�����]�v�<D���o�z��t�r��w�uW��"�;w���O�� {��� �H>���/�����^{��7�&���iS� >���3�<s���5j�8\�� ���c�&MF��n���u�>���Z�z��'f�� {��]� {���k� 8p�|����w^�� c�����O� �����>��wo��-�������^��w�u��q�� e��9��������_�*� ���~�D��Y����.###>>>��[�+Wn��Y������N>��a���n ��g�i��6a��&M�����e�����w�u�
2�f��}���2eJ����������,�������O>��������� ;v���
6�U���������������o�>{(�?���w�X������w���� �� �D����o��a&L��1�9p�@��=g��]�Z����W�^��� �M���st�����={����
�<�~$��wo�S%..n����k����[�|+y���X����������O8����~��� <y���[��L�2���g����o��n�)� 8p�U�V;v���4�D"={������ �F����.]:s���/HNy�=���~��G yd���e��=��s�z��V�Ze���5k |�����h4��m��;w6l���� y��| �c{�GJ J�g�y����6��`;v�����.]�k�
�j��AW\qE�
�R����y������Z�j��a��x8l�����Y��R��y����~(� x�� ?w����{kd���-[�}�����/� ��?���g���6^�lYl|���h�"�C�J�
*�"��8�]P�) ��}�����_F"�������l
[������~����v��S����C�E0����_��WK�,�uk[���J'� @ |� @ �2 P� @) � @ �2 P� @) � @ �2 P� @) � @ �2 P� @) � @ �2 c)�k����'�h��O�>��`��)���III�:u���,�� E#t�L4���������k��? ���� ��7{������x���;w.�� E&t�L\\��Q��u�v�9� r������ �H��;����/����Y PdBW� QFFF�j�� ���,]��@� ��� ����r���\R�F�/������!!!a����� T4
:Bn�����o_�b�B�
��6�����#..��������!gQf(��*�=���s'axop�Ru^��xT ���7 �%R�:/���
CN ����c���I��v����_ g��P�n_JHH�U����7�j9�� @ ����w��u���A9�bV��/_~��}9�l��1{J��Y �$�'O~���f��u�e�����Y)�D����������g����+ d-��c�=t��S\~�0�,� ��Z���[N.�T��b�� �i!�-�3Ka(U�����a������v�;�<yr���g��u��� ����o�;�U�V
6l��!�����{/���� ��-��o��9s����p�pS"@����8� 8*S�L�����Y�������p~��n���j�j�� ;w����/��]�Dz��y�
7�6�S���_]�~��e�^p�/��B�� �Z ��{��7:t�0c���F&��X�M8�4�b�)�q�A
�Oo������?}�����:��p~��9e 6}�� x��w������R�B�)� �5 � G4c��{����w��U�V�Y�+��w�&�c� ���3g�k������]��!7����2 @1kd�z���52��� ��Y�f�k�n��i�\sM�Y��R (�f����m�i��]{��Ag9J ���3gN�6m�N�ZL�H$ t ��3g���-[N�2���� :��3R (N�{��V�ZM�<�N�:Ag9.J ����_���y��'�t�MAg9^J �xX�pa�f�&L�P�n��� � P ,\��i��&LHLL :K�P� a�����i�����/1�LD)�Oq���� t (]-Zt������kIII�Vzzz��|�e; q�h4� a�( @0-Zt�m��7.%%��w�K��� ����o���1c� O#Zn_ �h��%��v��1cRSS��R(�2 @�,]�4--m��Q%���(e ��Y�lYjj��#�����R��2 @�,[��Q�F��o��q�Y
�R ������&M����)e �PX�|y��
�
V��R �>��Q�FC����[��RD�� �v+V�h����!Cn����� #e � �X��~��C� ������R��2 @`V�Z��Q�����F&�� ��j������h�"�,P� X�zu�z��~�����;�,�P� Em���)))O=�T�V���� P�>������'�|�472� P���Y��������[�%`J ���Y�&))�w��m�� :K��2 @QX�vm��������
q�h4� a�( �q���/������E�����7R (\�F�{���42���
�������~���>��Ag �L���Gzzz�Y ��X�~}bbb�n�:t�P�/����}9_�/�Oa��*l�y� ��BF IDAT ���
�����k�� ;�$���F� o��
���=�P��Lh)e ��q�������;w��)�,���
�����_w��9�,��[��&�7� @}���IIIm����� t��
����2 @����o���Z�n �F&��2 @�52�Z����G�Y�� p�6o������e��={���P� �%���h��W�^Ag)N�2 ����ysrr��w���w���3J �m��%99�y��}�� :K��� �E��i�����>t�b)�_� 6��2s ���[SRRRRR t�# ����2 ����uk�z�����E#ZJ �(l���~������<�L�Y�7� �_��m�W��M7�4p����{J _bcd���3h�����J ���o���A�o���g�
:K �� �`����������{����� J /�12�_������R���k��&�_f E`��
4��O:l������� �p^��1S���� @a��kW�F�.�������F&�K{�/ ��k���������R �m���iii^xa��k)��2 ���525k� >|x�2������K�����
Q��9���F�Q���������� Z�� �p� n����7>��sF�U��\�yi_r�/ p<�����I�5j��F&� b ���L���G� ��)a �6� � %##�i���+W 7n\��e��S��yi�� �R-33�i���*U;vl�ldBK) �W��9���� t��E) �Tfff�f�*T���k�id��R J��������/_~����@(e ���52 ��H) ���}�����������'$$ ��R� @)kd��)3a��L��P ��"++�M�6���o���F&pJ (���Z�j�}��7�|�|��A�A) �@VVV����m�6u�T�LH�S J��]K[�n5F&T�� ��,++�m���}����S+T�t �K) %VVVV�v��l�2m�4�L�(e �d���j���7�|�� 's� @ t���{�������6mZ�����!) %M��Y�n����52��� �%�v�����>�>}z�J����a)e ���52 }���3*W� t �bN (!����>�r�J�L�`� ��h�c��+V���J (���h�N��/_>s��*U� �|q� o�h�s����-��/F� @1�F�t��d��L��� ��*�>��C�-�3g��'�t ��R �� =z,\�p�����rJ�Y8jJ (�z��1{�l�L�e��|�������� @�g���f��;w����t��JOO���:���E���3�]\�� @����k���s��=�����R<����H (Nz����)�) �F�>}�y��y��idJ � �>��[o���)1�� ��c�=6m��y���~��Ag�`) a���>i�� hdJ#e ��y���'.X���3� : �H �����K��H �A�� =z��g�uV�Y(xF� @=���#G����`F� @�<��s#F�X�`A��U��Ba1R �e�����
���xF� @� 2d��� ,�V�Z�Y(\F� @X���� ���)%�� �P1b��!C,Xp�������� ��� 9�����?�F��p� l������_�`�y��t���2 �Q�F���������H �������7����??�,5#e /�����?>���5k�� (e /��r��}�����)��� Em��1�{��;w�\t�� �"��+����s��y?��O��B����h��..�Q �`L�8�[�ns�����K��R������2 PD&M���!�R ���I��v�:{�l� 1J (t���z��]g��u�e����0�/ ���'w��e����_~y�Y#e �M�2�s���f������2 PX�x�� :��1��+�: �c� �7�|�� �>}�UW]t�(�_� 6��2s �l�����s�;��S�V����K{#e ����1C#�)e � ��9�}��o���F��)e ����9�]�vo��V�����B�)e�%�?����� @H��5+��\s�5Ag!����}9 t�C �<7a��� ���g�i�f��i�^{m�Y�-���F� ���3gN�6m�N���!��� ���9sZ�l9e�����.�,'F� ��{���Z�j5y��:u���bF) ����k���'N�x�M7���G) �b�����5�0aB��u��B��� ���p���M�N�0!111�,WJ 8:����6m:~�x� �C) Ga��E��~�k�����t�7� ���E�n���q��%''��b/>� P<,^���[n3fLJJJ�Y( �� �#[�d�m��6f�������PB(e ��.]���:j�(� H) y�52#G�LKK : %�R k��e�����o��q�Y(i�2 ph��/o�������4itJ � ����6l8l�0� �D) �}���5:t�-��tJ��� @��X��A�C� ���[��BIf� ���+���?d�����+�,�pJ ��U�V5j�h�����R "�Hd��U)))�<�L�-��B��� ��������7`���-[���B) @i�z�������z����: ��R �R��O>IIIy��'[�jtJ� ���5kRRR�����u���P�(e (���Y�����_�6m����H) @i�v�����^�z�m�6�,�Rq�h4� a�( �(_|�Ebb��~�����?�,�p^�) @�kd�w���!XJ J����'%%����}����Bi�� ��X�~}bbb�n�:t�tP� P:���]�>���Ag�HD) @i�a������ z�c��Ag�S� P�m��!11�s���:u
: ��R ��l��������{o������#�_� 6��2s �����NLLl���������p^�) @���7�$%%�i�F#C8)e (�b�L�V�z��t84� %�������Z�l��g����a)e (Qb�L�-z��t��R ��c������w�yg�����G�� ����eKrrr������t82�L���Gzzz�Y 8�X#��Y�G }4�,�Bzzz��|�Y-�_� 6��2s �}���III
6 0`@�Y�p^��1S���� �u��z��%%%=��3Ag!��yi��% ��m����_?11Q#C��� ����m[�z�n�������R �b)6F�N�:�
: � �����4hp��7>���Ag�c�� ����}{���o�����{.�,p��2 '�12�_�������%��6���, �Rh��
4��O:l�������Pl���>���&�g ����kW�F�.������kd8*���w� ���]�RSS52�$J �n��]iii^x���(I�2 ��������j��9|��2e\�Rrx7 ^�F���� 1b�F�� ����{w����=���#Gjd(y�� �={�4i��F��F���P"y[ :�F�z���G���PR��k��&�_f PRedd4m��r�����+[�l�q( �yi�n D233�6mZ�R��c�jd(��2 �Ev#3n�������@�R�
�����5�X��F�RB) @�233�7o^�|��^{M#C)�� `�F&!!a���J� A��o� w� ����Q� �X#S�L� &$$$ �� �`dee�i�&33��7���P
)e @VVV�V��o����o�/_>�8 � E-++�u����m�:u�F�R��2 ��]K[�n5F�R�H �NVVV��m�����S�V�P!�8$� E$++�]�v[�l�6m�F�2 ���������7�1� ������s��_=m���+ B�H
W��Y�n����52�M) @!�F� :t�����O�^�R���@�(e (,�F���>�1cF������bN
E4}��W�\���C2R ���F;v��b�
� �R ��F;u��|���3gV�R%�8Rn_ � E����;/[�L#y3R ��F�t��d�� �R ���F z��E���3��O :��R ����G�� ��;��SN : �� � ���c��9s����@>) �������Y����{�����
� ��W�^3f��7o�F��R �����{'���u��S*����J��Z�^�w����O7F�A\4
:C���9J @��m�� ��7��/���{E��n/��O���~{��y��vZ�� �p^��� �$��fWv#�D�fE__��P_��G }���������c�� ( ����k�w-)@�=���i����w�����@��� ( N�X6��SZRP���'M�4k�,� � @I���J�r��Q.���J��B�<����,Xp��g����P� �?~/��?�P~sV������.��?�KF�����_z�����kd���q��� �� 9������f�\r��Z��� _b��A�F�Z�`�Yg�U���"�K��� P vn��{���K����>;r�H� �/ ���N���z�%�����1b�� �V�ZP��2�� ���Ag 8�k��_���x���\���������
6�|� �Kzzz��|�Y-��T�M8o< ���q���{��M�*UK��� _t�����!C����� ,�V����
��K�0f
�p�9 ����?�a��! ,8��s���%���n_ �F����B�� ��#G>��S�����@���� ���#G�������w^�Y�$3R ��5jT������������ (!�|6n����~�������l�1�a���O<������?��������&�S4 ���������+�0.��I���X�e����K/���w���5k�,���p^��1S���� ���� ���4���*����|>���_NOO��PR����D� %��]��<#���q��E32���t��
�|��1cz��=o�<� %� @IP��%e�;9�l�H$W�B����Y�5?O|��Wz��9o��K.���3��D� %AB�sc�LL\�� ���w���={��;w�O~���L �R �D��8h����1i��n����3��K/-�T��)e J���?:h��yl?i���]���=[#AQ� �Nj�����q+���p����]�v�5k�e�]V��C1�/ @I�P1��ht�/G���+{z�*�*& r���'w��e����_~y�r
��t�M8�� �L�2�S�N�f����+��E'���F� ���l�=c�� {��X�����}u�\����0Ea�:
��K7���<c��/^�'��t��[^U={�7�x�����1c��W_ PFL8/�M� Pt}���F&�d��v{gE�����w��a���� @I�y�������c����s�;��S�V�"� �R �D(���%�Hd��������526J ����5�������2Y?��e���������j�� \4���2 %�s��-�������h$ao��?of�v�����[�\sM���C���a�)� r����Kfl���� '�Y����<2m��k��6�\�p^��1S���� ��9sZ�n=u�����.�,
����: i��9-[��2e�FB��2 %�{����U���'��S'�,�(e J�������7�8q�M7�t���2 %��� �5k6q���u���� @��p���M�N�0A#��R �x������i����'&&�8
J �bl��E��~�k�����t��(e ��E��v�m���KNN: p��� ��X�x�-��2f������� ��H �����{�&M�����t�)e ���K��z��/���F�5� @q�t������#G����8.J �bc��e�����o��q�Y���� ( �/_��Q����7i�$�,@P� ��/o����a�42Pb(e ���>h�����Co����� &>� ���>h����!Cn����� I) Pt�-��{��M;�V;��
/���jyo�b��X#s�]wMB���E���3�]\�� ��K7�;��]�c+�� ����WU?���V�JII8p`�-�*#�L���7� @�3kuv#�Dve�l���m kd�y�� �TJ �"�q��\K6 �$f������0`@��- ? � @9�l�\K�8hI$Y�zuJJ�SO=u��wI. J �"�b���dd?<�LF�������O>IIIy��'[�jU������� ���~��z|����z��g���c���H���
��Y��������[�(a�|8l�9E3 P�l|�|�2�9�d (W����Y�fMrrr�~���mD:(��yi��% �"�=�Z�%;����v�����^�zid��P� ����g�U�~�Wiz�G"��_|Q�~�^�z��7� . P��2 Ed��wL�<��25���/Scb���Oi��_$&&v������: P��xKU����3 ��Y�i����������q�����m7=��� :t6�l���7R ��l����;g�:�=9������uo�V��u��@���(l�Y� ��������r.������z*�<Pz����H �"�q��\Kv$��*(2J �"r� '�ZR���I�A|� J��W���v��jn8����2~vv� �:�0�R6��� (v�|���=<��� ���;���&TL4�
���w� @Q���o�l ���D"��H������R ��}��7IIIg�Q.��h��@� a�� (\�F�e�� ����*����D�@) P�6o������E�^�zU8�m$����U�pR�����K �e���IIIw�yg���#�HB�����?� ��6�����O��tF 0a�|8l�9E3 r[�lIJJj�����>t(��yi��% ���e�����f��id�� cQ6��� �����o���6l8`���� �HX/���)l�y� �p��uk�z�����y���� ��K{�/ �m����_?11Q# �R �`l���^�z7�t�����J �#S�N�A��( �2 �k���
4����}���� ��R ��l���~��7�p�s�=t�8Q� ������~���Ag��0~#T���{� ������A�?��O�
t ���yi �La�3 k��]�5����� ���� ����� ���]�RSS52��P��K����� ��]����.��Bw-A����g_�����8z'l�9� ����SSSk��9b��2e����p^�� ����w��������� ����w7n���s� 9r�F8~>G �,����Qc��Q�@�( 8�={�4i����� =z�F((a��&l�9 P4222n���*U��7�l��A��E8/�U� ������i���+� ;V#,� ����J�*�7.>>>�8@I�� 8����f��U�XQ#� @n�����7/_��k����
�R ���������kd���� ��}���q� �����)e �-���-[v�� A�J8�/ @$����������������2����M���[��V�Z�� Vp�"�
����D@��n�UP�(���-�*�BX ���g���cp���Lfn�������;��� �~8�s��z�w�y���z����� �2 ��UuJ IDAT^��#���>�����8 Z�/ h��^� w�AE@��R @����TRRBE@�Q� �zy����������?4�����u�( ���z�w�}wAA���+�� ?�2 Z��]�S�|�[�H�1�����_��n��� 9r������ ��iZ�3����� �f`���{������_�L�W�����:�����F� ����r�J���� "@����� ����n_�"#��rz���Ww������{���?��*2 "��K Z�#EUB��#?U��4����?t�����m6[�@0V� h!�
�uF�j_�+2�|���������
�AQ @ 1��2��\Z���o.5M{������f�*2 ��� �b�����Eij�*=ij���E���oi��������52 ���2 Z�
�&
��pK���K�����w�O�/��r���v�=��$V� h!v�nY=�XI�J�XI_=g��fM� z�!*2 tH��t��>3 �����Uyk�����3v���v�����Hq�|�g� ��Cl�Ws.��������s' �DO -�+?p���*�
�b��U|]���������� �'�2 Z�����������&w�S��?y��m|||�s@�(� h!���x����'�H���:��Jg_ ���>7z��n@ ����j�E�G����+"��~����F� Z���3R �$ �e �%e�/8���$ �e �O>���� 5a>9$-����K ?�F� ��i���\�r��M6�������P�����_���i������� ����+Vl��%)))�Y ��>_��� ��={6 ��� 4Ws��y����� h�X) �Y�;w�k���e�����Hg�_��2 ��y��-^��� �f�� ����;e�����������uj���?�����e������K MM����F�-� ��k���{������_�L�W�����:4�� ����[�lIMmD @+��W{z� ����*2B�*�g��}
yp���Td �l_ n9����#uF�Z�p�s�=GE@��J ��g�Pg$�s�=�p��-[�t����b@xQ� nO��<����}��8����a����JE@���% �8t�.
��P���k��:v�����.^�x��Y[�l���s8�@Sc� �0Y�;g�{{�K��^_����T/L_�Y�Q�}�A���z Y�d��O?�y�f*2 Z �2 �$���j�i^����� ��w�����k���9s���]�t cL �2 ��HQ�KG�m���:�� A�^�����o���� ���� �0��4��D�?X�y���.]��SOm���k��a
aGQ @�\�=���.��cb�L��2CJ`��7��2e��u��� h�(� ���l���R���JO��?;e��q�B��m[���9o���O<�q�����G4, 49�iZ�3����� ��}����p��,V��� �������W�X1n��
6���32�P�|�7F: ��B����w"�T''l����o���w��"�U�( ��=���o�X<�5�I=�� �-p�TI����R�DIZY���������7q������ h=�) d������B���h�6���Y�^� ���:�%m�K����=&��r�^>�����{���� !e ��������+9��9{.����"zN�����#����~�/�p����T����"��a� ��)6�����X\��{+��E
�e��% ������T !�-�������� "��2 B&%���HQ�����;+�W��nl��Iq�m�]��)" �E !3!e��V���8&&� �L��������Wb���m�<L@7(� �~m�NY���������NY��������=����=W|��� ����&���� ��Yc�8$a�-� �/]��O�� ?� �R�{��p?&-���a :�J !�c���������s~� ��([Z�"#c�5���/�Q ���i���wR�S d[�c�������9�W���f�e���K�S{�Q1�k��@���W{�/ ��S��+�U�8|�fe�y��)'*2���K����U{>�d �fl_ J?:�������?��������p���w�u�j�CH��������R �/��+w���s� �1���uW���l���BdW������D���;����?���K��] K5o�4$��#i%�5���*����3 ������;{j�^��E5�t�>n:T{N�����{������.@����1����7 D\�sKkj��f���m�G�����������F�S�����2 ~��]�A!���!!�l�" ��� ���>�� ����Sw�� ��Q��2B�b�������fC��w�F�k� �pZ�;g�?�T{O���R����8��Z��.����y���(���o ���pg�:��j�J ?o�{�*2B�S��B�������������$�
j;��8�)�Y�( ���U;����Sl��=��H�����%���9V]��*�j� �� ���� ��Kt9�F�<�Z���'����� u�9VK�� ��Q�fx@�CQ ���k�2��\Z�]�k��e��� S�p��!����`T��k���&�f��K ~����Tk���G �$$� k��-� [���<e��B��oF� ��2 ~^�1����o������pH����T�4�!����-����� � �� ��5��.i \z�I&w�A����w�5���$�Q1&Y� Q-��� � V� �y��o�j���gb��2������u�% ���2�9����j �� �lP� p�y�����%O|�qR����vB�����}K�eV���N�X~����!TC4� h�2 N�������6�����|0K�(�?�� ��������MT=*�dy��@3GO '��{�i�.�F������ ���4&�,@cQ� pR��<bq !RL�3=�)K �xl_ 2w�NY�}n�CS|Bj�J�*B��}��������-��D�}$�,@cQ� Z���s���u���;����Uq�5�m!F�c��,��H���b���B �
�/ ���u�~���`� Uz����)��� B <���� j v)��1�� 8K�� Z�#E��Eh���Q�����/��7`��g�}6���H���� ��%� G���#IQ%^�����l�~(���K�y������ @(IM�"�A���� ��l�3�Vt[�����*�'�V�������o(�|� wD. �-}���1����7 ����KI��~��6W���'!�X�X��oH�1!�G��o��n87�Q�,���^���F��9 �l,��3� ����iU �;�tK�'BcR{C����T�Nx?�����>p����
gM����� �F���3P�BT��s��IaLNR{�0���mU0<������\L h�(� �Q�+�_����z��t�&
'_��]��7 �e ��(E-I������h�
���p����#� �Va������`��_c��������,8q���'��a���JCbx�@k��>7z��n@ @��i�YYN��i�(<��%�w� z��� ]5�� C{|_���b� �Z��5. ��>_���Io��� .�������#������z&V���w�T��OH)
��h��T�� ��>_���Io��� �,sW��5���;�LjA�S�ZuQ9��K��g�x����n��!�����E����W{� -���9�W|�]R���
��+2B�X���s�?�����t����cG�+) �^4� Z�I�}W���k���II+rm����m�ZZ��[u�� � �J��-K4���GQ h��jj���<�'��y��s���%�H�*�1�"&a�j� �� ���} h�R�g��!*_���_��3
������ 8�� ��O^nU ��2I��v�2**h\���� 9�2 @ �/����Eij�*=m ��J��B
��\�B�u�z��8��2���� �j�S h���&�M�%�c��19I�q�4��no�������Z��8q ���2 @���+���MQ����M��r�����Y=�XIB��G����!T�w@�Jk�#�1IJ�bL��=��K fR��Hg�;)�) @�����{�.���(E,�S��/�p��7.�0*Q������T�Ob�i���� "D���l_ ���+v�|'���|��wv�~Qi8������������ll��]c �AQ h�����"�"�X��v��� ��<'� GQ h�N��"���t!�8:� @��� h��eA#m�eB�[x�gd�%��&M h�2 @�P�)KSS��w`�IqOKy]!
m��E����g� �E �X�;g�{{�K��^_vI����,��#��2��y)/���������R^�2![�S��8j=/-�������������o �J�'B��>�� @�r��6 ,��="��o�m��g-FiU�A��-�������oT�?�������h����5���=�~ �f ��:hD"�����D���&[�����d�G�I�|0p`�>}��wAD� �� �G[�:V��*q������l)j��z���|����������������O� �� ���S ��Os K�TT{=>����dO���������N���W� _��lg�������� �� �_��2 ����_����}��jW� ��]�H�� a�,��Q�+���O{c�)e�� ~)�2 ���x�? +��������>�t�_��>$���L[s���ceq�K�<p��]zG"2 ��(� ���bX��������B� ����.�R��lWr��R�YW���s����e��E&4 ���) �N��<P��sy��#�����������/�jH�,� ���:�V9��{^ �/AQ ��77}U�"�W�W% �{�[�1e��� S��u����r�,�Y�Q�b �
�� ��s�)�� U��:qX����3��
)�� Q���7 g��2 ����A#Rj�%��e���Nc��T�?_��S��# �V� ���Y�����\�-U�;��aW���*)�������������[ M�7�'0 �,Q� "/ky��1G=U>!�s�{�}5�������_��oK�' ZM�g}>a04�+�!��s��a� ���V��N%%?% 4���������[\j�
!��)v4y� ���{��g��Mv[ �� �l�����m7���z��w���?M�:�� �����W_��_�� z��r5�. [,�����(��U����"�e����>� ����M z��(�i��?���?�}��u'l��-33s����7o������ ��� @�|ty�S���n{�P��U�2�M���r �P3+�H)�,Y2n��� ;�;a��/�����B�a�� =z���c
� DJ�������cbR���Ngp�j���2�O]z7q: @�hfE���t:SSS� ��{w� ��j�w����q��J8��'�1L1&I�Hi�BQ�I�����=a�
�U�q:�QQQ�G��� >���g&O####D� ���o���8��k���������&���]do���!&e}L�{�L�: �Q@�222N�� �h�kQE�� Tv1��N��!w�L;
�2 �[S�NY������[c�Y���%1� ��������G:Z�ZTQFU��"���b�| ��� 'J�����|�u��Ba3� �J?D hQZTQ�d2����#999���g� D�k��WD�)V� ����������'�����j:m3` @s���2B������\�~}���x �����:g� ��[���2K �=����+.�����6����_zL����t��?b4��lf @h# ���7v���� �����OJJJIIi�] <���=;'ke��R�7��p��� ^:0"=��8��C{|?���u �����
h2��(���F�Q^^^YYy���!�x��>}��'��w�;v����
�9���� /�~��w ��x�������B
!�b=���M/�i���� ����/\ � u��X?��� �PJz����Z{�s\���q���d�� 7��s@���W�f�R hJ R��jp��^�LE�5���Z�# �(� ���d��k�$��k\�u�T�Q1 �� -[K;} ���R6Xku��*��$���{T1&IiT�I1���� @��J �:���,��y�n=�IH2�Oz�}�~��� q}�� ��� n����%v\��� /���W�{�L�-�� aFQ �y�����%O|�qR����v�wZNu�bo�b1���W�|��K �z� �1����f��^E+��' �����zg�Y��F:Xc�> @_X) �����V_��i���{p�^,�iZ�������'�;�lr�9�9���;gxx� "�� �����+h��i=��%���]h5J�"��Z��HLBq�� ���K @h$�
�F��E�/5M;v������pE��(\��7� �'e �������8�V����o.5M{���w���v�Z�����aJ �
�2 @h�oshv��45_��45v��~�?�N�4��O?]�zuLL�4�
zV� yeD�$###�Y �S��OJ����:|�-��>jP��\���oM�4i���6l���B�c��r�Ii1���Hf h�222����R?�iZ�3����� P��]�S�|�[�H�1�����_�~��
����}y�J���S���w ! ��u��m��1!!!���z��b��-��D�}�j �� h��j��Lz��� "k���{����}����Wo����O��&��<�3���
�r����W�^�i��� @����^���F��9 DV��+���
5�Y�mw����J65j@�vx���7�i�& '����� @��S� j�XPep�O t]�v�5U�R� �E ��HQ �]��G��
�T<�[^����Tg�O^���" 8 N_ ~�b�[G���8���iA�#���h���^G���8�7 Z9�2 �/aL�7+��45_��45V�kC/��|� 6�����"� �sl_ ~��� r�]�I� ������l����Asl�<"� �+e �_�75+��z&P��e�c�+>h�4$F$ �Y`� �h�2_�������< I���I�� <���4��r)~�##-���� �5V� ��^��'��{�������=��g�����l��*�$)��1���j��� �b� �h/__�3.�}��
������u@� ��2 � ��r���>���c�y]���N=��<Wp����6�/n�| ����K �B,��3��=�%�n�/��z�{{���9���;h$�^���Ig �(e !���n_����rz���w��O\�������T��>�5oA�F �,l_ �"��4r��H������������N�UN��}H�,iHi�� �Ej�� z'%?% �����6 (�lSn ����c�9wx\`��W_�s��i �S�Jk� � ���W{�/ Bq�%]�C�`Q�����>��_Y���#�/�5kVB��BZjM�&�
Td �BQ B�C��.�QER��O���k����z�b��%O?�����S�o��=���4*�$k�$s�� 4Wz\��7�\� ��� �}��Z�|�)��<O��d&�X�����36o���K��D �b�|�oP� &4�����{�y�H��9 ����s�}����. !����A3��;���n���k��a� }��7��� ;v���+g�s�N��` IDAT���" >A�`�$�"#�h UFE Z
*�������g��< @��=�^V�qiG7 @h5���� ~v��9s�:�~��dddD: B�������H����`�����$ �LFFF�u>�Y���-Uz���g ����}s� ^��W�f���q�_����g�q���|.l ���W�m_ Z���sF��]���b�T�����}� !L������� ��P� @����;���'^$K�Fo�;>�V9���!����6Y �� ���qE�������.]����|��Wo���%�\2l��&
�\�����m�3��
9�@�[����� �Q��,Y�d��1 (**z�� �v���K�6Q8 �R�����S
�t{��k6�[�B�cG�! �Ui\��A�-_��n�?���V�u��Q�c���6lh����n@ �e>=dr��j��iS��]�!�M�^R��I~7,� MB����[)S]]m�����m���k�f��d25I4 � ��2pV��45_��t{����\��b��?\� �H�z�����?���v�����s��! 4����7<7�H��I� ��E��\*g:�I
�%���/l �G�V�t��m���� JII���|����& �X���-c�:b�.� �����������Q��I�u@� Z����Y�p����kjj^}�U!DNN��q�z���4� �>k���<��)>������h������5 ���.�k���52 ��sV}n*++���k�X.���f�}v @C�������j�(�&�b���U���_=���x�3�b� G9 Z}��� ������
I}��o
a �o�i���T��k6
��������% hi��j���2�>|�� �NW�BT���m�4hP1$S� �G�z��t�Mg��c��� BL� �v�?zs+��\K�9vd�g @���LMM�G }$���|S�L������������Y�m��+V4qH ��f��8��{�B)�A����� �M��T}��g�^z����������'�sn��u��
��Oo������g h��7�����,�����<�[�m���k6 ���R�I�lN @����}�z��+2B�={����!����}���>
;�#���� u��_��w��E��M�)W
�����6
����% @�4h�R@eee������ �VVf�gS��r��T��������Z ������
���#�1��K����% @�4�(������n��F 8`���� 6Y�������Q��Wa�qN�cF�#i��)5qjq�;a��[?������#� ���� 8r��]w�5o����{ !���� |�������k�%�<}n< h�2w�NY�}n�#5�<�����*�x����vA�S������ ��j�Iq�G�>SF�o�o�- u������
g�����]QQ���6c�����&
���� �f�w��\�����/MF���.�����][��=3=�l����G�X.8gU�� t@���z��7��� �fI��W�j���G ��}�����=n2x�?�B�M�R��#% @7��j���������:�/ .��<R�B$�
��S�A#���
���F�B����]�v9������3w���T ��DPi�����/��3��g�Z����=JI ��[)�a��!C�������+��) �&���H!����2����Eij�*=ij���E7u�&x��]�b p&��Ru��W�\�2&&�����[����P�)O�k��i�222"� B���~�>��lG���^�MM\m��&MQ��U}���
W���L�8�b� �Z��?3 �222�z����{�4�(3h��U�Z]�z}v h��=���o�E�x4c��hF��������g2�8��Uf��s��� K5o�4$��#U���� D�>_��S&&&��p�����������!M �c�K�' �r�|B���X�S���[���2g���D�c�u � �5����3&N��D��;� i$ 4W[�: �R4|M��[�� u���g�=�4�NwW1����56Y � �D�V�\�������Xf��]�N ����\���+ ^MQP�]��Bj��T��>�p�f�NO�1������ ��� �W�4h���s�� �< h�V���Wd� ^����B[�I1 v�=�]K�9��������cG�0 !���Ku+2B����� @3V\�
)�3r��'/�*�vE�Z4cS��/��Z�[� Q�IRc�5�Z �V�V�TVVz����X!DTTTH# �Yjc1 �:�
�`1��fe�~6%�*�mKU/�����qW% ��[�����m���v�^��)�*:�V9������ ���/ ��htQf�� ��O���Q���d���}��m�d h^�v�-�����l�����
��0 ������j:��m��
{�9Z��9f�M��n�����j�7��i�m7 �5�����wO�6�������B����y��]tQ�%�<} f �C[�:V��*q������lW�4��L����#s��w�0h����}&��Uq>�c����o� \=�C�s�~�& q����� ������q�n������k�.0���?f��>�� ���>s -����?�/[�������0���B����DCYy������*���z?`��
kC�=��a� h6��j���KN��vEF��];���H h��_��k���OQ��Vd����� �%]�mJ�J� ~^9h ����F� @h��[���������{-���)�b��% @s��#��;���7� ��ys�n�B �QI��g� s�UdCjl�&{�*N\ 4;�[)3s���o�����7�x��_����o���� L �� �� q�H��"�!�[��� h*�[)c��W�^]ZZ:j��Q�F��Gv{�v_ ����)� 4��d�B����ov?eHl�d 4
=6 �}�h ha�=�V�'��H[s�������O^Ut�������V nHh5'fH�5~ �d ?K���g���������i�&Ti ��d �=e�����Tk�&���JE���� �F��Dk�#�����P���Td �W��/��3�{���>���������o>~�x @ ���c���]Y��y�+K���A�����,W���3cR���g�� ��5�( =z������/m6��)S,X�� ��M����}�eL�:M|����� �%i\Qf���?���'{�_s�5��y�S ����.�}�x��l�;piS���l?� i� �h �C�z�H)
C��� �� ���z�)
_���(�sX.�v�'� ����w���Y��K��:� Z�Fe4M��R �s�l�.]�?I� {s�GOow�V�������u�"sW��5���;Rc����0������*��%KWt������>!���
,� ��4�(s����X����n �����]tQ�S ��q�-
V����>���*�&��]R=��=RjR��+�T{�B�������r�r�OB���m��(����_(�bo��� ��q�t���Q�F%%%]w�u���[�����[�lY������� ��<�:M�h�����(��}��D������q�o�W���7��!����&H
hu��j���2��������o�����Ng�~�f���n& !������>Za�{���sig>g"��S�B[_ @�����_�>}����( h���wU�,��`J�W :u�L�8suNi�9� ��d*U�I'���� ��5�Hl �tTKk�#�1IJ��g�:���f2� ��?aG�#��l5��wR��3cR���g�� ��5�(�����t�M��{��r���999�N �fI��WU�|��W���oU
J�x�+���m������gzL�j��������k�E$6 ���K;w��>}�����;���?��?���^x��g� ��������W��m�j����Z���|qg����\W���eNs�_^\*�R!�
od "�q+ef���x���������� ~�!��tG�$###�Y t*ky��1G+�]>�V��Z���q�-{h����[�����7l���?>S��/5oA� Z������|����q+e*++���k�H) g<��e���Y z�����*����K�����&���me��O�e���O��X���>�k�HCb�� Z������
}�e}$v�A}�� �*s�i�}���� ��
�jz4v�U�W�*���z�4_�����N^s6 ��i��������� ����M ��u�v���7]]�T��V��n����sm�B�)��� �U4?-��'���&��d�{�3� �J�V�L�>��{�y����^�w��U/���� ~�4� ��d]�}R���>��������.8o�����uy?9����G��*����6�=��v�HHn��� D�ll�����?����|�M�� ���<��cf���������) �B��X��L�=�%�t�=o!4����s�E=������d[�������� }�9���� �&�|��c&���o @o�>pj�,�6�� ���k�&��^��!���Vx� Z3}��7�� p: b�x�u���� l @����Lnn��_~Y^^.������� �b��� �f��K�O���5�����}�3� �Y��2K�,3f����� x�����]�v���M �������Bu���+��0����5����/d�)�(�� �f��R5h�����������j��5��p <x��
M1���� R�2K�>z�Q�B�
������)�x|&���w�?mN����*�&U�}F����4w�FG�"��L�(����U��H�� Z
}��7�H���j��.���m�����f��d25I4 �O����>�u������-�EN�O!�&q������BL<v��l_�_�Iy���j @ ���m_*))�(--�����s���C
z�������bU��z�R� �x'�G�����; �q+e�u��u�����������KNNn�` ��� w�e ��� ��?�i���&�Q>�/�%^ 8��.�# �f�qE�� � =�����W_B����7�wo�_ �Zt�v�wWn�rtt��,�u����F�����q���E� �6g�����r������ / a&��g7 ����,uW����x�N7�(���
��?*�� 83}���1����7 fY��F��?_�SV��=�������'<� h}��7n� Z�?��IW�������'> y h�B��w��Yg�% ����5���� S��M � E������� @�4oT��Ag^����/�, � h�T��z�:�y �pR����Mu����6�)��"�R��5�(< �����������?�� &�������: tGJ)N�w:)������~N�������'��
���AE�#F���K�c��W^y%���� �\ �����C�����)�th����)��Z��1 �U��2'N������~������ ��L�d��[��r��K����������.�� h�w$��7��D9 �sC��F������N�UN��}H���I�b� �t 4?��;��������C�F�� �O ��+=�O*���GS|�j�iH4�G���� �����^���F��9 �0+<|�j�,q��b�U�FE$
��W��m_ @���Y�����\�-U�tf�s�����v�_���s�T���4ct��b-�:E,, ��r�_1k���� D\���-c�Vd�|n�"��e���eeB��{��w~��g�����4*������� ���J������ �f����w�U��[\j��U}�`�v����;�������>0�L"��H� ��`� @����lkS�ym�GB��XO����&m��O������ ��iPQf�� g������( �*ky����l����h�O5�Skw�G���x�E ~���l�\L Z�ev����+������ �< ���}���'T�)����*�FE �&�����n?���Ow7&&&ty &Y��>>���sR���� �Dt��-����9 f ~X��&� ���_�< �6
Z)3z��3��6mZ�� |������tw� l�/�� �V�A+e����/>�/�9�����Hg �O
��L�S\����L]��- ����x��t��IM;���!���� h�L�wij��l���k6 ��� 5.um�� r�|��J�Y�f��� ��G�� 4(�����6
��%�0��= �E�z��������K f)���������R�Q|9���'gH�9vdD� �4�(��zE��, �����?����'��8�g��c�n]���{z~��������m�S�n ��{!�O�)�vf�H��/r� h����� /�������'L�P��������&����x ��������,����4y����;��u���[��(��fH��>�) hz�|�o�J�#F���K�c��W^y%���� �\ 8k�� tZ}�K��7g����DQ�&+��qj��:c �I4�(3q�D���~������� q( �B��%�);��-.�M��JDB|�#�`�: a����n���&� ��K�*�3rb���I��oi��S�Q�5 � @c�2��w_��i���( Bi|�2��\Z������&M��q�c�?�����
gB Z��e���}��� ��U �f�,JS�U�IS�g�,�*����?�~��
6$$$HC� r���� �B
�)S[vv��c�j��������H �5��w�|f����)Js�j��U�����l��)!!Aa����d��jN< -����L @k�����)S����K.�D���� �T %���*���,�*:�V��e; ��6n����!TK��9*�j�BiH4�G��~�� @���c����~��g�+2��>3 �Y����'WU�U��Mu����G�{.�� ?}��7��L�.]v��4���o�. Bf���!D�[}a�o"� ���BQAA�\0h� ��\�jUvvv����>�i �e�������rGj�y�u���:Ua���N�)�+� ^���"� �����}�z�� 9r���}�����i�� �N �FX�;g��=�^�"��z�����n��C�������3��e� �5�(S^^~������2 :7�����j���w�����/��c[���)3���"� k\O��n�m��A��F�
] 4Z^�3h�h�s��9�z~�p����r��K�)_8`�� �E$! ��q[�:u����n��k/��������� �^�s� @@�Qk
b=�G��m�=�����8^{\1���� �t D�>_��}��������>}z�� �a�2K?��_����������U�N �`U�����m�l����d��jN<#-���K N��B����� �~��7:*�j�BiH4�G���N��e-/�ro���������S����#�y�������6������� �B���z��7��� �����%�k����?�Z�G4TK�yh��]k���Yr:���5Q2��0���C�6����Qiw7� �����}������?;g��Eg 4?���'+2B��Q�f���p�}����{�o����U���,��V���;���+k�,Q�" 4P��2�V���9�W�>�0 �����uF
"��5���}D��)�e��'^�����a�����U���h�i7��� �!������{��G�<��o� E ��HC[��{�Hb���xI���r7m=����:�l#-f!���� �
�.�D@ �
*��[��g���/9�0 ��1����>�1u�7�XTyD};�S���Q_o�'B�9|� @�5�(��{��� �)����i���c��P-������NO�{�m�{r���G�D ��5�( p�u ��t� IDAT-7��{:��zoI!����!��Nd ���F� ������:g� �M��u�9���t��zr�Md 4�� ��O ����Xh�"�T�����f�V�%Z*&6� ��P� ����eol�������q���w��y�v�e�)�
���'��� �l�} @��?{w�T��q��,M�4]X
mY VQTTDE(��
*��0H u \�e ew�q7�����!�(��N�- �m���I�����E���Iz�~?���{oi_$��9�y�mSv���D�W%%z�/���A�$Uq�N���}3Md �u(� (���EN������v7���H>�$�%YB���������;H j �/ (�9�Sy�' �P��y�V����b<W<��&2 �6�� P��XwuO���g6__��%R;��J( Ne ����%����<���ORlR� Q� ��� ��2~1��wUv�����B�4��� ��(� (�)��^k��1����^�u�;WjT��eT�<��f!�$T����� ��F� Ja��<��iY�"�������U�Fv�+$y�� �[�?F��_;� ��$�`l j�0e\.WLL��_ ��0e��N-���
��n�S�[���8���� !r�����
I �a���Y�� �������<��^!��b� �ZM�e�"/1r���K��%�2IR�J L��u��yl�K�������s�ugl�tI�y����|����8��K �'e����L�a�����-[��) @��3�`Ks�� �f�7�� ����{
B�R�K�g��=��� �>�����s�� ��8��Lyy��_~)���|S�Nm����!C��������7.Y�$�! ��u���(t
IQ��y���n�SU%I���'1��?vY:����i��.-�G5. ����}i���F��<��3�������[�h���p� w�~��s���t�'�a���5� �z_���Wd�d!yeU������x+�
Z����Vi��Bcv�����9�����z��k"5 �����+n�k:K�B[Ru�m�}��'A�w�qG��,���g ��2e[.���%W=aY-��z������ ���Fm7D9! u�2o�C�}�����`iii�� �ZiC�c�>[����W����+�y g������'�R�|�+K� ���a�B��c Pg����
4���������s �����X��������'��?[7�9j:�r��-����.7��� csk��|��g��i�� �h �(��s�eff������?���_���g�y&� @��d���==�������` �2e�� ��:���] I}����6]�b����� D]h���7o��G ��5k���V��y�����-Z��P8 �|������J#��)��uj�-�mH�v���nL���m1�5���We��/h���g_��f��4M�����>��
�/����Q�F���J$� ���a�����*L�XuM��q�l���|��'�� v���?��~��2��m�����}U��QI��G���u?|�vc���� 5"��K���,Kx� ��hd[�>`%�^-���P�yjV��E��c���V���k�u�����"#�PK� �Z��EB� }���|PT� ���L�-[���93!!A�R���L�6�G� �H j���zY��/����/�����|�b�� �':���f �?N�?!���P��tY��HE �ZQf�� s���������f�y��q��O���Kd� �Z��:G��E�.*Mf����[��o�� p���-�u�����k���p%Ty�N���}���Lb\FZT� �'������a������OII�),,�������"�M)$)�� ��)������5;���w���N��T5���e��\���'���r��^.�3���'���<����~s�e����G3* ��2o�C�)�t:+2B�����U�� ����V��yy����:�Sv}��������L���K���s�������{z�r{�����
Z�+�V���_���h� JrO �@I��� ;C�+B�5����?} �XM���� ��~�|w\�/Vq�MZ$�N��yD��0��� ��B�}�}���W� Y�vm��m� �&����2�u��]���O���`���K��Q��j��� }z8������N2�<q������`}��Q� �&��2�g� >|x~~���C�_|������vC pv+�6�������ww�c���fo�_��j����NY{�� Z�������"�=����5 �Th3e�F�W_}e6�'N�8q�����/���h4F( P��N��Wd��>����j0OM1e��km*p����$ ���"����������->�" @}����J��� (���e^���y��������S#�{���o Y~������U ��|w_�� }�L9�����^i�w�p �O�����������/�" ����8+����M��]o YB������U ��U�{:�����75�" ���d6���'��Q12k��!C��5 �5RZ��*|�S�U��'j4Q��L+�}��{�Z��� oS�\��^6�v�����f: �\�e��_?u������W ����; �5^�w��w��f���.-����k:TT����{��M��[:���.]" �Z!�����3�/_��� 8{�M @]5�O]��b�������q �� 4���j:TT������*����!�'�p �Z����\�2ri�I��� �B�����NG�OKB��9���� q��.�� TP��}h�~u:��n,//_ �Z��SEEF��i�� �/�9����U������3 P!�B��={f���`����2#F�����#�M)�YN
�z����3��VI��A_��`�y�������T6 ���[���)��s���4��'O����_bbb����o�H4 �����������( y��;��A��o&
�V�� ���WQ�Y�f_}�Uu��v�m�� P��ri�����[z �������<����M�Y�� �������G��h���g�����L��� �`��}��Gc�h�gqZ����� *S���y5�=KEF�%6 ��L&SFFF��4 S�fL m���V�^����(
d�2����f���Z�LY���&S��}���S���R���nF P+�V����o�Ft:��/��< Ja�2���w��\�a��-[��e��>�!������1{��q���������l^�I��5� ��-�8p���+GdY 9r$[b �:��mYwO���k~������oP�(N������
���z��~S�N���� ����&��k�'������&�lx D���[�B 2d���aM�,��� �����i����>y%�RSM��M>�5y�]hgZ�x����Q���L� ��P��������|yyya� PS*7�Y[��~' �Z������@���������/�R(?���"}MG ���<���� �N��l��uk�������� �l �`�������|B�!I_�4;������7yM��'��b�S����K ���-�|������'O� \�$I��W^y���� �X���������{p���������l�~d�}\y0�9 @ s�E�� \��?L�>=�q �.)u��?�y�h���M5�� ���,�b��ti+ |���tO�[���� �1���9z�h�-"�F���
���9�N1����h#�dUY�� �_{z0�A�~��o2� �� �"(��>�F���" ��Wl��(�������m�e���W>���>��k�� �:*���rrr�
��];�� |0777�� ����Q�A�[��/W�i�;�,Uz6��A� ��-����m�f���h������7���k�E @�4K���,��� �{:66���j_�P ���2s���?~rrr��� :�T g�����)� `-������%��Rb���B��b$V#?��k.�� �V�)++KMM
�$��p�5 ��������|�����y������k.�:�,�{�M:m�Yd?{�?�S?/cM��R�Z�J������4��� �wKl?����!%��� z �EB��+�;,���}C��)��.3�c�-�k���;�IY�3>���#;��������/86 @��f�$$$ 9r$pd����e �@���4r"���L+��|Bs�7�)����A���}�BRW� ���6Sf���&LX�p�����X����_����#�
�
�����;s$�rIY����� u�{�
����^�$�F��gz7��IH����� �2I����:�����z��]�v�l���k�y����z}��)�$��W ��._c/y��
&)6.�aml�s�h�2o�Zh�s����]�Z��v�����J*����$u#�q�6.��� ���[�0d:z�h�-��F���� �>�5�k��=����ih���,}�9O1e[����_�$�x�������H��z��Q������ ��(��>��KB��� ?~<p��� ����� �l6�9�����j�C�Z�G8z��c�n������;
M���Q��p��A����F�AsR�`��:># ����2S�N��sg��]w\����S Tk�~��{��.�W^��v��LR��+��P���q����O v� �f�>�}��U6���&
:���� ����2�V���u+{` �T\ �_RQ���.���e���=4���ASMq��m�%�v��R��� LB�fn�~������=���.�� B�P�2�\r����������>�`���aM P������3�0
b�� l��mv��R�'��f�����S>6� ��vMY�Bor��lZ��#:��#;������f�D�� ��,�>7'N��������u��[�X���������� @��1�1�k�
&�Z�t���������j ��� �7 <�U|����n�=)�3��U�4c��#�
@�)��>��2������w� ����G
w* �j�L���X��V��%�Uw�7�H n(���c!���;�L
:�Xyb���W�b���#� XhE������G�S ��^i�^�
1L�{�'h0�h=t��JiFk�1*MS�q�6�O�r ��*�������_~9hp����� pQLY����<��5�
Zw�Cg��|���$K��� B�P{��j��E�F�1p�LNNNaaa�)�2� ��[���-�u���j� �q����c���o���->�P6����;� <�V2 �a���m���w�=}�����3g�- �y�Y���$����y>�,��{���������CW�w�7�w����Ya�Y=
��k�tX����H+ uJ,)M�����~�rY
D�:Lj������oP�4��&�=��sd�u;���w�����N���S9�l��������� �E����n�7�g� u����g��Q�P��2���9N DZ��Q���6����_���c4����7}1{���l����c�FA���b� zfR�� %P��}h�~ @=a������ v�����k]f��CKM�BS��r���Q�**2B������B�C�7���Q�;�"#���4�fN �s�( ��3�����c��<]P�y���"!����0F� !�G�b�>]�����
�2 �
e������G��_*r�]��3a�I�-^|�����U���cK� ��� �B|sm�H���������^(U*���4_������6j�wG'E,� ���� ���:���Iy+�4U�YMj6�"���r���>U^�����c��^�)���k]���J�� (
3e @��N���4c��V2������v�k:��:����*�$
!>3��X�}��S���ww�����f��b���~E�6�OT� ��w�Re�� @���}�?��4�����;S�~�V�K��f^����Q�� �I���J��4��� P����[�����tm��)��$ !L��w��wI�1_�W>��_��w� ��B���2*� �r)��> =ev��y�W\�u �L���+�ed��mj)?���+��4������r��jh�V������L��N!�Z�8J� BrQ���p���N��bd�����m k* P ��� �E����n�7���e���y���v�M�Sah^|��%9S��������UuE��uwM���{s�Pi����}��wB���/ P���2�V��5k���� _1�r���
(��|��<O��B�s�n�'I���a�x�q_�+2~�8��~}p����*������fd��A�qI��� +�����sW�X���8��}��F �����_�9E.wX��(�j 1��!���C�*�2�b��*2B�N
�� �� mKl��T�B���+|y @� {OV9��7B,����� �_y��� ��*!���O��V>������K�sz��K @�Bk><z��� ����������Lu�2[4 P���U� Q���M�����ko\6yMo�[� ��PI�d�����r��fz��)I�q j;e���6Sf��Y���?����7.�� ����-?L������+{�a�����;{C����B�+�N:b�� �����e�d%�}cl�EE (_h=en���������N���}{�S E��V`��������/��V�[<��7/���)��uj�-�m w������
��-=�z1� �B+� <��^ 9rd�� �Z�,�-�8��5�>��?��I7�y�4e[�e�{l�7�L�� :q"���N�}��f!�^x �� m�R���"+++La @��\<�,x$$9�
<6����xo������}KB��� }�e*���Y,������9�� ���:���p�S����:���\�?�F�j�BX �������|�� ����/ !�l�2s�����JURR2m�� =zD" P�v��,�[����f���x��W��9u�����Z�W�);�\�<I[�m������"�Y�!� ���P;v�x����{����d!��l 7n�����t���5O��f PK��-����/VBh ���IB���U��tj�b����We���r����G��m�' P�(��>�LC��?~JJJ�Haaaff�g�}�lJ��� ��mL��\g I��2���O7�+B,��a���yeF���61�
6�+��j�r�����_�E28 ���yk ��%��X�B����\��� R�>�4YX���EB����{`U?�G}�+���7l�S�v�/I�����w+����G.3 @$��S �b�����r���7|,����� �-=r��� �~���V���eSz�������{:Sm%_�^uG{C�4}�^ @X�V�i������322*F��]��m�p� uV�M�M���|W��{�MKN��| mj������dI���3����B�B���(� ��,������� ���?t�P!�_|�x����P TgC�c���sU��DR�sS��K?-��l����M5�����m�R��Rm��0e �y!��q�\o����
�={����{u:]d�)�2� P�6�9������z����{.7�La���n|,��6�y}�0h�/����Ao_HV P�)��^���F��9 j����
l�����+�����,���� �7>�a�*�/=�lZ����S�\��� ��2o�i� .Dq�7h���H�LY�
w�������� �����)���.0+ �"�.�s����� ���alp���F*3e[�e�ohcz������� , �����&�cU�Gv>t�Y ) E���7_�E @�2��A�>�S�^-���p��r�l���� (*�t��x������*7�� (�y-_�z�*�J���� �7=���$w_�q �/���?���|���i|���r���� (�ye�t���o��^z��G ��������
������
v����G��rW�����" �;��Dc������b��-o� �eNNN�s �3e��N-���
��n�S��I:�)��Zm��V��$�\���f��� ��� IDAT� @Y��(�������h������MHHs( �x��� �Oa=�Z��/$����61A�}����7EU�Q��������%�l����~���9�9�*���u�%=
/
�B�<<t��� �����w���n��{[�ok����tAPQ&�p���\�K �r� HB��X3�����z!�KNLjp�6.#*/ �fH�\�wU���� ������Q&I
�_ ��dS��?[-^��M �>w��#=�K���)����,������'���������q��I\�= ��9r� @=��[�0l�]�+2 ��%�m!�Z�-����� V��9�jX�jZ�����n5���\�a�Z�� �a� �4J,)�2�i ���_vI�x{���e��Y�������� �TRz\�������bQ�^�q� l+z���$�Znb+v�/���q�'lq��m���oT_ ���yk �L�����~����T�s��G yd��%� ��������m���J�b�� �e|n����y��dn����v!�,�w��K�^������D���=�B��p���3�4B����h� �ZQf����� (**����� �r��E�E( P�?7�H��T<��������6� �����VQ:��
M�}�k.n���=d��p�t�5�����J � �e~�����Ww��%++���������o���� F( P&�(
���Tq��r�\b�2�y�'��*�������c��������>�i �)����n7�B��7 8P���u��)� �n����Fd���0V<L*�[56w����w�iF� ��o���Ur�V�j� ��m�Gui��
�9�!JJJ�?���f�N}Zj��u�C e�'�e��/fm�z��j(�z�FG^�� �x! !$�C��������~z�$bT ���'��=��6}��1M��\� �� P�B+��m�v��
���iii�������` ����� �E����n�7���eT<���������d!����������<3p�����N��
����lx��>yVo����^��u���� �,�e���7i�����w�yG������_u�U�� j��|��<O��B�s�n�'I��S�TO�f��"���U������*�����|��_�Lj ��$��LZZ��+* &%%=�������N j����_�9E.wXWe~/��-������� �B+�������[�� ��zOH�#�� L��D��D�/�k�4��7l�'�G@ �Z/��� @=Q�
�������C���v:���J'��$�K��>�2 ~�e���~�����R!���; y��%K�D& �I_�p���>�����>�J�t`�*K2*I�H>�$*?s_�� �s�e,X���y�����������!+W�\�hQ�� ��r�q�G�Z�j���E�� ��AG/�S�����h SX�,YHIz�V�
����x�S�"� �6�d��}�3x����l���������M�8��p 2d��U��X�$)�� u��<��_��?�+z�4�rc�t�)���&���+��<p�����F�{��J��xf��" �Z���m���n7�B��7 8P���u:]D� ���3]?���q�Z����'v6�H�w_��1��l{vE��� ��</����;�F�`� �B�}�������ln�������[�9 P�^i�^i���,�$w�U�������k�8sviF�B���= ���> �m�v��
���iii�������` ��=��?��w���Xc �jm���� \1l���������)�7 !d�7�a j���2����4iRyy�;��#����}��������d 5f�Go������B��������'+��;u ������# 1/c��&���j�G2/ @�sQ}n���~������+��"���F��� ����z7��8�,��k��f��������s���� �/n�F������$ 87e��+1��(�? @D� ��%kGb$���n� ����N^���>��$d�JN�9-N��wz���hT�Q]�E57 @U�yk ��K ��hS 4�k^�2����[50V�i�/��}-J��M��W�:g`EF�rP� 8���2999��
k������>����T �LY��Z�����^k�)�,����-N��8 N�l��,s���� n��dyl�#61�9���� �
!����kZ}�# ZQf��m3g�\�hQEQ���o~���" D�)��.3�z��s����Ucs�I�����[-���'��'M{rD���o4��dy���}��i/�H7Z�.��A\�^ @mZQf������ONN�0`��C���
DO����8���d��m���z��1�47���!��,K�������001}��[n2�N��h�if
h �� �YhE��������I� Gu� �+�uW��� �?�������&TwLi�,������un� �U�Z%��}[g� ������j��%���� @m�\k9��<��]���{��6��OT�n�����1W���:=B! ����2 G� i��e�����+�� �����&k'�y�>��E�nX�Xm=�5��)n�B,�����vv����� ���(3s�� &,\�P��zW�X��������� ���������[<�����[������K�u���{^Y��'����8�~W��h� �3B+��m�����z��'� =:l��k��f���z�>B� @t������+2B�%�?6������]���
�������}�� ���d����p�$�W �)�,�����<�!]�mvJ�1Is�z� U�1�15��d��G��^���C�����V�����n�|f�VQ p��yk Z&��l�Z�7o �@
��� ��mYwO��=�%���������W{�^Muo~��~aX�� ��2o�C�{���EEE� � gZ����B�_�4Z����"#�� u�� �'�e�N��W^48e���� �U���/K�
����� P�V�i�������^���3|y @d�7���_z�����yC:�5 @=��K�?���Q�/^����@ ,6�;����8|�z� �
��N���uF�����v�KcQ� ������]��c�O��~A��k/�r��.��� P��V���{wFF������#I�>�D �p�� ~�:�������g�$���z!�F��� �c4!D3��T������]R���keQm]���|t�*��1Q{ u^h���z�)��0i��#G� ���n�P8 pa����+2~��t������1��%�������������E�/�[���o�R��}U�� @��6S&99�� \�re�� �0(.���{� � oP�Y�
���g�����S�������#/�+$y�������,��� ��B�)����V ��� S
c�w�����l=a��Ui��M�Wd��>�3[� !Fv�7/cM��R��2�f�q� P�H�\�De�I% @-�1�1���2B�Z��%��'�P�~�mjt�$�;� y�>���{�N�1�4�G\>�K�h� 3e���6S�������/_�v��l{ e��0}���� ���L�O�ll �����q����v�������!�,��5Kw���*������97mKO��jU����o�LE �:��O�����,U �P$���3�o�������7��z������1�,� ��/:n�
!>]�����[�K���M�+�� �A�������?���/����������Z�j�������� ���II���g��(�j�^ �z/���� <m���i?j����_^�xq� �pr<q���O�v|�,_%��QL P��V�Q�Tj��9h��� �@���G �����EBiu�x��l &��^Z��n��X������E��� T�m_m-S�?�Z0�m_}�#MY��Z�����^k�)�|��k
����[ Zw��X\�{
=�e���� |��r� Z���'�w�}���|�I�� � ���v�<���,{|��v�<w���*
1���]=>�z��s�����r���z� �a���5-J+7������ �B����m��kW��� h��W�X��m�m��m��M����� B��,r���r���X�Wa�2��'�[~�a���'�D�%��t9�-gm�~��j(��c�����2ln���Z=��6Qz% uK�n����7��g y��M�6�#��(s�, @=T�7H��I�$�m����+W���n���W �an`O*��y�G�RuJE ��u���Z���k�=�i��O��������Z �I�����������u�4x���-Z!�����- �DR7��yg�4Blz�XPEf[�C�'������s sj��1��5�l>m�6�������9u�������el�
m!� >|������ {����k*eQf9
P����K^>��I��K~X��?�_���T�������bT>�C�O����\�3����|G3 ���^i�h� �9���m����Sw����kW)�7`�����
TA�7N� �E����n�7����1e��,~���'w��yEa+��5���{yd���cT�fF�AKr��<������P� ��������[�J�wk���YN @a����'o����&�sy������NY�]cFv���tO�I+��o����s�G;4 @�)��>�-�/������
~���� B�3��c�9�Z�����'n����?��T< �a�Z��N/q�6 @� m�������s����t���+V� ;6�� ��Y���u;����q���j��!sR�W �o�}��~M��[�8�i�N ���V�?~� /���{��L�>�h�S �s3e[|^i�� O ��%���:� Y�Wu�4a���������� �\����������C�S ����2����G��o=e P��i�'�n��+s��M�V�*���>���Ci&�������:���u�%=�� (�>7o�����x���'N��`��pSev �O�,�����<�!][��nv���e>u���K]��DI#y�/ �NH �Q��}h�Z�j��E ��8Y&''���0��B��9 @=��k�c��_�J�������} ����'�Z�p=7$�� �J����-_�����O� 48s���� ����t��O<_<�����u���c e��BQYY���MLL { Rf9
P����u�~K�h��
�e���-jw���� �h���m��b��-3g�LHHP�T%%%��M���G$� � ��������::eo�{Z�$��| �����f����� ��ZQf�� s���������f�y��q��O���Kd� ����h�n���2����q�$���w�5�� �v - (Sh�w� :��������������>�,��B�s� ����vX�������8^�q��
�k����vc�^|��O�.I�'Yb�������S{^�����'_�n�%|k �Ro�C�)�t:+2B�����U�� �7���v�<!� !d�q�y^�$icCn�b�2ox�������z��������b�o�� !DQ�3 7?�� �[[qV���r���Z�<���azA ���{� ��pX�+2�����P�2�l����36ZB�����7���<�W��h�YG^�)������1[|��lZ��#;�����5�m8^ ")��L���W�^��qz���k���s �����FN�z��U��!�0���j�7��������d�/{��� ��2�|�!K����F D_hE���g ><??���B�/��b���u�� !���eo��#�m_* � ��cy�������� l`���y�����Q���y���������47�q����� �� �����z���6l� ����������U�g���n@ er�����|z����6�O�a��K~���a��s���g���w�%��qE�������ih���,}P�_ @���[{%fRe�� �u�����k�%/Un<���=v������R���qa����l�������r���b$���[#�B � e��_T&��WRR��a�0R e�� �HP��Q�x�� �
z�����|���#����������.�oG�����i�1-�'� �� P+)��^���?�|� z�!��������D u��l���,{|��v���v%N^��hi���:Z�0yu��������x%{��/�R
�"����M�"���U~��wq*g�aqj��^�QA ��
Eo��fyy�C=$I���o�]�v�������Se�� ��!��d����K���ho�Ru��� x�U � 2'��4����77�'�u�����z�����������+��y����l0]u���S:�W���n�� ���yk Z��~���j��bD���}��]�6��B��9 �2m�s�����=���WK�\~�$���4y��N�:pD���.���Y��s��&�.�o� ���9��$������?-��+ ���yk ����$Vd�#z�>�� ��LY� ����J��+��6�9 xX��~�l ��<b�u�����7&iZM^�T�B -MH�qD.? �)��2�,, ��l���|G�4�E�!��~,�>���A���J�|Z�� Yn��}7>��_���@>Y���K"� QZQ�g��K�, �����t��H �J9�
<6���/h�7�
qI�U����Qi��%M �����m��j�����HB��.����RgL���'K�n��� �(
m�����8qb��M
�R�����={�|���u{�� ��-�.�S.����7�T�%F�^��#]/�0e��N-���c��N�Wv��������.-|��;��1�����H� �:J�������j��/ :t�� ;6m���O�O>��nWd 8O���B����6oXu�^�G�
�m�cm��S����z��s�� �-7�����5�kf,=���$>xy j)%��F��4 ���>������/_�m�)�A�g���@�-��z��d�C����
� !�L�w�����_J#�{G�Q]�E"? @ ��[��f� ����x��i��D�� ��� �LQ�4���>p�������
����#;�������I���Td � %��F��4 �2}��d9�
������ Q�8>+{c������S�.r������ ���� ��2o��) @8u��Dc8����(�����u�< T�����<+2B�fI4q �S(� N�EL�V1*��MP !*o�m�����x�����j7]RKg�e�if
h �� �)e �vc�� j�7W'O�,������wzA���ji�z���������������`t�V�qZ��Ur���u�� @ S�t �����[��>Y���/��>���q�����N+ �����r��[�����.C�c�N�^h DE ���mY����{[�CO$|u��'mjm���Z-��xo����z|U�W��y�#�G51 jE �,gZ��"#����xb���6�K����������Vyn�8������}� 5�� ��_�d�s����VL�an`O*��v�I��R�K�t����1[��[� �8���+}m:��bj ��2 �������O�),����� �����f������x�����O�$�2)n��.4� �?$����U1��V��y��� �@��_ P-l�gC�%�_�S{j�R����M���l���+��6�rN��~��Q�
P/)��>�-��� [TT�( �:�,�{�Mohv�xW�(���!�>�S[�������B+y�c��\GE �>
m�������+��L�2g���E �v�eIa�� p�n8��1�K>5.b� �t���i�������w���< ���, !����#���~c��� @�ZQ������?���� �4 �"e�����lI��O��J�g]��e�5
�Z!��K�w����8p��9s$��� ���$�9g IDAT�� N/Y�fO�,K_��+�S�|�,�u����� ���V�y���F�1i��#G�T ���p� @�L����L� �?�b������� �5�����C��s^!>�� @�ZQ&99�� \�re�� P;�L+�:}B�������{dIL�Z �<�X|��u�9O����ol�� P���2���n�����0� ���7�����}� !��� Gx�i��h�\���u��{��U*UlL������u� @q$Y>kB!I�� �>hkZ�h_��� ��$����.���c����jT���{����+M �� ��2o�C�) ���h����z��#
^��4� ����77��~S��� @-���B����a���k����� ���
w* �����?�l�����;�+�WE��uO�f�|4 ��e�m�6s��E�Uen����^{-� P�EE#m^�� �V�Z-���7�p�| �[�-_�;w������Oo�9`��� �; ��1g�s #I��M��}k�t�� � �e���RSSG$Ir8 a� @-�u`{��z��T�*IN��;��nZ��=h� �JB[���j+J��0 (�)��^k��1��km2e���^��o����� Zwu'����z�~{W{6Z @�B+�$$$ 9r$pd����e �3���7����k=���e�a���|����{��w�t�N
L�2�T����3��s�o @�ZQf���&L��e�^��e����3}���D ����-�2���]���q����� �� _PP��V�5- �qo�$su����
_ T+��L��m�z��'�x������
�����/_n0" ��3��c�U�������c��-��e��V !��:R�B�fI,\ @�Bk�+�h��mvvv$� ��,�����g����?�<66���!>��.{���\���f����� ��.��2UZ�|��_ %�/\UY����SDe�W1gsw���.�B�}[�Q]�E,) j��g�����~�)p�Y�f
2$�� ������kJl���'�m2#�b��� \1lGIC��e!��"����t��'��' 7����p5[b�N���:=J� �ShE����O�:u�������eee�N @4���y�}�3�#�8�l�b_1�I���D���������y*.j\�5~c\c����x�w���rJ�A+f�::/ �WhE��3g._�<999p0666�� ���
V�+2��=��MMu���'~�r��a;�:��b� �������{|T�������[2�sIB$
!�P��jA)�����K�B�k�v[�VXCA[�����n�Z*��.J�Z% `�O��� ���L2�d������df�L !'������|��� /g^���>O�&��%����� ���H���._�]�V�o���d:i2 �E��2f�9""#"+V���� p��Hv��B)H��(^��3oY#��3�J�#� &N<O��5������<��|����F\M����������f{nQ���k�1�[�,�' �p} �X,�����������v�e �WQb���hx�dT��!��neG0"�Z�3��!������M���g/kt%�"����_�������b����_|���s�0��h���`a�A��u�(KSO�� `H�[����������"��|�����% p��m�
�Z��;�y�����m� =���T�o�����1[ �4]za��g|����O��ti��,"����m���+A�49����;���� >xea�I
]?���������`���P�`�9��k8]w ��D����~~a��i��� �>~���dJM��?~ |���c�j�:�(q�� ����v[��D�"�EIHL��)a��<?���sO������>-����k;����ok X��6���eo,������V�\il��a����% ���R^���I��[Pl���u����WC�(�qM��� @���h �������_����]w]�� ���j~�3"#"Z������LkMd ��joE�=�?F�r���,����o����,�VY�H���cm���e3r��^'���?��I�5��� `X�+(�����p������� pJ4�����\��X���m��FD�-"]��I�=m����bm�UzY��5���=y�-�Lp7�[����3�]��� ��WM�����p�*� �P�1Q#��/f��4����}Q�%��v=I4���QVw�k�����[sm.�A�����Z?kS\� ���B���m�VSS�/K �YSo%�( ����/ ��^���� f�I1�����}�'<�WD~{mQ�sF�^���W���Q�?��~ �(} ����#�����e�� ��gJ�����`�R�������������ei7W��g����7��_]}�o��6��hR=oN2(�dzn�kk�L��4��x ��y���#F.�������[ ���8/9�$e���%��y1�$�uT��+z}�-k���%����;�s������{�x�%�fM��5b����� 0����%EQ��^v� p��[��c��e[������u�STf��T-9�����v1(����p��R�F�������?�.��;^=~�q��}� qu_�A ������ p�v�������k"r���dY���S���]D���_.�d�������i��xO�kF���������������:$�T�{]C� ����Yy����=���SG����E �e>��C��m��w�� �������fm ����:�� �"�~#����."�sv�\��1�-"��'����{���|����n�rv������.��L�<������\�F�����{{��~*�zrw �a,����o����~��������AEQ�;��k�%+ � ���M���;vM��k��L���������4�9�("�q�=����N�A� s�kH����t��
7 �ed���72v��N 0�����,"{��).. �� p
F%�G�]�2���� [vd��Dm����'.9�J�4Q��}K���#��F{������V�������N����� � }+���c�
�: 8u�O���"+j���|k����K9��� '>���-_;���i"qDdl����{]�#'n
Fd���� _�� 0��-(3~����������X 'o����� �|b9����c�; [�>��_[Gk�M�+��=���5���&�'>q��%�s{����� '�}i���?��OSRR CSS�������3+ ��v�n��g�_��U�:����������������%c��f�� �K�~v�-|$7��MO ���)++{����_���."v����o...�>}��, �>h��vwhoQ����� ����� ��F%�~uUj�"2"�f����>v�}��qnz ��hZ���;]}��O=����cC#G� ]�b����: k� E��_ 0X��T���'�6��oN������@�������['9M��go�lN��:"rfzk������2J>�]�������b]��p����� ��> ���)�v��#2"2v�X����| N��|K((�z��_�����&�J�������RT���3g����z��d��}�bSb�rdB���! �^��� �T���z�#��K��� ���b=3OQ���� ����1����jv;�f�O
� c3?|��K�U ���[Pf�����m�7�3������4iR�
0B�[�z����H�n�l�x\O%r+J��V u�zm9��ucw�������Zm�d���R��].��-�U�LN1��M������^u>I. 8=�����p\s�57�|��W_-"��_���?������qUL ���� �� ��'?v�� _�VU�}j�E9��2���^�s:�+����U�ya�%�?x����I�3k�����1��������9%��)��[�e���]=`7 ����G�>����<��������\t�E���w,���M/��/ ���;�����d���\:*���'V4 Y�loQ��[v�-�R���Y��I?��v��z�-��j�&x��LE���������'S� ���G�>��1��w�y��w�9� �d�����Q#!1�_o^Xv��� �����?���'�:<��K�m�3�����Ob� �I3 � �0*A��� I���;�+c�>�l�S������]���#7���5 @�#( ���'���zkU�&���<sMf����� �K��<��%�
�ze������� ��� @/.������1�����IT�uZ���U~+J���W<f��\~�h�(]N�����!"���:G��S�� ��&�0!=����-����w �B�un�F��� `��h�d��\X���;[���I0@�/��q����?�ViO��j���
b F
}>����/ �b�����"��~�����N���x���u�fe�9g*�&"���y�� "�`�NQ�:�����K���� ��c�Ho�N���q�>������ _Wt��v�i��2����K] ���I����\uk�?�K �^����L �~U��w�:����rL�4���:t�������("�� �xs��o�$�'��j��r��Y6 �2 �����}E]p����G�t)N?��������LE�����z����r� @�� ��{�� /"�wU4�[W��<�-"���1��5� P�/��X� ��{��O�� �x� �E{eIk�*[����p���P1a����������T |���?y�����=_�-`}����l� ��1�> ���^��y�
_�A-��� 4���W�Z���EU/�\zbt�_
�������k^���.�����Y8 p�� t�� �5�3�V5������ ���-��~�p��k�����q�� �#�2 ]�9#3YF k�h [����W�]'�0 `�� ���s����=�_QQb�y�Y�1�j�2�~�v��_�>�<��<�hn���M���� ���h�6�k�;E�� '�bC���kC}��6�eO�,M�8�W���[���������q�w�?7������x����w���e�oS�2L�R,��>���������f1>�h����p ��h��5��>�� `�x~bE���������>-�>�7��)�<b�8�j ���<�*"�hR�~.�L�t�dEU��?���g>��I�N�AF��)j^�m���e3r��� 0D������ ���Z�� ���<Z�����W_����n�#tHS$-��<i�|������&�3 M����wV^1'��K �#�2 ����gj��K�LR����A������ ��,�qDK���o���v��zgR��u��RE ������NN�� �nQ� 0�f��4�:n�6����|�_� s�Q� ���uO_�<y��)�g[���]�m�����p�%e�����sI�]�����e�M� � S 0�
�����Zu�Y�������-X�Y����4J������Z�y%W��b �Y �qzMm�^~����}o XJ}_ �� ����>��0]8������� CQ������@�OE"�9�#�������b M�|�'S 0h�U~��.�{�{�;��3���{�F���J ���������&��������M�� t�� `�������~���."�cj������cS��{�o+"&�5���w�:�/�Tw��'���G[V���rR��*��|��NwG#m���v���yG @�� N��{��2��DD9���#2A���K7e�����J{Z�ERM��I9�JM���J�����k����4e���P�f����pw �I���*����3 B*64o������d�
oI;�z������f��.�.�H1����O�����("f���w$��U��/������ ������� ��{uC("#">g���� ="�~"+b�b2���������"2"��+����5 �*�2 �,��E��S�:p�GI� �'�l�_x0j�RL�Z���D ��#( pIy����.��v������0���|������������ZJs% ]e n��L�-�/N�MI��7����i ���b���<EU����K=�� �4�/ \����R�'�6J��O���O+=>����w.7��c��� ���jw�h� ���� �t��k�DmI�����SBo�^�l����� ���.�A �� ;B��>�f��U��w�:����rLg,H:��5�k)d���� � �������$"�f���b 0�����L @?{����N%�A�'�5FLx���g-��}R��
gU��eID�b��u �k�e�N��-[^~���'�]�6z��/�����NQ�)S�����4���� 0�Ulh������J��{F[B�yAA���T���=�w����8����y� ��V��4M���� >|���: �� ���^II��o����o_t�Ew�y��_$ o�W7H 2$������KO�i����d�O�������@�� ���R�� ��2�2eEy���E���<��� ~��G 1�L"r�
7��� 9r$++��� ���o����eW�qt���l���mg����������(�������\( 0��U�L��nwNNg��y�����
�z `�I�3E��Z����� �A��Q��T��Kg��/��J��d��&�'>u���L� �E �ae�nwD���� :t(�s�n�Z`���&�h����gV�N�
�W
R���]�H��RiV9X�
�CWV��Kl ���������^Zl#((��z#�2V��������
�2 F��j���e����uo�|P=����m���� �"����-�*��� k�=M"�� ��WH������? ���������^Zl���L�L&SD��r�} ����f�� ;�>9������E[�������_�-�5%z�>�1`��Z��N���p�����I=|�����K�� �aje,������������z `hY���`D&���EDL��D�|�h�m������"�(�)���/1���T�����}�V� �����F �A��D�h4������������ � �����H(bRQb.�"��Dd���`D&���Y ������� nI��
������Hn��?� ���
���?���;�~�����+YYY����E��� I9�>�_�� ��W�9z������9 �|�5 "���vzc4f
��fR���6�q�����t @�����@ p�M7������ :t���>����'�g�N��W�r���+��BU��'>��#��^ J����|��� L��� i���O~�(�Y8-���XfK�����O���%����DI3���2k[\9)�u
�L�� �-E�%��CQ�+�*J��V u�zm9��uc ���|P�jKyuc{Z�z���/�����[T�����}�s����-���� �o�z ��R�*����_� ������� ���
��W���q �l_Q'��li��+������� v�Y3�x���,;��e�%���W������~Y? �e 1�^��������5��m���������|m��F����?��Y��*&��6����Ed 0��1{Go��� �q�>���W�bR � ���j�-;X>y"���3�E��?�ZeO����FC��hw��Ed���� ��> ��� ���gj��>�HOy�����8����tNh��� =��x�UW��J #A @ 3�dn_^��d����Fd���|m����s��)�UC�_��dGe�L�����72v����� CA @ KSE���/m�f�j2{�|��;�;���M��4���p������l[���6����hp�"�w�����aB��� @�� b+X�V��#��bC��J��T���������WD6���67�9����G�.�;���1��Q$A�h���X�K����w+ � ������� �N�O�h������Y����3������7�� �"��}�������&\WOl����V�� IDATn��c=�� ��L���d� zQQbo�����-�z������(R�5K@g5��T/"� ��������H[���:u= �xe =����lk
Y������I'�������L��E������mn� � B� ��� ^�7TJ�����Gd���|������f$]a4�E�� �v��������C}{��b�� ��2 0�Ulh���.���q��}E�(J��T����i{��Jn{�g �d�9���_�:�������2g��g��H�#9��3��_\Y80� %��^ `��^��������5_����^��}�1�(�iT����}�����p�%e����'�HN�#�����`z�i���� `H!(� ����� �g�5�^Z��"RQb%l0��8���s�|������^���/������F\����
���SP �Cqqq�q~��� ;B��>�f@y~RE�g�.C�b �zZ�w�����������sYT-�7�s���2�h���|Z���g�N������v .�|�'S F��k2���?��:� EdDd����:dQ5��M�;�}i���v��S]1 0,���������+�c�tjw�~�� �_������=�wZ���b�a2(O\7��2 @�� `�
��$��H����\��5���q��W�w�"����f��QD�x?��f0(�f���� ��N�[��F�� ��=?�����j2yE�O��X���F����dK����p������ �E1(������7���QM �E���d� ���r����Y���m �U:"2��,��g*a�M�+��
��M�OK��a���,����� �� �� �,%�]��:k�� S�]����n.u[|Y�Y��tf��vi�t���#2"�k9���tz��)������rR��.���% �A A���8vl\����m��H�$O{pP�Z#��iI��������r��9� �C�% X��m��e-u�
��m�w1�W7|Q���=�-����6����\���cwg1)� ��{/��s���Z* 0��) ���v��������;�f�u�����m��L��Y���Q��U9[����#[imnlMML����2�A����Y. 0��) ���l0"�Akw5�q�S��Y�w)8�����[J�jg��+r� ���_��@�������@7�2Q��� �� �a�� �<j����DDv�n��[�6/,s�}"�8����� ����.�wm���o�[��5|�����k �
�}o �E ��� ` )��_�ud�i���KE�����UGC�}C��de7����D�'"�<�O;�����Q��� ��z��j{T �!( ��zK[���;��k�m��-��~�F�IrI?a�8����j6����A4QD� |���`����\�CnZd�& �"( ��07Q�\�g5�qE mM���������K�T~���_����m�_�v��zgR��u�����.�������N�����Y�k�O��� #
A X��y��y��[k"[,u���/%�w��g����`��pK��msE��/< "��' kK��
��-3r�L�=�e #
�~`�I��)�%�et��<��E��0N��g��Dd������������Yq�dW
�hd� �p3sMf���y�]���������+�^� r�"��h���d��gd�J{����;�_���=U~ ��CP ���k>=�E��[^���$�T�:#4a����������������r� �Q�������dP�����P� 89����R�(�� %�#� ���G\���.SBV���v���?����'.�HU��'��:��j�!)�?��&�,��M�� tN������a����qEy��n��5L���-?������������ �z�[�SZD��Z�v�98�����<q�4"2 ���c�Ho�N�h���^;k��g$��0��)�&�nS���*��o��pk�>�E�X�%��Iks�M��/+ �> �����������^ D�(�?�_��y�[���]�;�� �����1*����O��2c6�/������*�%�4 �f�������� ` � �{-��1P�7� �@�;w�}�hc0�������^0&�CGOx�F-!����������e�] ���"�A�M�����?�������{������� D���d� �V��9��i?|-<"#"�@�KGnvb7H�T^�r������i4Q�LG J���2 �-��S���.<�[ F,Zb�������[�=b���k�^2)������Kg�l�T� 3�t��g\�!��fO���Q{D$3!v*
�x�) CX�Q_�[g]z�E��2 3(������9�c^v����6$:�nEQ
�d�\09|�h���
�l�z �F� �j������I��K�m�_�v��zgR��5��:�XI4�~4v�%�U�X��>�v���I��[P�lF��u�8�\�t�����9�b�� =���}V y6����K�L�5{� zV�4�G�X��%(���������,� ~���o�2H����R��8&Q�a��b�. ��h��5��>�� �@%�]��:k�� S�����*�7�ukM���;���g��GD��R����[��i=�QU�uQ������ �> ��) CC����+� =��8�����b}s������X���co |�����]J�T����A>���}��u� b�c�Ho�N0��`Z�=�("���� �������=��(&�c�������?)l��s JG_��C��)Y��g)V�����EdR��� q}�)2!-1XG� � ,�|�����F��r ���
��o��9��F���'s
���5����N*��j��R��+��z�G���%��L��,�'M]:=�$� tE���z\����_�p��:��7?��G��o�=�*_�qwk�kqV��I�9V���n�x����+�� G'~z��8� �-}>�� _k�7�x�!���z���-�Z�������� |SX���sJ� ��~������U[�6�E ����� }B�_ |Iy��)����_�q���_������/���6�pK��o8���r��TK�Q�T�,��S�j�����r�h�
�2 0�f��4�"��U�!��=��5�]D�=����������8����L1�U����|�f�d��3��f1��?9�; �gl_��W�4U4-�����������""�c��H�f�cN�A$�uP�P�[�k�]�M*�;!8�tz��)������rR��.�N�% `�� ](X�� [D ������EU���Y��:�9����e}5�Q�WIV��~���^Vqh�� �` ��� @_*J���W<f����w��%���nzxJ�����=�w�L�kG��f��x��� �) S t�bC��u>g���"�1��M�}����������h��-_�5����p����X �D�m��F��� ?��m'�?ieo?������z��� /��k���f�+7�g,���@y1Q�EL+x���m��?K�� ���Cs% ��|�g� �����6���NL~[^������^�i����TMD�������������fw �E5 ��� �3�2 0��Ed���=4hLtO����i;.� z�(XGF���o�_������j� �i�}o 8�� 8%����*"c��G ��k
�W��Y��&�g[��"�� �u�3��K�gdV�]'�h ��� ����e}�u�����56u� �8��w���
��+l�_x��.9����
��i� A 8�*J��V u�z�i�u��(�G�Zp�/6X�
��~������JM��o������R ��T^p�������+E�Y�k�O>�{ p*�X|Xo�Y��PT��y����u����G��Q�m���M�>�~�HFB�����De7==�`�)����*{Z\���A�""����nA��9'}# ����G{2e �� mV���-���;%|�=�#"�� =�p�S� ��|���wgM��uS��X ��KqQ�P\\<�k0�� ��p4��-z����y|�����A9��x>w��BD #Mqqq�q~��� �w�F�9N ���
�[o��nedoQ��[J�_hdQV�SS�� ��|�����I/�1�����}^1 0\�����K p��^��CDFD4EL �m��&V��e�<0�PxDf����;f�;��m����^v�h���i�ZBs��?����� �Cz ��>�i �������n�L�Um�eG(MF�DS�i^YGm^�M�wm���5�ZT�Y�;�f ]R�Q��7�/Qm������K�,��M]2=w`n ��hO� ��P�k[��h���ei"��gj��#"���=��������4S���H������CW�Y���&s��PDFD�~���� [�����w�Z�i��-��-�����K��h � =��F��4 ���Z�(��m�� ��������^0&�#�r�����6���.�n����C�~���|/2
��xkf�x�u���� ��> ��� �,������ ����% �L��k����{������������Z\�~�8�q_[�)� ����% �g�5=���e��;�k�<1��������K�����m�� ��2 ��B�cbr��'� �y��������-���`l�l*/X�mnx)���R��d?se��g�~��� ����% �g3�dm1�]������M?t$��b��q�����<��$�<��V�_����d�#�����o]:����_<���_� �@#S �Y��T��w��u��[T����n���#iH�m���V��"��� ��p�.� _�- mS�� ��~�A 8���?,�oM����! �L ���,)]�`7/,s[|��?�� ���6�M�B��$c =o���aQ�)�k6�,���f$��� �N�) "���=�mQv���T�TM� f��/*�f�ZW�)�����+l*/hv[{��v��P��32G��UUn�d��� 0p� @?�(��Zu�����O����c����:�����������]_x ���Kg�������=�����2�b���Isr��
e ��T��#J����Y���5�r��u�������~V:+"(OA�����&-��_���0��t:]� �!��2 �?*64������L����1��V����D\j�FR������������� ��2 �?v|�>��"��}��V���w|��$;"f��:������{A�h���k ��2 �*J�m
��q{F[��������;�n���/�c5v�������.
?����^/_��"&�1��v2� � A 8U�����y(�D�^H/ �����1��3����d��������yo-*�����p�S�U��z�P��P����Yr���Y �f1��?��� �i�hZd�="(
% =y~bE������Un��4��I4Ee7=5�R1t��7~� ;��nv[��^"����q���f��H��������rR�� ����� ��7���0��������O���$�7?{�W�0Fu �� �9����^��sg'�M�wm�<�S�L��y��SY< 0������� ��P���c*Z7�`Y/[{N���,��� ���U����R��(^�S� ��E�����D������� � �������g��:Q������);�_�sD&� {���`DFD�Y���.\n�F��n��+�� 0�Q����^� ��9��k8
�������x�������{���
�1{-� 6����F���,��s��F��n��� ���� @o���w�5��%���#�)�
~��J������P�5{�_�~��c��Q% F�b�7�/lq[z��hSK�����u�hp}7�
��{�) zF� �K��������
,'�)�"�s�#8���)��/�%�MM5�l�."���`�����z���$WO�?��L���I����>��L��W�x7 � 2e ���5������m��k3�S���s*J��nf��U�~��7��0*��u���v��\ >�lzb��3�F����%���Z}���f�=S����� �?=v��}��F���}����4w �L��� ���� {oQ������iU6��*�����H4�X;_D��s�x���
�[
7L�]<��} ���� ��7���0p*64�}[����G�?y����'�{�y� n���WRD���>- @4}>�SS "�^������4�v�X��@�� �J_W `� (� ����� ��:�1�_���K>6)��r6�/��������^E4%V������ @Pqqq�q~��� �w�F�9N ���*�?��3SQ���6S��/xp���4���p�?��=����X�/��k����}x���{�[}Y; ���hO�% �4sM������{���3�r��2{F[����W?�{J����wf>S;"#"�E�_�<r}nu_ `� ( Q����:�-�z��R��'"���dpt73������4M�C���#We��=��c�? ��BP�H�o�����?��`"LZc�U����g���yaY0"#��"5.����.m*/X�mn<��#$\�����I��u\����/ :GP��V��y��:�3 "����+���l7�������~Q�������P4�`Wg\�����J�Ye���>lS#������'��i����z\DL����+ C�� �h�W7#2A>g`���/���k�w�Q�k��r������>�h3����_dc]����E\ySy�1g�I,)�r||������$� `H S����:��Z��# )�"b0�W~��5+������_^5�>���w��}�}a�3���Kg�������N���O������� �#(`X�.��|s����������"�gV�!��G�=l1h"2>����C����}���z�`.���."�ZB���_ �iN�{���C_J����� -e �bDQ
��v7���w~��I7�������
�'�$���/�j ����Ggm��k3������wT\�Y0"�y�/�,�������k�Q�����> �z��5��n���������� � ���i=7~�(
%`hx~bE���������>-�9yG�����}��6�j�6��-c����E�����W� ��u�M���T��C���"MW|��zSy�][/����"1�lg$��=�v_�n������=] @ ��hO�_ �Gt�������?u�"2"�OP-K
��`���g���d�x�&���i�<C�C��tv�]����oj�v���4k/� 0d�0|��"7
%�v+i��6� ����� Y��onhIv������ ������T��I��z���v����2 }#(`�������GT�a����i%���+ 3�3 � ���w����q��m�E�MO�6( ��M�+��=�$H�"c��� M���PP �� ��P������{�g�^��ww �����%]��+;z^m�s�j{g8Em���o
�}�2�����t�
��-"��U ��)��K�S�W��b����no�v� �a��K ��� �B� �D$�����
�+�����5�q��6o���4� wh�s�I���B���}���H���X�|J�#����i|F��������R~��-"���2 ��FP����MD�� ���!��LR����.��� �Dn2��Bm����U�)�KG2^����Ex�^u��sy9���8
���kQ�c ��Y6#g����e5�7}�t��m#e ����K ���J�mn��&�h��N3�b�I��}�����n.��(}�Y����������K���3{����&����N �1�i����Y:=����MHO4�� ��O\7��2 ��F� }�9����MKSE�v�:����rLE��F���������y���-���EYMO�/%r}� (��:_�n�Z��G5�k�O���2 IDATH�y��s"�Sfz^ �a�� }��&s���������nn�(��4�4UDs �������8����#�f��8|I��7fD&h���7O;���5���P���WN�u���%��� #A ��s�K����h�w������D�u�'��9���sk����M�������� fm��������
��% @8�2 t�`YZ����s{���D���F� Q\�/�����+C�^in������G$��Q���sU�����i F
�Jz���DoD��j0/��;��� � gwzi_�������ra��"2��=g����q�S� @4�2 Vw��NN������<�l0)���=1.��)��J=�ZR���jE�+����.]�����Q^�X� {w�w����b� �k�6C��� �@���-�R�J�I������.U{/{2������Y���+��9w�'�{��q�� �k[������Q�i�� � ��;����^ �S��G{2e ������*J���m�������=�[T�c��s����`����[R�#2"����i��OnI �7
�@=4K�G�}�Dz
~k�dDD���|����I.QD4��e:��L��Kg;�}�$���mW& #A ��fIq�(�wi� �+SD>N��U��7>���%��V� ����2��g �� �
�T�w&e�ZW�)�~��u��>-^Dl�����z ��@�[��F���!!����f����Pi��O��$ME�}�7kz������j�-;�_�����;[�����^�E���1YM�L�k���������5��5�k{���&bT��/����s�Z ���G{j� @=4K�G�%i���������eQ�O�%�5�� ��Z�_6�����G�Tr�'�)qFdDQ4MD_@{��z}YM�� 0��} ��*X�s�Q<z-I��� {F����U���-�z�i���w ����2M�R,�����
��3����N� E$< �t��{�����'w5 �A ��kI�a��I��3���k]wNu�j����S�
�S�dc^N����B�b0�$;*�}�%�,F���
tI�����z #�� ���5�F[����f��63��bCs� ol���\U�r��P _����s
B� EU7�?��m�a���cJQdBz��M;##!�Pn�� �@� �*X�*�������P?&MD���I�5m��p�%*�&*����`�[s� �X�>����eDq������K�����h�7}�t�B��� &= �}�hF��~L��3�r��2{F[�1�g�o����Dw�����[��n\2)>�#�m2�(������ft���|P�jKym�+� �A��G{2e %}���������|����j��.�O$���sD�������z����e3r� �5e��|���x���h���z����#2A���n ���Y9�-#&�s&�b =+..=��Zb�c����3� ���T��)&c�!a����Oj_|�.�j|t�a������ ���]�L�����S*��r����21��� �s�|�����F��r���^Y�Z���Vd��������C��1i��W�y��w�)"{�Tm���x�#8gQv�o�9 �w)dSy�][/�`,F��������tUQErS( !�|�����F��r���^�����E����@b� �;���"���j�-;����E�����e lMv���Y������M)����=�8:��{���� �|��� �k|��PDFD ���7<|�yaY("#"n���K�[�]��?���c������g5�X��D]��g�vW � ��} �`�(��Zu�Y�������-X+i�W+j�cB}�E�hgGF�y1�"��� >�:�Yi|���i���
� s����n�)u��� � A ��bC��u������+�DQ
��v�SbWNd%�= >�j���EU%��G)Z�����c���M�+��
V�9�������/ ���v��>� �c��A�{uC0"�s��A}��`�f�����B�~w�?_�CD6/,����H���/����R�E����,��� �2e �����I{�����?�����T��R�r�#�������^�^�����=�-�e�>u�9v�L_+��� �@ (`�$���?�D ���B�� �;�V���z���������O��e�D_V�>&'�Q��^K"b4(7O�#" ���} ����&3����}_C����?F��W�O1)W��ywTdY�Tc�4�g�N����M���+�^_V��� �We %�@oW�����$;���M��?�E O������%�tT� v�6:�2�N{�[c~s�����q��I�\�Yj���t��{�@�N �^�} �i�����n�QR�)��L������i��){��}�l{������%9��}�"z-��6w�>��u�":l������) �3E���vDR�J@����������K1m������_
�gBG ����� &+/$� i/ZT������k���j��M��r��F'~v��^O �O�|�'S�����2��o�����?�O���.I�`.Z7�`I�h�}ygGS��*I����
� ���<<��� �}��g����$a� AP1 R\P ��Vy��b��B��u������������h����H0���l�e�y���c�d���L���?�&g�<�������9�=F���� D�>��k��D����Fa�9]_4�5�R��> ��P �����k���[S� ��WkN<,���*N<�8O:&N^;t���;U�T�2t�_���s��_2���QNI���-?�6����doz-�$F�V_��|Kae�5.D�vv���� \ ���{'�� '��!�`H`�g��a�kq)��D���b���}���t���;%.:D�o5�<������%���� v"*��c������
S�n���)�?������6 ��$0?��� �Jqn��%U-'�������\R���j�T���������
M�1#{����=J���5�����:+
�p%2D�����N�KVZ�������6�cd$!� ���K �+�21N��4�C���3��X{����,1�I���� ���X�����_���Z� �&�����:C�� g=X��0%��h'�/}f�U� ���zF�_� z;e �W:i\��v�h����� �i�����[ ����!����%L���u�T:������C:��T�?�.����|��k�� ��A( �� O=�C�JC���A�|M���g�3�k�M�O���=�R�G����nt�:m��m1��1�L��dm����H��������������i��]
�o!��^�|u�g�Td��&��@�I���bS����%8�)��tC:��{ "8I-���:F������#�E����e#QJc�`� �(�����#���9P��� �+�) ��� J���/)t��Yl����8%� v��&��WG����1��~XPt����
���L�\=�=%��N������U� K[�{Jek#����0#�� �a4���&�Z���N6��|p�1��*� 0 �#T� ��Y ��Q����EP��c=G�s�� 9e�uQ������Vh,��D����U�{��2t���)9%�l�47��C�N��8}����F�<D-?����0���f�SMF!0Q���P�vN��p�p{��� ����>�h�_ � ��{��| 9���-oq��)"��ma����dl {�����
C�~�p��9�W)
����\Qr_~^[lN�hq��O)'��Y"���{J�a�L�� �]si����S��� ���=�/@�aD��r�� ��Lu/2������!������~4�����q1�/J�;,eA�/6}ar�]q�����>��cikveT��c �U�A=~zm�������� @�A( =W�ctU�1��&� �yE��xCS��o(tV��Ro���-�P�7N�_�Q��8X��Q�1G�M��xL��G�M�)K�g��+�5���~�Z0;����E�� �[D�/ �]�� =T���sIU�I���-'��Ur���]��f�z��!������;�����u~N��UO��_ [nt(d3�������SA��}�
i��`�J#��or?���� ��� @�[Q���DD���|���i��Wx1UwQ�%(^��|uw�i��
�3U�3{d�R��QR����#���*���c��F�|���jz�.����M�x�(C�� �1 _����)��\�V�8I���f_���J������9��W J�p��hk�����v4;!���EO�O�s��Z)�.�$"�q��U_��3{N�U�*�#���o�}�5�(��� � B �����kSL�I��%U�XJv�:La��� �[�G�����6��5YN� zvN����9�g�|��}�AVbA��^1�F�����JI"�@N"�j ��]���(��zV�����lm�B/X ��z,���C+u#� @ ��P�&0�f �w/*n���9z����)o
+4��ti������ ������dz�A��� D��yO� ��u*��',H��~*�@��-����a�S����_�?����i�~ �s��} �)��� ��yU{T�u�Td�S�����%���� �
m�
>��\3�qK'/�+L}h�L����?����o����P��(g� �e���[O�m> �C��=
��o�9�wF��>*y�T�D��� #������=h�[e�q��-+�(D��J8�_��r
AW3U{e_�2M\��� z � ����Z�V� ��>��&u��9�S����1��>m�]�7�X����kM�.�_n���ndvh�:ePT�����u���, �gP� |���&"A����j�t����^����a
�GU�h �6���1OE���V:�����(�K\���l�����7]�B]���. �g� �����@�c [�c���R>���m���%��-0F���7|<����v��q[L��c�
��u���s�9�j�{8(n��"y�� �z�S�k��A�\3+��� _ ��;P4a���������f"�:�q?�r�{w��G��
�>�5��|��
w��$��?9�����������T�M!�<���#�W"�(��^7�w�% �����M`�h�W���Z\)�:�Sb���&��e����#3��f��L�DL��,�!�K�����6�D�U���� ��!0?�c� ��� ���!�LdH�e�-�?d5��;�!"F�;�!���`?ogu�u� ����2 ������Fx��6��� FD�(��G���<� K[�{J�)(��������c���R�Pn�g}_ �A��2 ��wY�����D�$����DDJ���oD�p��)kq�BYs�����;� ���*�.��Q*P� B �����JC �����a^l������8t�����58��m���4d���������z��"�lnrS�����R"�0��} ��� ��� J����R{^J������#o����}�w���h�����W�*t�����W$l|v��?m�XY��)���F�J�U�� �`��_��V��� ��� �;#�_Q }gDqq�������^"3�����Y�]��[���i�(<x�I����|���JY��1�����>��l.�Y��,i�U�4
���}d� ��n� �pu�lZ����� ���j�*����^�o��*�f�,��(�m�y_���z(
����R�C}>���[�_��{m���U�����[t�sN���Z�_�~�(1� 7F6�I�k5s��8u{��jnr�|�z�c���
��z��|����� ���n� 8/�G{����[Q���8M���5tf�����Md�(��O�� i��/.�����( ��FD�f��w���I�e{.)�����qA� JZ���� ���:�b���gc�17x[����} 5e .D��z+���������m= �C�Zd#�Zq�,��tI���+����B� ;����O�wf{����q�-%>w�0�8��6������^�N�'��
�����s���s�� � �N� �w���x��G�{��Q��.�J�J� i_��
���W��ME���{�g�����eS�T��#�`�J�<+�Z ���E��J}�|M\������ �_!����]+
��k���{U����m ��m{����L""�8����~�j�DO�O���~:?#+���;��\'�^J����<%qy�Sn�� M\M���4.� ,�2 ������b�����'����MY o>�]�'yn�Q |���O;:Xq��kx�����������c*[�������������_��_k����o�i��q$�sT�nd61��Y����� ���� �������n�47��f�s��;����t�����xtVbD
�u���+�"�47�8D}��8��cb�[��������&���� �=;69�@�I�(�g:D��������x�a�����u���^�+ �~��2 ����K�7�lmtHb%5���]��(�����(Y��&Mb��g5��8���D��d�c *� ���Tn<ADz�} ����KSG��!&����,4q5S��Le���G� ��a� ��hE�&G{ j�����C��M����g�|6��ak���q:���{�m*g�U����|�� ����"�l��)���}�i�o���kF�� r�(1�����l6<~@;���
����x��=��2����xk�r�\������%/��M <�y�N,\ �w �+�9�����
�sP�T�t��]�v�����w�4m�^46���l������F�R�W� 7[ %����L�V ����X�� ��\sg��Uk�/J(���0Z<_82RbYfon
����h��K �� �C ��Q^g�=����
��-p'2Dd�8���DD����?-����J�cZ�*y�;�!"�P|"�y���
G�(1��������C
���~�r��T"��������[j �Y�m���� e .��M;�TYk�D4���[{���)o�8���#���3g��S�\_����o�U=^$�Z�V+�����R���!�{�����BtZ��%������ =~w�u���Z���x�Y\�eW%r \�)p�����ij;[t��d���4��{G�s��zCmt��� �_h���=[���)k����c0��2?kt�k��av�� =$>z�V�� <5NKD� 7Y��M:V�� 7���S����[
+��q!����N��� �B� Ek������$_����������s>��{w�� &�p�����Vl���Gvd�D
�5�,��I�g�qQ������zN�S��P����w|�+yi���k�{�t"��b�GL6'�l4��l��� ��� .A *�'��\��]��Q��B��j}�����K������|g� 1���1��5 �v4��+uS�(�Eskg�����MX�8���Lx}Q��Q$I�b� )|����D���P=�?�[l��* ����{E���C��������G6�����{Ex�tb��"W"���3m� � B��Gq��������3��8�({����
�����}7�=���)�>R���j2��4�m\Y��Nc:�z�h����p����
S�n�,kq�BYK�owd��!����w���9�dk�h
>���?������� �bBt*��nx6��n ��k���f��s�����J[��%���>J^��Fy�T�5 ��p| �<������r��sI1�� ���� ���
?��]��9���c,]��S[�HD��=�Z�<�- �,�e �����br�G6fI�B�� �8��M�^C�^?q�-����,y�bl\P���]K�d�������*�I�<I�D?���"���S
W�v� UB��D]�w ��sX ~�N���g _"r��}+k<'�69��6����1���������=�#��S� IDAT���w��*���z���!���k
��8[Or��l=��g�B-�Pck�v��������b�����L�N"CD�g��l�
�O� �'du|���g��i���T�5_=�/!����"����a
��:Z;�;�E���m����"� n��lwJ�6=-�&�w�����#�2���C���w#'r�N�_7oBR�^��m�{p ?���9�8��w�i��� F���v:s��s M��{f�/ND+
��8�-���1�E7.HD�
S� ��B|rJ~���f��{�^�
��A��0�]�Fk�r�\�����5�����9'(�%�=-��6� 0`�� g�`1��������Uv+p���3J>�:h�0;%� ����|� � K[�{J�)(�����������^��=������M���_o�e�m��
��iI��n�K;��������H�������a��q) B ����h��K �6Y)�>�m'��O.}������$r���|&2 :+5�,���W��&�w��Z��w�3��XRtdbt���v�~XIR�&��MN���.68I-�Xp�z��X$2 pN��%�s�w)����T(u����y�A���=�m1��[�s������z:?#+��=�T���N�l�qp�s~������l�5�� ��Cyu��2h�e�[�z�OY���mN���YkT6[�B�kg��8 ,�2 �6u��R���+���&����-�[�]����/�����-�a*=F�(>:����� f���H�l���R�v���y����MS��z����% 1��Dt��������l�� ����K ����&�����|wI���7�lmlKd8Q�R<�|�J�gF������f�� cr����)�GK*<GjL��U���_�������l��[��6 `pa� �9��_�T�0��&� �>��oE�h��P���6���m� u��,��g���L���Yi����V�0e��Av:�p����,�M_�������
�D��0N>����`��4�5� \e ]qn������51�� JD�e�UP��#M�{d��W�L�| _�J+"�<�/��������1����{� N�%n����r��GGZE���SnA�(�����{pS ��P ���+9M���5)����M\���n�g�|6��ak���!a^l��i�C�N"��+��m��x�2�W��g�B��B�6���y�g�_(� ����2
��$�z��ST+��^�7�~ @�� @�����Z� �}+j���.��\��]��Q��B��j�������!g�.
Q;�6�,+���?�� AP��� ��6�uk
:��_ &�=�� ��*�d^w2���:ym"�0�8��>"�[k 0�)��TM'� F�U����?�[`����}bd�F��h�,�:R��y�%.y�����R�$I�w�p #
W�� �H������_���WO�c=���*"�*��� �I���3{��4>g ��t���V�U���5�D����A�3Ft�3��>�d������,5e6�,��Y� "JB�"�DV�������M\�T"���\�����1f�������*��������j���*���k.�9 �t)�����'� NRO_�� ��f��]�&u��NDTa�v��T���V��d��k�^���ndvh�:eP�����7��c�8��S���� �6Tx_!;=~�� I�z�BH
��5����#z� ���8��B��%��]��q�N����� �G?��'������
N��e)+���c�d'�:���6�G��n�k���v�l���w���\�E����2{y/ �p�G�@\S� �98��6�����wIi���j��9��]�n�8��=��^z�%D^%���S��c~N��c�*�zc�����N[8��k���mN��4R*���l& �O��� �~��N����E����J\]�\5}��Z��+_�5�#��D��kxdd��� -��g���9�DD���F��3����]��|9�C��g �C1(
4���y�U�Q������B�z�G��V�����-�#�YDo�g*� ��tZe����tS�6�ZW\��5����� K���}���y@iCA�����l�.g �� ��S @yw�fD�Ld������B97���Q�qZ���:'��)�v��5�,��I�g�ur��� |J=A��Z���M�bCM��}�Nvz<�l����f�S���g �C�} @�:aQoN-���d#�����W���+X{/kY7%�C�t~F'W���()���V NYslN$X�L�|��a ZY/�W�<��������g �+� �� ��'k�������}F��g6�G�� �px����O;:X�)����RU���v����XI�������Y}���o��}%AuZ��C�����$5Ja�l��F�h�8��m �vG��J��V�J5h���4�5�R�z� ��� ���uk����/}�]�������%��/�-��R����"���"��,��]?l�����N9#�)�
:�Y��z)����kU���b� �R��~���7��Z����K�z eru_Z���K pN
�:7�&0�������;���{p��#� ���f�����%y����� *��3���V,�f��}d������R�'��
��iIuM-'*N{��3d��b�)CFf���c�%���14� � ���8�(d=�����o��!{*^�����:+���;C�U
)1���;��F��N/�'�!"� g��d�_�\�T���k*�
�:$�n������� > _������j������J��S���:iQ ���Y� ;o�� m�����~���p (.uJ���g{�o��<?!l��:y�(���rk��( �9
;e Yq����������%CD4��� �O;:\g��)m���!� �*���ci�/ ���wE����U���Z�STz���D}���BV�W
� ����2 �����~�GC����)k�q��2����/'�8UZ�k��,���WB��M�)K�g����KC �|�g�s���84J���9��s|Y���/q��:|w� �e Mqn��D�Q�3�����w
U���/S���M%~�����n��E��k �c\Y�n����/3y����q�^���[}]��
d���F -��*������MG����`��,?���� �Y;i������# ���P`��~��G���D��j���E #"�P���K/:���/����/�e�������n�
Q����YQUe �z��J �l��T���,H�[��A�����i��[rpI�&��ND'[��ob��G"� �@�P`�8��[�����S��o� �
��b�/�[� A�D�E�������Q}Tk�]9B%�]��~ob]7�!�����S5�����X^�
����FD>�?2)����YQ���L�a"29�+~�P B��U�c���td��+���Ro#"C|���n��G�S_�Hy�n�"���1��>�^�����'q��o����kZD�?�`��5�r��7����V�G�d��#t�]W�Y�]��l��E B�~T���sI��$]��f�G#j��6����x�����%��-p P������DtE�����T
�0A�x*����_�5��^�p�G�� �z�,u]m�1~w���O�{p���K��9gVj���2�����S����}����}����� ��C(��v=R�4IDd����0����t�������o�8��>�u�����Y�
����74���������e|'�n�o\�T� {�xa�� c<^Y���wo
��$R ��]1����/25Z�p�p{�aZ����(�mOd����E�h����������}���R�I��� xe �Kqn�������*<xD���T�V+�)wu}>{l��#����/��LD��-p%2.?�?�j�k&|�Jd\��u�� �*yD���]�T����_�� ���v����V�Q�Y\��q���0Ze�-��R��t�������K�{ ���P���[Q�~|�/��x�}� &�Ys��9�����C��#�����K�&#�������+{}L��������=��ch]qe>��=��5H�]���Uw8Ldb�l�����<��ij�6!L+�) ���(o ��� �9e �Kk������ID|�c�u��,�B=;��cC�������C��M�7�F7{���U6�eo�:m*LY�=��PQYs�C�f'�����z������y���S��he��IJ !�Vk�h����7 1�����Uv/ p�@(�_�TM'�[5�~ti�G���:��&W���51��O&�����k��)U!j����{*�+�q�9��c\�S�8/lW����K/�Hj"�pD}P��LS�04�5��T���r����@�vv��y|�� � 0�����y�1|��'�74����U����O.��}��� 7k�6�1��3���9��I�����2����l���]I����/����r�J�+U*���9!X���1���j���t�m{y �k���N���^>�N�vJ_�J ���5$��� �<� ���^�����j���^ �;8W�{�g�>�[ Kd1��^ �**��`�C�4��9�'GV���6e�E��X�]��ndCq�-�Z��_6��d� �mm'�~(�p'2Dd�9Wn-�����~~�!|M% i��k��5��& �V�Z��8?�k�-��@�q�������v� -}�_NY��� O:=����{a��0t��DTv������+L]�m�� �^� �%���(7�l�fP�`}fNo.n��j��p�[�a#���5 �<� �QS�_�[Q��D�����2� ���5, U �qT�TF�0T���M��k� Q���t)+��������s������cn�� ��**�8|C=k���nj�I+� � �P�_x�^���3J>�[���r* g��hlJ�U� j!��b_ S�:�����$�Y8�& �jI�z m*L�=6���z���g�&,�d���jp����'���]������� � @M�~����?�'��wW��j ' ��U�����_�3eb�pW�!+"�x�C(��E�D�4B"Z���N�.��s���l�*��<����$��������U~U�\l�\�T��zyM ��P�_\�:����ln���59����~���V��6�Y��b�&�8�|3N�J��.�1R�� ��T�Rk�w2���/~?,W/X�#� hTJA`�&9&�{�J! �/]��pR 4�6$��k�&Z $I��Q�2�0���_ ����@�H����*KMO��#����!&"�>����������G����� ��;��f��j�!��j����$C�����������s�Z� ���8l���Ow��g �������J���L}rj�mw�������C�/�
v_�uj\oK� �7���/S��4��W,�� �L�x�w����Ndn�i U��(���������3��C ���9��m2D��j1���wi��i�/�L��Nd�j��v�� c�u��y���*��W����?n�z ���<����. ��S���d��;UJ�� &J�1�nh�gaQ����gg�aszrd�F��vnO���V��Zg��MW��
����?�Vx�O����Iq}rX�[�E N�{� ���G) �����\�o� �N:���-�9�_��2=�E��%���z��"�3���?(��
FDe��[�6��}xR��!:y�� � ��2 ������uX�����/�����gs n ~z�������Z{<f����ZE��e��9�� '~�Q�>i���K� Q
B�f���[�������N�/*���T ���[h�F^?��>��U��G�� �2 ��8��w��jw�j�T ���]����{�s���[��u����8�� ���+�M� �f��H����2�����������)������>T���u�Xu�f�� �9Rr����{������%���w���ew��i9���L�V)\+��j�2U~ � �d}R�c�.A���.?����:�����uC[<G�p��S��e�:�xb��sB�%x���+��K�d��EJ���?�
S~�m����^�>��){D��0�Fp1�Q �����X8��wg��j �D`~��N��W���sI���#7�nw-*$��2�Z�lB��ze�����T
FG����{���S�S<���%�������u������ X�\Y/��?�� 0Tx � �~���5>�����l����g�3���(i��!)��I���U0q�������Y�wY�*G��b^�;|���8��L\k��h���
/ >!��{�egm�4����� j���Zm���3�h`D�g����[d#!
���7g�����2D�����g����(�Z�&0��'��6��+��� ��+�T��<x�8�����N&|�Q���]�B"~�����8B�3��K[_SP��u���Y� ����{�����,�%�A�����7�tQ�����>�T�� �\������];;m����_ ���h��2 }�UM��9��If�r�n����p"Fq��n��������3]�b��C�n�$����<��� ����3����_5H!��))���1Wn������[P���#&���N6��|p�1�� ?�+ � _�K�T�q�>����8Ts rm�aDD^e�� K��~q�����~q��4�������z:?C�Z�����k4iYI�9q�>Ld�(>lN*��Z�Jd\L6���E�� e �RkE[]�O.��hw}d�$H����=]����V����5�8D�� &�0���w�J�""A����L���u��H`g-(�RRgV<����;rTv��=U��=gP6� t�/���U� yOk"�>�����3g\�z��sRt����v�����c���X���.��M�i�������uN��Icl��� kZ�3u�9�� �!J�����h|P6� tv� ��� ��I ���e�/
V<��_w������N"zrdu�{Q:��l� ��}��A�xrJ���x7`�'���SE�(I
��K
�?�1��h�|u��3D�g�4���A�\3+u`n
�e �Fqn�W�VZk���#�\������*�'��/�-��m$N��5 �v�v������ ����e�_�<�� &+���;C�U
)1���;nK+�y�k��u���k# �"�N����MG�=��� �����u�&$��U
!)\�n���� ��@�h�o�� ���+�(�9�y����:�'go�Y�`P����IB�(�(�j�J+HC5�N:[��.�6�}���r�<s����l�T���m3=�@������� �[`~�GM��r���Nd��(��� T2�8�N�8��$� Q��n�X��i��q���YiE��g��3��j�}-��5�������! �R E�&0�4 �^T����������w�>r����� ��i����B=G�������E.y����E���Z�8��pr�.�(u#��2�� ,1T�vv��Iq}v �� ����)�[>�`6�����P�������F|w���y^���"������
��t����-� �nA(�[>�`#�O���t�k�'�;[{��W� �Q�r� � t_���WG+
��J��k��
�.[)��&�gkO�J1gB� Q�Db�n n p>B(�[)�����'�kV�Z�����|<���u=]����������8)����R��}'��j� ��unM`V�@SSSs���.\���'�xU{T��} ���I?�[_���7�� zf<��m"�~�a����;F�/�<Ed ��� ��S��>}��k����~��'�((AED�B 9���@�#Zv�.�COy��&�_l��>�Q0v�g!� �
�2 ��Jd,X��{�F� IDAT�O�F\Uf��
OE�Z���[��y>� �COD�t{fYsHd��r ��/�����333o�������S�C���w'D\�m�b������]�b?&����*[� ��:' >�w���=�C�m� 8� ������������+V��JY���'F��SMY�G������
Ec={0��w��3�A�% ��C(�C�Df��y�� }N�/����x�jYqe~���^�]���P�������O�� �. � �2 =���x�����={��U��:���-o%�m�/e�Z�n�$����z� � ��
����� %~ z �:����q�������<�����Wf�K���{�{W����\C����+�Le�u������rw �� B��1��]w�����}�Y��Mo�x�x�j���>��ci�������+�'r���F��o(�� �O� t��h�9s��W_��s��m�d#q�-��X�?�NB�Yt���� 5��+���� ���k��UW]����w2�)�d#���1� �/
*��S�{��I�~eP?����W)�l��88�� �(� �����Y�fM�:��^�|�6����g
�=��VD�yv_�-��g���������""�qD4�� '������� ��B��555]w�uS�L��_��������'>�V>D�tf�.�}l"� n)1���:]
1t�\V�J����� �+6 @���@���g��������/v99�����M�U���9�������){�����
���/� �N�_7o��(SR�~�� ��`% �1�� � =w1�������L�0��W_e�w�[Omz���z""N�g�W-H}���W�?m���e=)��[;;m���~� ������)pV&���o ?~��� U��� 1���KE���C��{I�8�^ �7�2 ��L�9s�����[���D����_^/��&i�|aYiE:����^C�k �~�P���|�
7�5��=2.�/����g��w�~��e� K��~q�����~q��4��S���\F�{
0��HU� ��g�\�Lrr�k��&��?��������V�(�Q`=���w,m��)��A�?z��������w^]o������2�_�y��/ e}�GF�O,���{ ���h��� ��#F�,�!��\~cm� w��������T��b ��F�l�q��!�Gcr�����&��&���mR��]��1��� &��@�q���|��7 >|���=Kd\��3��wiSa�o��t�0���������s�r����fk\��� ��� �qM�&0����Y,�o�1!!��7��M"CDM�3�o�t����������
��_^� =~�o
0���=�/Y,��n�)**�7{drJ
� �Rin>�I
��.���V\��5��s��5�o�C7�T �A�AQ� �8
���f�7o^PP�����T�$��))x��Oj�&"�?��� ?����N2� ��V}]6��3%0����I/�87 p��� ;e�Bg�����g0z���� \������m1�/�)�|���)�U{M�����g�$>�ud�
�
�S�
g ��\��^�����Y"CD+
�p%2D���j��C����`� ��YF���R�[�O����� � �� ��n�����Z��
z��Q����8N��=���8����k�W�R���#q���:`� �2p����YYY�&77�7� %��{'UXU�g�M�cP�G-��CCgO�g<��������������v �P.D�DF�R�r����K�3��D4XC�R�=��V��� ��I�!��' ���R�-�o ���+��VUo��
����p��?_�T����T}�3%{d:g���W^]n�U&kta��B��o �]�76���FE�P� �CY �A��2paq%2� ���{}���,LN<��g"�-1�Y6��CB^_-�Z�R�A��4uDJl��� pS � � \@DQ�����v{_��qs��'��~N�U�a�� ��M��>\�O
�}����"����LJ �� �W�i��� p68�
Q��������>�H����� �����|t�>���"�l��)���a��?D�KDW�'�D�� � � �H���� �� �(�w�y��h�����6�!"k�[�-]N���k�>2�I���{�������<UCD �z��!���Y #���I�u\�� ��q�����`1����M����������>�j�>�h������K�}u�k��D��F �����[2���X�W`�n�% ���� .0?�c� ��DQ���{���>����Hd��)��X�����\�}�I"�����cI�1$2 �~�|�JdN�>�� iC�&��xcN�DDy��&�_ ��o ����q�
*�� p6��{'�� '��(��-:u��'�|����~A/8���-oq��)"����6�B94�~�_���?�T �� �����pR�`/ `��G�@\S� �9��$I�-��� �D�[�;j� �?�����u�4JE�Fa�:� �%0?��� ��8�<����'?����Od�H�\
�D��lN��)��F��3��Qn `� ��
���Jd� ;�y�f�apb����b�9Wn-�U \���y�s��� =z���?
�5XJru���L ��)�X'���[ � ���������?��e���Jd����
"NDw�:Xk ��)���nx{�D��������# t � �'8�=�����q��������O1�N3 ��z��)m0Qvz��y���*�ePk�
��F�fV��, �b��@�%������ >x��_| <�+�������� ���?k����R�>3G6�s�r����f+�/ �%0?���M`�����7�����n�:�� �~x���'����~��w�wV��?����X�9�7 L��� �����9��o{���A�#��4U�Fn
�F���|�� ��:�h @�C(�0����K���7o� 2��!"R�d# ��X�o���3�0�Z������MX� ��7 ��8k[���l��]�vm��=<|���4quC���hr�<wz���]%)��l\;k � ��P�U��-���/$�����,��u������]]���� 9`�(7Z�^ �K8��v��U�{-@D��Ol��u��������6�Y�l=�%����$��� �h��oo��2F$���E��i��^ ��U�V�?��Z| ����&0K4_��|���?�|���C� ������"Di�Y�M �z���}���U*D�]
���o:b�9]O4��o�R2 ���=v��9f��������KAR�>MV�����;����1b��N�!J'�K>8LD��MH
���/ ��(��@�q��i������w��8� �~0��rB6�d��� ���?<E��/���� ��j���+W���t ��`�e��G{���s� ���O?�4����M\�T��������w e} ���g�9x��?3'O��y���^�Kq��������3��8�807=����V�\��'����#2r���I72�7����e��qv��e} ��b��l��^E��M;�T9M����\RE��d���B8�Z�j���;w� �D�E�����;�m�������F�fVj�� �� bcc]�9��|���C����\#���K�.=~�xxx�^��������������� x�����/� sM~��Gf����rEE�c�=VWW <f��� | ���5�D��i����A(� � �g�}������s��a�{-��C%k����
����K����� =��� />>���?w=~��W.��2w"CD�����>:m�4�XKKKff�M7�����y��U�V]���'O��l]]��w��n����"��m��?< o���! )��\��@@{����|���Od�(���4�/���� ��~���4�V_�O �����/� ���w��- }����f�9>>�1FD������jrr�?�]�f� /��Jd�h��������m������j:a�0�����[P����?��o���3&&f����o������ z�I����I]��� ��7���+ @ HMM%���O?��co���B��|���^[�z���3o�������III!!!�\���4==�s��o��e���������Ai._= �=�`� �^x���_?'�����GZ�"�U��/�V��i��M6g����=)~A��o����(���_�Y�j�0SS�6��}��MA�4�B�����c���"m&�f�����n�Y ��!]e���~j�H� �t���������E�A��gny=��g�9|f���q�����~7 @S*++CCCW�\y�kg��o�y�f!����SRR�������t�R�6KJJ��1=cV �� Y���-�R��f�����/3��!��-_�<>>~����1��l���R� ��.������,D��_�O�^),uk��$��� �) <h���G� ��[7!�N����swwB��� !<<<�� ��c������eEI� �;�qww�������qNjj���D!D�Q�;��D d�x| ��f��}��YE"#�����6���,Y���e��*���|��I'>>�c�� PKRR�N� Q&333��'O��1c�,������������>���)))J.#���������������'�?^�<|���C�,sD0M2^Q��$q�,g���W���� [�in\+�#��t�����9 |c:�j|v���� ��z����������2'//o�����������dY��]�v����������c�W7 ���iiinnn�������f����PZ��??o�����f��u��q���m�����obb�qw6m���bMZ��+�@Z�r��+�����m[�k�������Hy���o� ;uQ������M���� X;m��gLhE||�5&2B�>nN�$��)�/5���b�D��k�[�������to�T�; X � 4a��uo���5&2
��N>���N������ ���G�K�� ���;�c_ �`�_�o��u�������������<�p����$����vx�����mm��6^;� �� ���2P����������� $2w��x�����hg[�����l���m��z��l x`�@M6l�����o_�v������|v".�����Q��\ mK+
nM��v�$���IU5s�[5�K} P � T��G -^�x��}���W��z���D�!g����r�[]dY�H9}���A�!�� ���?U�� 4@�o��m�7��m��122���������������������%IY�j���'k��Q��W"RN_),Un��y% �;m���bMZ��+g�6m�4�����?���j�R>9���wz���V*w�*% ���O{ _��m��9<<����~0!��=g�0 �$� �v�2��������?0��(rn��>s�w�U��b�b ��F��������9s���{===���>y4w�6�}���_$� P�����}����g��F,X��}����z, ��P������1#--�Kd���;9;�q����]�'� �@?~|jj���k�{ W�\�|�rhhh=VeZ������y���F'^���Q8BX�� ;�O��g��'�|R�Z�_pw�����umlok������]^���vQ P��%K""",��O.d�~dGvq~�A�]���� �'��/�)���c���S���������Q=�xw5 u���u���gY� <�����sg�?��c�F�:t�`�� O-�(7N�T��������
�>B�WRR���S���� '2 �:ww���d�s\\\��=�&2B����U�VY�� ]A�9�u7-Y�����>����'���<��Sj� ���X�CFFFJJ���;��~���]�v}���,Y�G����~�:��q3K�}�)s��s�� ����w��4 0�N�: !rss���6n�hkkkl2 �����i�,\��n�� ����Q�,\��q� �"99y�� �w������� @�PYY�r���-[V����'N�1B�<{���={�l���m[�����,���S��
�7]�#��v���G�C(�����2v��������K�Z �� 9rd�n��:�.//���]��K��/[�d�������v�G�������%��������/���D ,&))I�����(����������I�,�]��Ig��RSS_}��/����g�Q� xp ;vl��EB������ OOO!D�������z����������?c����
:t���U7���=u��3g�<���x|IS���^�5i�6�����g��1$2 ���O{ _B�HKK3f��]�Hd �
B���{����|�����?��v- X������wopppRR������ `5�Su��7�� =z�� }��U� � � ���C�F��}������j� ��!��}:|���/��m����{N�Z �>�2� � >|��m�����v- X%B����_�����OHd �o�2�7G� 6l�� ���k ���� 9r��^HLL���U� �� j�����?��G }����v- ��$$$x{{/Y����P\\���o <8 `��%%%�X ��$���5h�$q����?x��
6 <X�Z �����HOO�����ueY 6l������ RRR>���-[��w�wp�j��3%��W'����}Z;Y`�w�����)�� ?~|��!���'� ������3��� �$�����������?Qt]��4��u��'�_)5�N����@�233 ?d��k ���o����-((���|���BCC%IR��t���uk��,� ���������G �7o^�������JsW��lI����SJ���gK���v��������A��]�6((H�Z �A������]�~���seeedd����;g� ����=99Y� ��gO%���v7n��?������W�s��9
�/������Df���j�
��'�-[���,����������on�������* 222RRR-Zt��dgg��5��w��@�-�V����9
� �,+++00p��5$2 ��������II�|}}�����N�: !rss���6n�hk{��#;;���{||����
�����d�t��F>�l��Z _� ����
���{����� ���c�lll��? ���
]�re��-o_���s�F�Z�~}� �[�����$K�����U��K�D(��N�8����b�����oj� �������2��2�,���w�������G� ��[7!�N����swwW�N�>��k�%&&Z,�Q��vR�5����K�/'N�8p��+^z�%�k ����e���:�N���,X�����U+�5))I�����(����������'ON�4i����D��/���� ,~�N�e���6Ij(g����~~~��-{����� � [�n��� �����aCiiiee��#�������������*yyyC��;w�b���G� i���q�.\X�|y@@�*��
m���bMZ��+W��Df����F�R� ��6�������3 x��wHd � B�3g������ ��:@ 0+B��������{���Id �$B�����~~~��������� ���4\g�����DD���i �be�s��)������� ��H�o��m�7�.~���~��������j� ��i���K� IDAT�=w�48J"���o�� �"B�����K����={�� �� ��P��t�R�~�f��9q�D�k ��#�i(�Df���&MR� @(�0\�|�����O�<y��� �[��x�Z��� �����v9������F�C4�^NNN�~��L�2u�T�k 4�.|Rp���eR�wn� ��}�*�h���k�m^�ZR��^{m��9j� 0�������������5o�|���!!!3g�Bt���u�����, <x0++�s��f-��g * �W�c���C���u�w����vj 3�v������G" �_����_��q�����`���^�reLL��/��,�������|��������!DeIN�9�����uaL��o�����W^ye���j� 0���������Cacc��]�� ���* 222RRR-Zd���{T���n��ZB����� =z��yj� 0���/{zz'%I
��@�N�����aaa7n����@U.]I��Te���5���"�2�����������] ��n��UmNYYY�9������+W�l���e�j�>�Y��v.m%{;���z�m��e���Z0���FId^~������] ���o�������L �]�vU5&<<|�����uB�t���<ww�?L����F�F�{/��;e (������#G����P� ��,\�p��Y���B�^?g� �� $%%�t���e233311��u�-�Jk�����]�~���#F�X�p��� ,����aaaeeevvv���e���Ey��^�wuu���vppP���:th�z-�6�k�&��������������
���T� ��bbb�z�)� �
m�����A~~~@@@`` � �����w�������� �k���� ����t�R�k �f���W�^BH���� 6m�T��P-���5���Iq���<��s��-S� 4J�?�y|��)��_��W �����y��������v- ���X����>}����{j� ����)((8p`����/_�v- �>�X� �^�z����] �Z |Xk�3Dsaa����.]��Y#I��� ` ����*-��5�r%%%�
������kId �=��������CII����Id x`��] j���� ��O�� ��T��K�6���%�VNMB���]��h�� �Q� '%�i��]||��
w6 �C��}���B���i�Qc���|U)���p��D��� '� ������ 9�{�����AAA�/�r���4{����;_�pa��QC� ���_zzz�u�m� <h��>}� 4���s����oo�%K����g���������SR��:��W�^������T�v������V�Z%''��T�l�#�B��J 6�u�VG�A�����t����� {l���$2 `����|�M I����|}}����v�������G� 111NNNAAAIII=������??��C���B���� &���(M3f����|�������z����%IR��������
��o�q@����������M3��(�(���s$���-��;����=�u����C��iC" VD�����+�G�&M��Y��];c�������������56l�w�}gl������!��mk0�n9???..���Yakk��w�]�v��VWW��={V�+g��
���3� ��d��msZ�{����~�HId���7l�@" V$>>~��EW�^m�����Oppp��M��NNN�I���2�������_/I���s�F�N�>]u�U��$������cAAA5�N�>���^0`������SO=���R���������V S��Y��wj]e4���l��-[�$� ���}���7 !._������K�W���O�����?�\��EPu�NWm�Ng\�tk���=<<� ;��g�?�������#�g��|���}�Q �����_[���� ����u�V[[[�� ����b��� ������NHH�����m[�l�1F����������z�~��=��w�M���7����9w�\��������}c�&�$6m���#���v�2bLd���� ����'g��a P���,!!����6��h�"##C�\XX��;�\�x�� :t7n\QQ����b���/��B���k��,`kk;}�������@Q?�8���Xf�����_|�����O>!� k��?$''��|��� .���� ;VQ^^>j�����$$$��� :th��I6lx���o��1y����B�f��EEE�7�q�����O?�tFFFzz��������z}eeeHH��Q����n5:p���5k�m�f����|��k� \����#F888�� n��.��j4o��#FT}����f(��K�S{{��?��D `EEE.\h���fi1(���i#F������m������ �^[�n��������x��a��)�C��U��1c�������������7..���k�6���bMZc�+WQQ����]������� `&� exXF5z�>$$������>#� ��!�Q�^� =ztAA���; �. X��
�z�+��r����?��D ����/Y���R~~>�� ��q��E���1c���q���?wrrR� �B�����������w��E" @G(c!z�>44���~#� �1e,�`0� ;���k�v�j����� �_BB�����%KT�{|||@@���>;{���-��kW�^o��`���)�Lvv���;Id �A�b�
�m������wk ?~|jj���kMl����_�~66�������� �v?������Xl������K���'^�p!99���Y�r ����[eeeu���A�
T_�����9�'�XRV)������tB�����+@�e�HId~������� X���~[PPPYY��K/���J�$����7n\~~����===���._�����e����� !���,Xp��)!D�F�.\��[7!��K�&L�p�����TeI!��3���kYR\\��]����z��c�����=g�DFQRV���3�2UI�,�]��I���%Y�'M����?�� @����l�277��� ���������M�4�3g�q�>�@�����w������YYY-[����(// 4h��9s ���� ;v����{�������m���'On���n����z�4gwY��� G;����-��j�����1��Y��<y��'N$''�� @�q���e��)�W���EEE}��77n�����-[6o����)��������b��5!!!J"#�x������,~���y�W��6��#���,O�2%+++%%�I�&j� � GGG��$I���YYYU�y���[�n]m���?���U��h��I�&��������wrv�c�gG�(�N*��A�)S�dY�:ujfffjj*� 44��1U���T{ ���!dY���V$���,K)����5uZ����nj�-�2�I��i��}���$2 �0������o��ey����G��q�>}�|��U�,((���o���2)I��`���5���� 1w��K�F����� 9rd���M�6U� �
\\\&N�����z�~��>>>�Z��q�7�xc��M���W&�_���+�DDDx��'SRR�\F��������� `!Z |Xkj9D���s���������j�� ��u�������@//�
6���VVV�1��J�/��r���/^l���#�<"�x�������[(**���8�����>""�G� �V�� ������������>k�,e��c��-Z�HQXX���� !��w����[m�}I�5iMm���y���������E �T jI�� �/����p pO���������B" � w��IDDDrrrzzz��-�� XB���`������D j�v `��H����qn����-\�p��]_�um^l
�6�U +BG
u����;e��[o�E" ���~�Ydd�?�������� ����2���w���}���� }�Q�ki�����r�Z���5�o_���z��..�]���Vt���JGj���~ea
��Z��j�N:R3mM ��i��*�)))IIIIJJ���CTTT-�2>x�l��
6����O��9�D�h��@3����B����|���-�����-|7p�u]��`U/���L[�#} 5��b-��:�H����#m@w���<m��K�.����EEE��zLL � �/
hLI���_/�8}������{��[��D ������u� 8@" � �L��9s�u��jW��$IR������B����|���-�����-|7p�u]��`U/���L[�#} 5��b-��:�H��5-\��E(S3
�]� @;e T@( �B 4��~
���� ������s�� !���{���vi ���x� ���� �
e T@( �B � ��P @�2 * � P� �
e T@(S[%%%;v�x���,X�v- ` j�9���������)S����[�< ��:���� �����W ejE��i��]�t��W_-**R� �5��� NLLLKK��o_�>}�N�j�"@S��s��h���sZ�_%��I���_?s��� {L�Z �:��s����Z����^1r���W�^�v��5������_� ����@�J( PGYY����q�������*� �g���_������@�J( PAYY���C�9m�����_�� ��t�I�
��K�i�~�P �����j��999����U h����~ nW���2�*� @�����I+--��� �*�='�* ��.=�e�UB �
+**�����a0u 0�t�I�
��K�i�~�P �;;�_��8�����[7� �3�s������sZ�_%� �c���S�N���B��>���G m����E����9�W�vu�9-������ T�a��������������s������{�V�4 ��{��}�^�vm������ :tX�j�������s��h���sZ�_�dY��� �4 _ P� �
e T@( �B � ��P @�2 * � P� �
e T@( �B � ��P @�2 * � P� �
e T@( �B � �.!!���{��%u�NLLL��]�z}������>��������{�/���g���[c�������[�s��O?��.{?}��c�=�� ���u�;������ ����o�\���������U B ������X������������M���� ����v��;��t����Ls��J�3��G u�����f�[gZ������_VVV�^x�������'�����wdz]������{w�����sJJ��]_ @�b�v X��A�
t�k��u���~�o����g���b��U� Re��4k�l��� �qhh���^���� �� ����~[PPPYY��K/���J�t���7�x#''���B��7o����?>::���A\\��]����z��S��]�ti�� g��IMMm���2s������������=����JNN^�`AAAAII�������j��������C� ���� / �(..^�`��S���5Z�pa�n������6m
��m[��M7m�4e���/0 ***!!a���C� ����� ~X^^.���/���k�I�$����7n\~~����===���._�����e����;wn������M�6
������_�u���g��uuum��q��=�����V�V�W�n�[��l��������C)�5�+!�����������������7o�r������3g���.]��n�Z�,�������:w�l\`�� #G���P��wCY����j����o�{u ������������I�&���s���}� �� �� =zt��aS�L)..�e���b�������q����[�n����6���?��5�n��[�[o�u�����-{{{O�4���[�,��y���'77��� 3fLbb��`�eY�Q��S'�������7--M����� |��e���'&&�`0l����G ���o
��!C���eY>z�������c��\YY�p����z���W�Z�t�R??��~�I�����K�.��|���~��]�xQY���+�����m\q��! Tj.,,���gfff-[M�I��[��l4y��+VT�c�\]�z���W93z�~����=����~Z�#�e900��y���������^�o����7��@�N��6mZ����5�\�VW����}�-�&���_�����9�L������>�������g Y����^ @�t���e��9;; !����������7n�]��stt\�|�����Y�f��
����j����G y����`�� [[��3g(�k�� 0`�2���'$$ovh����Y�$I����r��$I�<������,��� ��g[[����������kU ��e����===�]�v���B,\�p���m��U�i��uBBBtt�q-�N�������I�5k��k����&�>^Q��\���'�x��j3M�������_93666�< Ell��!###%%e��E���������3g��@L���a>&����U]����FTT�{����cGer��Uo�1]3 ���K �������h��$���7++���W���y��T����������R��NNN�I�Z^s�������s$I2��|���u��Umm��E�&M���]\\��H�d<���B�����UI��������� ����~��p����''M�Tm��?�l� �h���W��h����'88�i���l5����:���~��ww�j3M����/+�����%3�S�NB��������;w�����������{��uoGrw5~7����7������0���x�b������w���� @�� ���?mll�}����s�
3aeee&�_#�r]��N��}�r3�� �?��c��� �E���7o�,��|�rJJJPPP\\\�.]j�jB �����is���j��8W�n���t�IPeeehh���+[�ly�rrr���X;W����| %%%�����f j��� �!55���ZY����k�����
Y�����_I� C}m�^���3--��L�N����O�/���jSAAA~~~�f�j����w'�z��={��2pG^^^&�\\\|�������������j�jZ ���:w�\�� ��s��}��� c��`��k�=�1<<|������:�.''�j��`8��_���{=
2}�M����������UuNjjj-k ��e ����e����� z�~��>>>�Z�RZ�6m����*M7n\�~}}���'�LIIQ~���|�����������������v��a��w�������|!�o��i�����+M��_��W"""j�� :�7���HQQQ1}��^x��=w���� <xP��ey��=S�NU&O�<9c� cpVVV������U�V��x����O��G�1q�.\8k����l!�^��3gNm�h3JJJ��t!!!�dfffbbb�N�:��]��y�W����| QQQ�'O>��2y����/D��7 ������ �m��5...00���k��
������#F�P^��,s���I�&�����<f�����5k�l������>v��2NjaaaNN�2�G��������[��!<<<--���������}��Y� ����ryy��Q� 8��������i��W hIDAT�r���I�&988(5������������K�TZZ��g��3g>��#JkQQQDD��{���>""B�t��� |����C��_�����s�-Y�d���K�,IKK��u���W������V�^���+++CBBF��l��/�\�v���7n�������:�oII�[o�u��9���������O�<�M�6B� ~�!99Y��k��]�p���488x������[k�Fw;���g���rOO��� c���a�\ !��=VVVfgg7|�pY�]\\^|���H�����z{{�;���:th�g��L���G�W_}���P������Yc {E�����o������V�W_��^���[��w�������+**j��Y�� �����M��}�&&&�:u�t���'On���^_ hPe �(AC��f�`x��w
�1��s��SO�����������{ ?~�� ����P�@( ��/ �l�� g����u�_��{���zS��w"���k��!CV�^]/%����OM�8Q�B Z��2 @��{L�����'�
v��0���?~���W��$-]���K� � �*x| @�2 * � P� �
�? n��l�� IEND�B`�data-unsorted-fits-into-ram.pngimage/png; name=data-unsorted-fits-into-ram.pngDownload
�PNG
IHDR � � �W�3 bKGD � � ����� IDATx���w|U��?� l��������.pPE��$�PAd���ReIq�B�U�� [�� �Zk *"2 �2$�p�6M\$9'���g�w���>���9���F# ��bA P e � �P B � e � �P B � e � �P B � e � �P B � e � �P B � e � �P B � e � �P B
� ;v�y���� 0`�%�\������?��~��7�xc����~{����_% �7�x����z��'r�l�������W���k�����&Lh����P��� ��e��m�z�Z�dI��%��-��y�c�9�A�k�����{?������g��g��SN9e����H�}��+W�\�`���_^�|��c�/^<��!��D";w������ �/^�D�v���X��k���H����+S�L���k���7��D"_|������:��C�Dq� ���c�-^�8!!�\�r}������"�����������7�}�������~����{��7�_��W���+V���K/}�����/�X�b$���� |��g�=��#]�t9��_y��� ��~RRR�b��>q�UW������5�\���\�F�&M��*U��'��(�,X�`����o�F�]�v�S�N����#F�x��G~���M�<9s�I�&�n����.�>}�a�Y��1��G ����/������G"�Y�f
0����>����M����/�����+�x���U���{���z�����:u���[���o���� L����9��Y�f����n?�<�L�z�.���\9[n �����4��?<���.���K/�4���F;u�t����m�v�������l �G�|��M��c������?�:x�[����u�M�6�~��Y���� ��q���kW����~������;���W�l�l~���:u���S�N+V�8����;v�HII�<y��}������~[�^�9s�����;����y���u�����s�� ;v4m��pk��{�����a��C�,�g��n�!�� x��W^9�Sek���6h���7on����� c���d���;v��x����7n��c���� A�EY��
�_��[�e������ 8u�������q�����1���}��d����n?b�{�� TU�-�[���E���Y��+�T�Xq��M����m�[��
d��%���FW�Xq�-��O8��� {��c�=��%K�<������� ��s��j�*���R�Ju���v����.]���-��r�)� �S��y���w�yg�&M"��)��2n�����={���G�U���?��t���7����'V�^�W��U��/��r�
~���=�����g��u-[��W�^��M{���k��l;�w�}�=�\N~�C��o{��+V|���G� [<������ /����%K�T�r�q� ������V���K/��U�Q�F?��c�-������P�\��e�����u��m���� ���������[/��������?���������1c��l��a���\sM��
W�\���_}�U��M������:p��W_}��K.y������8��+�m��H$2~���o�9�r��u�+W�����������u�}��'-Z�h��a�z�b�a��m��u���A�
4����-Z��������o�����U����[�l��
�x��S�L�_��5kRSSG� ���6�����g��`m4k��5j�y��M�6��wo$����\�v�J�*e���o��{�>�������_}��� <`�� V�Z����7�tS���?��� ���;��[�n���7o�|���D"��C���� �\r������]�����/_>'�D��&M�T�\���>����i��3�h��^�z�������6n���_����nZ�re��
3��6o���I� .�`��I���i�8W�h�:�,�L�����k7n�x���-����n���9)���u����wo��cc.�:����?sJJJFFF��}���
�8�H��}��]��I�&�f )�����o?h��n���� z��l+7n�X�v�/��2�8c��v��e;���Q�L��~�)�� P���6Vb���}����;��� ��Y����.��}��K�.��_��������o���'�|�'�|�o��F��^�:��X�"))�������~������� ��7j��o�[���n�Z�F�E��p��_~���?�����_7h� s���~����f��h4��������~������m=������SlPL��-�4i+���o��!�#e�}yl �������;t��m��h4�g�� =z<������v�JII�1cFl�� ~HMM����b�������v��Y�(�}����6�/� ���n�Z�N��;� 92sl�![�M�6�F���y��� 8��ssRU�k���/>��3u��Y�ti4��|����v��m�����f��l��]'�xb$� ����G�
�}�]��5�����}���/�8�7a��y��a��~�6� �J �]P)��^x��[n��E������uk���~�m�F�5j���o�?@#���fN@�)6��� ����m;��c�����K/�������W�R�����8q���d����>;���������;��sb�u��������S�X���:��/�<�br���h���� 6l��A���o �������^~����y��
��~K�P��C=���P�f�F���Q#!!��+�����"��c�=����V�Z5�s�J��x��~��e �}��*U��Z�B�
�>��3s���s�� �S�j�}��en���� /�p���F"�b�����3��GYU �+W^�~���{w��u��7��93�?��c�,� �u���o��=��>��������D"%J��M���O?E"����n��n���=��_���}���-����;��5jd��f�����������F*T8��3>��������9�0~���?���Nj��e�������K�
rRR�k�������� ���|����u�6{�����w]�v�>}zlP��,Yr��% ,���wk qZ�w���
�M��D5j��M� �����1��yz����~��%�\2g��5k���3��{����2dHf��R�J���^$i��UN��F#� A����� EV%J���{w����s�����
����`����*U��|�������.���q���w��T����__}���UL�6���oc���?��~����O�Mf��+������O�(Q��C�K���2��\�d�� x �!��/�����C������� ���^{m��-�9�� n ;v����c��)Sf��e����]�5WJHH����3���8*U��~��O>���+��[��C=��]��;�x����4hP����}srl|����:�sBBBJJ����SRR���7l���; ��*T��m[���y�?����� b9�?����.��� ��#~�s�=C� y���"���+*W� ��"�j����g�U"�HBBB�f:���*s��O>Y�R�����Q�B����/���[�p�����y�� & ��9?�I'�t�I' �����
�W��Q�F��o������o99�!Ccv%���'� @���sg���O;���m��m�v���={���_��m� =z�p� �H��O>���+v���'������n�lbb��]���-<q�\�B��L����,\�0�e$&&fKgv�������<�J�����SO=��G�#����N[�~���n�_zzz�5��g��u�>�m�U�J��M��D�m�v�]w��5�q��� \������-[�����.��~���&M�0T�j�������Hd���S�Nm����/��y��l ?~� |0i����v�� ;v���AY� eUqT�T� ����+�5kV�R� ~�a���9 t�:�����7'������b�������m�u�!��J�*�z�����?k������%��R� ��.����]�����+Wn��Ys����k��#Ko#��V��l�a�����^�z���G� 9y��l�_���
�9<�!Ccv%���'�/������o��f�5'�t�� ��o~����? ��;w���s�!�m�������l�}���9M�F�3'%%� ;6���[���9��k��D"��w��� �n��?�y���)
6\�l��U������;o����Q�J���g]�Hg�o������� /<�zb2�����/����?�����F�+��]�vK�,9�N���W����� l��m�2+<��S����~�����9�:f���'n�W�Z��?�8sq��}Y_�}4U�{|���?��n���.{����`�������o��=�i��e
������3cC`����w�y'��[�l��iS�M�9���v�:`���+W {���&��/!!!�#iY���n���1c����g��%Y���o�5j��1#�����gV�����h���3s�����W�d*��6m�����m=�V8��3>�����g{�t��=Y�e��=~���u �P �ddd���?�1�m��u�����o���m�����K���?��w �F'L���C������� z�����g�5�������[���J�,�DN8��N8a��Q���W�������t��SO=���GP�5�\��������}��m���W_}[�7o�����l�|��g�y� ����%K0�}���nO�+�)!!���>,�� _x���g������;v�[\�dI�����]�v�����W�����?>3v��u��O?���_g~�c�=��C�^�:�ddd<��#Y�_8����T����~� '�b�z�����kG�������Q*_������}��H$�����W�k��66�������k���;7���������={�sr��u�Y���� |������ .�:uj�7��y�2'
�I�h�b������^�F������)))K�.�0aB��3g�l����M�YU�ku4222,Xp� w 6��w������� ��\����������� ��F��z�������Ce������g^��c�9&��ddd����������r���:��(!W�7 @N��=�t��c����� ������s�]w5o�<��Y��}��������+� ��'�l��!�t��������D����;u���� ��H$�F_y����'gdd����������k�cG }���O>�m��o���� .���K����`��y�� �z�Z�tiBBB�2e����u���]�����p��b���-[�_�~�IyW�^}� wl������{���b���={.[�,�����w��}�������>j$��?���O�H$�u��u����.�Y�f���c;|��W��u���_*V�x�9�������N�U���Q�b!H� .����m���E�k��u�v����[o����YE:v��|���#G�x�����?���/�x��W6h� 6q��������/�����O 9r�i��v�� ��?���M�4>|����k����O�j���'��1cF�������{�\�r��}�J�:��3��o ;���~:e������V���sg��-�������o������o�u��b��U�X�o������c���_ �DV�X��?�q��]%J���u�|����5� ~�U ��� ;x���#�;v�8������I�={��l����������A�U�V�W�^'N,_�|�^�.���8������'�LKK���{0#G�|�����������1b���{��m��y�v�2�b���/={��%/���={��:��!��H$���� 3fL�a��}��w�>c����+�*U�J�*=�Pl2� �����;c����g @��~$��sg�[%!!a���5j����K�|+q����}��!C�|���e�������/����Y�zu�b�������1�O�>�]w]f����U�V[�n��6�D"��w�Y�f���h�Q�F .�6mZ�$�8[����Y��#�l���x��U�V}���Z�j����+V<��%J��F�m����m����G�+ N��J �]P e (��}��}�� �@��m���f�� .�6J�<5`���.��~��ARu���y�����,[�l���9�x8l�����Y��P&������D� r��w_.���5kV��=%2�i��������w� t!E�/���j���}�E���� L�-r>��H)]�� ���w @!f� Fw�}���~�D�y���/�&���I����7l����^�V��_~9����?�����~��] ,��h[�r�E�P �� � @ �2 � @( � @ �2 � @( � @ �2 � @( � @ �2 � @( � @ �����O�0�E��z�:�'NLJJJNN�������sq+ @�](�F;u��f���m����/��0o��Q�F��1c��9�\sM�� sk+ @� ](���0|��.]��~���a���/��bbbb$���[�������>W� ����2��k����+g.��Sg����� ������k���%Kf]s�i�}��7G�5�����{� U� @���h�%dW�B�={�d VJ�.�k�������{C�r$$$�v)(?l������r���r��<I�
�W��� ������4�����P*R�RP~�0���4��v�'7n\��������, /�����(;w���Z�f+ P�� ?�s��3f���+`�L�R�����u��u�2�>�� @a2a�� |p���\pA�� X e"�H�%������3f\z�������w��A��
��:�������=s��- �K�+R�Z�~��K�E�Gg �e� E�Y ��:}�����$&L�������/������<��v�l����4(��>���_ 3fL����z��i��
:4W� L J(�|� %_� �e��� :t�>}�E][�/��M��o��V�Zm��u��m�|����+#�H���k����V�Z��}�z��/~�Yg����Y?�� @A��[o���S�N�LdB+�AQ��3N(@|� %_� 9���o�{��S�L�������i��S `S�L�������l�Lh���%
�0L�P��"7j��� �����r�c�������,����:u��w���{�U�^=�Zr*��w�&�c� �^�������� [,�Xr���ZV Q.3m���m�����5j�8����{| ����i��L$I�������M,�y��w����P �g��-�����9�J�7}���m�N�<��+����&� �9�����T)[1�J��1cF�6m&O�|��W]��� ����^�Hb�b�Hb��XO���3[�n=i����D�2 @|�?�v�5O�X�xF�6������|f�%��9�e������������h�q��� �� �?F��|�W����xV�V+� ��>��y���������rxH8��F� �l[�'�������������7 ;vl���� ��?51��*����y��5k�l��1�k����%� ����I%��7@(Q��}O��2�����i�1c�$%%�����A ��9-+F���{nH_��\������9-���� ~�a��MG� ]h���~s"!!!���{�NKK � (r>����n���7�LII��Qiii}���\ a "�9�pN� EA,�5jT�:u��$���{| �����p�
�����$2�e�_ �,Xp�M7���k���A��'�2 @�,\��Q�F��/��LD( ���E�RSS� ��Q��k�CB D-Z��a�!C�4n�8�Z��P �����&M�]K�� ��x�����~���E!��e �0���O6l���/�p�
A��OJ] P�}��g��� 4h��7�t-��H H�}�Y�z�
t�m�]K�� �Y�lY��
X���P ��e������3��h�"�Z � ���_��[����������%B �}���u��y���Z�jt-�� �j���u��y��'�r"� ��;N IDAT �i��u������ w�t-� �d�����={�l��u��O( ���+W�����;�ZB!!�]C�%$�J pT�������?�����{o�z8��F� y+��t��5�D&��2 @ Z�fMrr��?|�}�]K�er$�?�����
�5k�$%%u���������NKK�����G�P �
�p>x !�v����kw���}���V����2 @�[�vmRR��>x"ZB ��[�.99�c�� :t���� �i��uIII�� :v�t-��G��&�� @}�������[�~��G�������7R � ?��Crr� w��D&��2 @.�%2�Z����[�� B �hm��!99�e����w��C( �X"��E� =z]KA"� ���
RRRn����={]K#� ����SRR�7o��W��k)x�2 ���%2M�6}������@
�k��&�/3 � m���N�:u��y��g�������7R 8<�6m�[�nJJJ�HdBK( ���7��W/))��g�
���M( ��������{�u����?�Z
<� �#�12�j�0`@��B ���l�R�~�k������ ��BB( ��-[���W�f���>�Z
� Ol���W_=p���k)T����� ��� l���~�����o ���t9G(�]�0�6�l9 �k���
6<��s� Rp�HX�� _ ===55�$2�%� ���}{�F��>����R� e ��Kd�U�6d��b�Dy������HKK � �C�D��3�:th�Nd���2��A�r`a��&l�9 �����7n����O >|x�Nd� g���\_ �h����I�&��vZ!KdB�% ���T�Re������;a�1N �[v����i�����5�x��A������} @��{���M��+Wn����2� -� ]�D�l���F�*Q�D��-B (�v����Y���K������'� ��h�����7/U�����%2�� @�Kd�� �P ��={��r�-%J�=ztbbb��]B (Bb�L�b���#� �J PTddd�n�z���o���D&pB (222Z�j�e����~�T�RA��P �����;��c����&M����9e ���=��i�&cdB�H (�222��i��O?M�4�t��A�� e �����h�����'O�,� � N������ $2�dN (����w��w~����'O.S�L��p F� @aKdV�^=e��Lh e �P�F�����U��L�R�\������2 Px���K�N�:�|��A�C<�� �B"�>��K�,��F� @a�F��o��g�Id
� x�h�C��/�6mZ�
�.� �� l�h�c���-��,F� @�F;u��`��L�#� ��*�>��� }����3�9������e �������y�f��u�q�] �M( R�n�f��!�)�L��# ����t- �������g��u���]KH���ev�������h�5�]B�� @����c��)�f�:�����`g��H (Hz��)�) �) F�^��{����gKd
� �=��;��#�)4<� @���'O�<{��O<1�Z� F� @�����7n������H �g�}v���s��=�������d� �W���_y��L�d� ���F�1w��SN9%�Z�}F� @=��s��
��bF� @�<���C��;wn�J�����b� ���� ,�)��� �4h��/�<w����+] y�H �?��O���$2E��2
C� 4h���s�V�t-�� o��aO=���9s$2E��� `��
������s�8���k!�) A>|x�~����#�)j�� ���1�o��s��9��3����f� ��W^y������S�Z��k! B ������O���gKd�,�/ @~{���z��9k����:+�Zrd���J��w�}�m+v����\ry��+* ��h�5�]B�� @�y����u�6{�����7A��#��~��JF�c���m����TohQ�'�]{�/ @�;vl���g��UP�H$RrY��D&�����\�x��B �'���������3�?���k9 �~�}MF�5 � ��q��u��y��+��D"���}M��k8B �s����������/����k9l��{lwB��.&��}^� �)4�8�M��s6
� &t��q��^xa�� �����pv��XS���� (&N���C����_t�EA�Rt��k_"� ��z��������S�Jd��9e O������w��)S.����k!��8z'l�9� �0�2e��w���{�U�^=�Zi��H �eS�N��pHB �M��Mk������+�!>� ��i���m���w��Q�F��vB� I������k ��O� Kd�����k!�������� �<7a��� �3f�n�z���W^ye���]8��F� ���9sf���'M�$�!�J] l3g�l����������k� 1R �� |��U� &��U+�Z(`�2 p�����7o�|����]w]��P�e �H��7�Y�fc���]�v��P e ����7�i��c��IJJ
�
*� �?��i���G���p4�2 p>����o���7�LNN�
6� ��G }t�M7�5*%%%�Z(�J] ������^{��:u�] ���2 ph ,����^{������k��� �!,\�055u���r�P ��%2��
k��Q��P�e ��-Z���:d����] ���~ ��/^��a�!C�4i�$�Z�g�������1����+�K,k��\��F��!�\% �"g����_����o����k �� s�oz>�����2e���X� �<�]{�/ @v�~�i��
_~�e�L$�����&2�H$�c����+���� ���>��~���
�����%�?��fc �2F� �}��g���4h�m��t-a�P�W��91�J
� ���e�6l8p�@�LV�+��$���rB���WN���% �D"�e����S��g�m��E���Kb�������}�Lr�Ea�|8l�9E3 ���/��S���O?}���] �/�]{�/ P�����zJ"C~� P�-_��N�:O>�d�V�����E( @��b��:u������;��� � E��+RRR�����u��k�(� P�\�299�G� m�� ���0N> 6��� �#��7�$%%������{��������2 -�D�k���%� �Y�fMrr��?|�}�] E�P ��b��5III]�t�������2
�D�s��<�@��@$"� �(X�vmrr��>��}��k�� P��]�6))�c�� :t��/� ���u��������� ;] ��0��;l��2s ����OJJj�����>t-)�]{#e (�~��������[Kd'� �P,�i��U�n���L( @a�a������-[v��=�Z���2 *�D�E�=z���G( @��a�����[o��g��A�� � ����qcJJJ���{��t-phB� I������k � b�L�f� {���k!���2��A�r`a|Mw���e� d��� �������g�y&�Z�pv��XS���� ���iS��u����}���k!������ ����������$���� PPm���n���]w]������P �)6F�V�Z �8B
�-[���_��k�y�������P �f��-����Y����? t-p��2 $�12W_}������J�6�|o @�u��������� <xpBBB��P`��k ���&�- P����7l���s� 2d�D������� ( ���SSS%2&B �.==�Q�Fg�}���(L�2 �����5jT�Z�!C�+�K��n �b���g�9t�P� �� ����}{����V�:l�0� ��{ �0��cG�&MN;�����Kd(��� �N,��R���#$2Va|Mw���e� ���]��6mZ�|�Q�F/^<�r( ���7 "�w�n��i�r�F� )��p� ����Q�J�(t9���2 ������5kV�L� E�P ������y���J�z��7%2B KdG� -���� �={��r�-%J���P�e L,�)V���1c�.�� �`ddd�n�z���o���D�"H( @ 222Z�j�e����~�T�RA�� ��222������7O�4I"C�eN �U���M�6#Cg� �'##�M�6?����I�J�. t9$� �$##�m��7n�<y�D�2 �����v������1� yn��}w�y���?y��2e�]���2 ��X"�z��)S�Hd �P �<�F����U�VM�2�\�rA�!"� ����K�N�:�|��A��bN �D4}���,Y"��2R ���F��o��g�Id�`�2 ��h4��C���O�6�B�
A�!��% rS4�����E�$2��2 ��h4��S� Hd���2 ��h4���~��G3g�<��c�.�N( @������y�f��u�q�] �� t��m���3g���@) ���������g��u���] B �J� =�N�:{�l� �/ p�z��9e�cd�) �����{��7{��N8!�Z��� p$ {��w�yG"G��K ���{O�<y���'�xb��@Ae� �'--m��qs������0R ������� ;v���'�|r��@�f� 9���W^yE"��H rd��#F��;w�)��t-P) ��=��s��
��@.2R �Cx���� :w��J�*] B� I������k �W <x��9s$2,iii����k9��h4t
a���* E��A�^z���s�V�\9�Z����koN �O�����'� ��/ p C� 4h���s�V�t-P8 e �n��aO=���9s$2�w<� ��6lX�~����{�g] fF� �_�������9s$2���2 ���#�x��9s��y��A����� �D"�W^y��� ��@�1R �������O�9s�T�V-�Z��0R ��{���z��9{�l� �'� @�����w��}�����w^��@���F��!�\% �p;vl�.]f��y���] ��pv��) PD�7N"� E��������3$2� @�3~����;O�>�� .�(�L� P�L�0�S�N3f���� ���4#e ���'v��q����P ��x���:t�0}���.�(�Z �/
o������;u�T� �D_� 6�|�9 @�M�2��;�|����W� t-�pv�=� P�M�:U"!$� (��N���]�w�}W"a#� (��M���]�w�y�F�A�d'� (��O���m�w�y��+��� �2 ���3��i3y�d� ��P ���9sf���'M�t��W] pP%�. ��4s���-[N�8����
� #e
�>��U�V&L�U�V�� � � ($����7o�|����]w]�� �&� ( �����Y��c���];�Z� 1� @ �ja�S�o�Z��1O\~��g^��2o���M��3F"HB4
���KHp� �#1z���>��gwl�\b��W7kY-�s�?����o =ztrrr��
�pv��XS���� ��;k�S_������p��f��?���|��G7�t��o����� C!����� ����[��Y�}s�CF�Zt�Gb�kVo�t�G���
��%2�F���@Ad�_ ��rj�c���R�b�Cz-����S$I������s���7�p�+��R�N����P ����^������K���A�Cr8����?n���k�����z�u�� ����.R�Y���%+^��qCj6kq�������� �x������D
�0�s6��
(���j���'d}a�����F9 .LMM :th����
�pv�M� "-�]M�d}�R�Df��E���C� ��@! � �C;V����g�����V.�e���������9�}^���[v ��� #�y%����6l8d��&M��q�@~���� �' �v��e�]�=�����r�Z�Z�8�|��w�>cp��;c��2J�Xu��j7Y�x���_?x��n�!���B'�]�0�6�l9 �6�u���_e]S��Y�j�"�!_�hyb� ���q�I��?��A��_~��o��B�� g���K yeo���k�����������?h� � 2^�
�WJ�=5��rU� �y����9n��A��v[nV��P ����OBb�����r�/�������+�t������$Id�P
�#Ua�� ����4o������X��-;���Lj���|�
�pv��XS���� ��j��Q���H��o�?��Zs�� ���k ���&�- J;V���� "�c�9y�6pH����S ��X5j��3���� g�X5������G3�H$���mq��.�� �M;V��2�����H$�w��-���Dr8�e��%��-�� �C�E(��� �M���%219 ��b������%~�m�!��
PB ���7}]�59��r����� =zT����}�6P@y| 7�({��_V���C
u���o�����G�{��'�D����yW-�0N> 6��� � _�����3�`JH,W��aq��o��&))�k����w_~�EN8�� _ �Me���x����&K,Q�j����Id��Y������Kd�
cP6��� ��n��5III]�ty����
�pv��� �e�Gm~���/�\������|�}b�L���%2Pd�1(
�p�i @8- �e�]�����-�(W,ih�sZV�����k���:u���C� j�"'�]{#e r�'�~�Ld"����}���!��D�c��(��2 �i��=������u��%%%�u�] ;v����� ���� �' �F��|���Y�KL�D"�*'���d��o�����>PuPD��k ���&�- ���7���{}�'�2�)�ko�=<����
��pv�K] @�rN���h�������F"�=��&�+u��+�
�0Ea�8
����f����D�R ��� �z��
g��D� y������T��(��2 y��.e����X�\�+��`=@�e �������p�M�V�Z�XbB��%��T:�E�����D� �o���)))��5{���A���P �
��o����x��/J��j��g^tE@�q��� �� @8
�|�=�'�K, [,�Xr���ZV��@����7� @���ys���3�H$��gw�E3, -� @���ys��u� S&����7RrB �\�y��z����U��cN���JYo\@( p��l�R�~�k�������si�r�%37�K,�����V�� �p� ���-[���W�f�����Z�������o�\��{�n�n��6��w\�b��[��J�����pv��XS���� �0�������
����� ��/;3���(]<���*\SY.�-�]{�/ ��[�6h����/� #������L$�� �"=���N( p$���7n|�E
<8!!�����##����[YB ��������z���2�`�L$9�L�lk��o
Pd er$�?����� Xzzz�F��>��8cdb��]�t���P�x��g�������� IDATH$IKK���]���q��� �l@ @ �o����Z�Z��C�+v��s����c�L��%V8��a�)l��r @���}{�F��8��a���$�B"�]{_" 9�}����W�ZU"�
�# �KdN;�����Kd�\�����w��Q����������!!$�B��i% H�Dc�X�Vm�J�/���D4D �x rE����B1�B��������1a�� l�$��e����|6y<Z����~ \������;����}�]dd ��(�H��(�� \�����s��lNOOgY4�����j��� (����6m��d��������]n�Q8���t�d�0��. �(%&��F��4 ������`HOOW���O��k���u]u����� N?�b� ���W{ � h����>}�^�oUF����5�"]n���_ t}H� �y<�3fh������*#CD�p�Q���� ��2
H��222Z��!"�
o��AK�nI �z<��{��j��m"�Y����f�:Kr�� �%��QeV �'edT*�G }�q\�� ���i��k�G'&����W{%�Ii��� ��%����kjj6o���j;dNtbPe���� @]F�j�v`F��� Z�� \�A�3g��j���O:0#C�� -BR .k� ��;����c��H�� Z�� \�A�7o^EE�'�|���:|~tb���� @7 ����_^^�e�����!"N?� ��NL:s2�O���H����F�%� �=AHNN.++��e�^���
������������S���8>��d| x�|�W���F�9 h3�����\ZZz122����~����� ����<.
y����W{�� �����{��
>�������F���������� � j� �eD�� ~8??���F��b<��%�"�" ��2 p��22������[M&�EzJ���EBE I �L����#���?���m�������F ��/u,3s�E�� ] �/ @�'����>z������!�qQ:����Kc�} �����J��� $)#s����������^ te���� tg�(��OBF �� ��E��� ���AF I ��DQ|����������G� �� 9$e �{z��g����k�.dd @�PS ��g�}v���;w����gg� �i�) ��s�=��W_���+44��� �,$e �[y����m��{�ndd @�p| ������[�b� t �) ���� ������w���u�Z . ;e �;x��>���]�v!# ]v� @��h��-[�����W�^�� �`!) ]����7n������ t-8� ]��e�6l������k h �� ����W^y������.
;e �Kz��W�}������}�v�Z �I P����u1�+4�����M�nn�k����;���� ��Q;{
J�0�- \
�����^�O�TUVE
I�������+W���������k�.I����) Jqh�F�����Egdc�x����~{��=�� @W�B� �i\����T�Y��L�i���R����_��S � �20�w$��K�����X�"+++**��V
�1��{Gi��� ��s�gX����C�d8�6-9�e��~��p���1�������_wub\�����|������v����Q�����4��� tu��ykNF|L��|��W���Z���^�����x���7��������6P�����4��� tu� �H�mb�6U���&6$0��~��-{�������K��B���(� �Cm��G���$�Y�,�W�����I h;���v&��d��L����{M��0���pF���Dd���k�A- # ��2 �F�k��z��[&�^���Y��w��v}\�e�J�i �������+������+C�L}cl����w�� :��T)�2� t:[� �PQ�Q��u������kYh��T�W�}�u��N ��)��^�� ����=�bd�3���������X�%���9��+D��Kl�� ���K �F��Y���k��%�>.ed$"�� Y��i �I h��NF8��w
��z�dd;�\�v��[. z���� @��� � K������f��1�^L�����j���%��K����.��D �iE�����u1�+4�����M�&�
6<��sjc��;�� (j� �d� N9���=DT`�J9�����d���5�B�aFN���[��37���R�}m���u��f,�����v���=b=�@� �H�l �nF����F�%� .�A�/��U4��{��� ��������Ye��v����� ��pP�������j�RDS�. 6����AD��t{N��,f
Q��4}lR�~! ��(�� ;e ���VY��Y-�� |"&����b~�?#CD ��N���o��c,�� ��2 p���<���H?���s����P�,8�q���G:���X�SHd� �+� � X2b����~m�4K��� ���K���2��]yk����zk����+�b�K��a@w��#UJ���g �R�k�����������=����2�=
���s�� \���j��K pa�FT�q�E� ����oh���7o�6
�gd��Ui���P�W"RG���1Z
�>���3Lj� ��� �x� g�r��L�>o��z9���� �n�����2�B������B����?&�m��1�Q�7��r(���JEZu� �L�) ���!�U-�����~b�f��m�<���?����!'���g��FD����2�G���a��z �ba� ��:�b������s���:��(��
�r�W~zyfL�:Myf�+?��3o��-99���?�]�F��H4�c���+w���I*���� t'H� tO�M���>�R�����8��U1����Y���ED�� kv��^ �x������f��}{rr�g�}v���{ E�O{z���+��6��� t3H� tC�M����f��H�T� 'F_?���,�Dd?�0��_}�������g�� 9��\���5��CfY�Hg���� �$e ���%����brWh~Y���^���p�<�+����J �0j�:��8�O ���]���&��c��y��m��E���+�1N������ /���'6�8 �n�~ ����%������GD�OVJ 1��DK��rUH��B`���/�����5��5^�� ;��������pC}/�5�����O���V&�_��_z�8�)��� ��� ��|�(YDJ� ZxF��H���Egd#O�#�=,��?5Y��4|���/��c���Ke����3))i����F�
�GB6���"���� ;�������w ��;e ��� ��5)�������c���K�E�l��P q�=�����7������o ��j��R��PR�}\b���C�����,b�����������L������=;33��o�=e�����W�|E���Q�g��<���`��
� �NAS��� ����{HRH��z ��E����t �� t% ����D���������uW��?�������> �k����if�
����E�m��)�~o oP,Fm >��c_W�q��#��Ma��{���>}��
n���&�YxbFCe`�C���u�P�t�%������ ��dg� ZE����) ��H�a
�o-��U��Z�f""��O��b� &�Q5ri ���h.�����R au�Y�,�U *?�$U��t �x����}�Z���fX{��Wf�rF��=s*�����C�� ) �j� t7�hN �W�h��2�<P���@����R�@7�}���7�+U�`����������1R��0=+���Y"��o��i�>����22DT�����W�JO�,R�KEM�� ��S ��iy;�����K����#�P��Tj�a����� � WW�f�`�;?��{[t,3s�q�����M����0aB O2�3F���1�T�'�9����R ��sR�V���2�H�njRl�o �\�) ��� ��pN R�-�W�M�6
��^���ED��t�"*jE�H�Z�6,��9�#edZ~���U�\�����:��GX�����3V2��Sd��x�P`�J9���dN�� ��(����(� �Eb+���#*6� �A����� ����*�U�&�����6q�� >�2o6�/
����A�{K�m��4(�����s�����2 �2��W{ _ �t�y��7H�����,�M�\��}
����?������}c&N
&#CD��dG�� ��.EAg���������
������}7�Nya �AR �j��J�UW8�D�(�mkD� �����9}�g�� �/1;
�Q4c]F;������'{<�g�O F�!$����U?�5��)q���(s� @sd����f��� �4&�����zE� *V��vn��4���z��O�����?z��T3}V�o7j����Y t'�|��N ��fa�v��� �g�� "�R� ����?�k�����yK ����%�#�"����)��eD
&ste��c�u�IDab�S����I P6t_ �n�,�"��c������#_�Xe�Nq�\���jyS$�)���Q3��`n� ["ed�.E������ �K I ��&� �U��`��O1��0�w��,j�S� 9��:����# g4
_� �������u���W~:��^��["�� �� ) ��,1��i��FN�4�)��x��Y��G 9rd���w�����������S�ZF����
rW~�5;�k/}��^ ��i2��l66R Q�# J��2 �Mb\��P`�\����P��U���y��)��1��_\���~�:u��+���N"j[U���"��_J{v���� ;3����[���3��/�j�� =��3��;� .% Ve�h h����� �!�3ZF���%^p��~�i��)��/��������t$�6���s�m��DDG~|_��E&_�]��}��������@7��W{%�Ii��� hW~�='��8E
���6
�v��{� =:y���gd�����^[^��� �'�w�r����9���3's�I�| )�� 5e ./�������J#�_����c�~�������������N�_��4G�>���8����e���y����y���? @��� ���-q?�Uq��up]���:v���I��-[6kV��|[���yJ�T(ZxR����I���5$���.�um�< @�P� �{�M�>�z�Q���Qi��$���b���mnA�f��L����q���*�[��T�� �4i��/���x��3A*rZO���4���'�:�E��l�(�w� �I ��D�#8�YCT -�r3�Y)%^���l���b�!��
' �1��+rcW��$�O���N�� /��/=�L�H����IIm����hCH��A
�������P d�^ �P E��% �.���a�N�� D ��X�S\'�79���3RFF�u�-:CD�������)g��u� �������4����x����I�&�� ��={v�~�%#&���2R�k�e 1��A�^gI��� (�2 ]�����n�"�� Y���"^ )����>�*�?�U�������Dd��8qb��I/����9s:d����W��>���S�M=W��>+v��hy\��`�J ay��'t�� �� � ��H i�e�)��~���r�C�!���>��tv(Y���=5�M��]7uV�����'&N��t���s����&+�))6>)6^� �� .H� tR)<�������g���q�V�%��p�G�\c #R��b B�������n���KHHHMM�7o^ �j�rMk� �� �� ��_J�����������K�������22�Z��m�^"z�d�C� p�wJ���=u�������������\� v� (��� ���L�i��YM�R���"��jW~�%�T.� ��j�k5��GO �`/��S�&L�0g���=:i[yO�j�������L� �2 ��w�r����9���3'���4.%C*M��_[�7�o�M \�W���ri'���:���������R'q���5���K��>}:!!a��7�{��v
DT�V �1"��j"/c��B�90��/�VW����[+����o�Z ; �%���x�Q�7��o ."���Y�:���kFo����� P��`�-/�u���{�Z������>2�u�c +�,V��o�I������_�>}z�� ��_�]9���A��#��Ma�W~z������6�J�^��Q�p���������TT�
�G:S�����Ux� G�k [7�(���m�
� e���� @'s[��gU�Ht��k��/a8��2��Ls�����"2�����/R`�&)#���O>����.A6��Q�na��32D�6�����m���=F��T���|?�{�������m�k�< ���� @'�s�"�����D���j�@F��M-�W�K�4wo�8���?�wn*,,LHHx��' y�"
�����z67�z]L�
�/�brs����d�
�V����|��*_y�#���� t$e :��7���E��I����� ���>6���6g�Mn�),,�0a�c�=����J �l�����en<�d���
<>^�Z_�����D���Yl?�J^~ ��
����� ���� @'p���g�����g��j������,�A����k���Bb��~kL�Ow��c��6EEE&L�?�c�=� <.J����p�V1���k��g���|�^��O{�^X�H����D��^������]wG�+���t�
?;so��6��� tt_ ��\�������^`���y�S>����A��-��9�?n+v�Dz�]75)6^�s��Ql�uH�@D+O\?���]�C��j>���{O��}����]6eee7�|��>���O� 7>R�;k�*�!��g���7����]i#��q�}����?!u_:��m�wU����;�xZ \�1��V��g% �I �K�~d���������A`#$����)2 ���|�� ����cg����+��#u��3�����5D����U��3�q�U��98U�b�g��IHH�;w�3�<s��:�$ed$��95�HDC-$��k�hG1o��F�� 2+�CCs���>Z�BNE������g% �I �K��(�G!5�0g���� v�i�Z���T�V ��K7D {������B"��K ��e
�Z����22�g�~��g�Y�����Am����L� z/�����S�\��tDD4.J'2���C�w�����R��2$)dH��� ��!) p��
�^[^���FHM*rZ� �2�S�4��[����6���,��%�l|�g�[���ef���+ed�{�� ��(O��Y"�_�^}�)R��i\T���!) ��
� \j��K�l���hC��C>��D�6�4.�;6JGDg���8q��Y������� ����p�����[7�p4q? 4;e .5}\"1�='U*c�O���ja���dg:x�S�1�=?�a#C��Fxm��C�}(g��MHH�9sfjjj���Q�J� Y�( �aDQ������� �L���"�M�U 'u,�[��;������x����Sn[�x�������3f��� ��?]!;���������� ��(��^�kRe�� ����� �ce�O5�:?���O�Xje�~i|��a��S{��*//�8q��i�-Z����W]��E IDAT��U�YpM�9) EQ�����4��� Pp{U�����S_~��_�,I��� �L�|�W���F�9
b�JUU��7�����l��NZ# @�S��=
� ��+?�a�����8��L��&��#���I��M���TWWO�<y�� Mfd�� �s)1Q�4�L� ���3���E���.�-�V��/��2��?���{�K#�Y5zzb\|�����o������w���j�O��W5�b3��8���nJ���J\��(�/ JP�y���d`Dm >�Dk��M�>�z�Q���Qi��$�49lP�K'm���{� i��x���Gf������Z�S=�u����h"���h� ��2_�U�� �.��(�G����$7���Rb+��x�V��J)�]o��I�^�{��:08@;��zL�S���r����m_����V��)S���\F��*]�,R�( �2 m�6D�#�~�����3^���u�-:8 ������'�^}P�� ��?~�i �h��nA�t�a�Z'O�<f���_�����YY$�Q .*$e ��4| ��� g4
_��I�E�<RX q�gh\���X>��^����|^�����X��L�2f��7�x����l����R�23[ �� �N �H�������Nb���y��~��g���,�TG���1Z����b���J���X����>rU��5��=2�qQ��6 �%��:7J��j@ ��o,��������ImTMx'j�,�tY��N��'7�
��;^$
������"i_����J��m���w>�0L�� \���j��2 m�����|ZD*�������C-?q��nOm���w�N3d���������32D���������~TZ�'�7 wT��Z�YS�7�� t!JL)�2�i �mk,�������bgM��G�uSj�\��E�2��1�3;���=���j�j�������6�&����S}�m�
4h��U�� 4I���J\��(�/ J0{�Y�� ��������'bB8]���V��H�Ct_�? �>p�,�SSt������2n!���5�����Z�J�B ��)�� ��
�vA6���?�r ��^����ZG��%"b��D"^���W�����=JL}���� }�V���_]�������>x6f����9��F p��� @��l|����&K/����=���P�;~���Z�R+��K�3�z�8��O4����Fp�;K-G�������G��x��+{:�UDT`�J9��������_� :�2 mdc�"���I�l����P+:�(T<}������ �����sR������o��u���������v�I��>���=��h2)��&���N�S� ��1���{HRH�~ �FH� ���H���DB�l����Q�C-9��Z�9���?)������?Q�W���rYC�{��m��p�5 �s� �D{#[�h~��n�a�mUM>e� �G������ <Dd+�d��� I�4y/ \ �) p�-1�����T*����Xx@FF��l�������_��=�J������gX��LQ���V� y$��r��f����w��zV5q~��*��!�������wh��v|E h5�� ����E��`�]>'f7 /�#�?���5�� ���������"��@�S���'��M�!L��5��=��{w�}��Uf9����[�o� �6�zI���m���?�x��AK�.m<���?�����0�UW]��k�i4��? �H�b��b�e"*��M�oa���/�� ��[���589��O(���\����T�Y��L�i��$YM��������ik��]y��
}8���� �����%Q���?�>}������l����/==}�� {��7n�c�=�� � ��UV4��h7�zuD���T�>��Y�K��0��v^�VE�\����a�N�� D ��X�S\'����=x�9���UL����I��H���k����� ��%e�Y�z��O>9`��&���o���qD4s�������� ? �4x������L��9���r��o�y%'i����; ���+���>|�=���$�H��!�,�# E�Q �� YDD}��# �w���T4��U�� � ],)sA���QQQ��I�&����) @0��sb6��Y�t�������I�u���w�0,�����*-O�� *n��t� ��}�\��!�J �3��RG0�Z��Xtr��K�--����Y�����"����� kVJ�����E[��^(o������] ���^M�����j�������S�|�2�a��/Z�h���mY+ tM��Sd:x��Rd2"%��_��@� o�-���'����j���{���e�O>�E��MD�*2�|�L-
�L���g�{��h�-/p~������x ��zu�
�6���sC�^gIn�� P���/Y���W�
�j� �����N����
�����@F �r�0g����8x���;Z=��������~����� ��G , ��H�V�hV����:2��n����{���� �X<`����������$�����K�������tcy��'��� (�����{���5�[%e8��%Y�n�?��� �(rZe����-�p�g� ;,
�-�j���}����fi�-D�����3= �����|b{����)x}b�SX}����MDKFL6r���������
�W*U���T��`ze���+�����Fct���vK�0��Oz����}�9��6�> �����%�V��
�����) @0�
!y����[�d?�� nF�����+sK�� ���Q� i�M�;B��s����O>���f��Zo �?�[7�p���%��� �����Y����*]��� u���p.�7����[Jj�/ ��[%e�H�V�����[��v�����o�) @���s� �H�lNis-w�Pb�$ #�l���lb}A���c�Z?���V{h���W�y�
W�F�����XyE�0=[�#�zV���m�G�% �����/��O>��c� �@D�7o������ �S �H%~�k Du�p�q�������q�� [���aY.6�)�������y�V�%"{�=��M�g�y3ul}�
�� ld�p�0�[ �\]l�����={vMM��n?u�T^^ =��sc����x��eee�'OfYv��Ao��V��-
�Y�_"�p���T.��j��F""A���WQQ�����tu��M���t��Gz��<��V�9�H���p�Q�����f^a ��]��Uo��:�T�[ �d�bK+��� �[�L��
�hY�{�?[;��l�O(���Hs�� ���g����e�?#CD���Y ��Gj
��PVS�����1=�
_�w�r����9���3'��/ \>��j��v� t������,M�X!99�qF���$ZH����Ri��F�� 2]� ;H�a1 ���D��(3� ����9 �3�'�����13Z[PF"�������KKK�l����;t� @��W{%�Ii��� �K/=?'��t
���-q�k�T�g��� ��wO=;�
��H��� ��P�����4��� @������g�[��G�
!{����UV��_�� ���(�e :�2_��[Kl ��p����� =0��{a�r��4�3��>^�U�. .
� �r����u��(���B��y��>�*��V��]��Dwv�* �BR �������^5���B��+{W�}6;b��QO�R68TSq� � I �`��&�^���w��������^|��)�K��~U�a��e���W�� H�un�F��� ��]{�U�����f���Op������J�5 nP ���� >�dY�!-� ��s���w��Zi��t���p�������# �nI���(� �uM}F��D�����r���������.&77���{�W���C��T*F��� ���>��j��_ ��F�RG�z>�� g�r��L�>o��z9����9 ���� \��sR�V���2�H�njRl|kg�s�"�������GD�OVJ 1��D�l�fP ���K�e������o|4nzk���&�GAn� �K;e ����?�r ��^���{U����'sZ; ��7��:������x �C��4�W������G��,u�&�G � h?$e����x���^ ��������t��E�w�v�e 1��kF��$��x�0{�<BDUVN������Xf�`c���[�~4�"i[��Y{K�-�f�G� ��Y�x��u����4$e�"��� @WT���"��������O����Y��Ygi����>�-�����#�>���{���~k��tYU��T�R�3r -�Z��W ��;�O,w
����_ /�d�(�� �m,^���:��kij� @���qCD^����Z���K;������N��a����#n8�CUV����?��tl� ��J��'��F���}�z����o��{�n�.���ztf��k��DA���jaD%?�DDN8�B���[7�p�G��W L=�(��Q����C�B���O4�b��K�. I ���-���Pl`�D�_�Y~U�9W�v�#��?nt�Ft�H=����������2t^���������*����[�n��kWhh(�$�/un�5k~�������D�j T�d����� s�Ih�
�pH� @�7���I���N��}k'�r�r���_jD���o}�fr���_�4 �p��/��b���aaa�q�6�����@E���$L��9|�g����2��{? �2!) ��1\^A�^��I,L 5,J`Q�Q���3K�
&� /��������K��!"�6������A�%�Ef�`�;?��'� ������������rd����^��� �I �����pmY�H�<Yr!5��P����U���+��l�yk�H��9��&���n��e����z5����Y�<��' �!��N0���i\�Ndh�qG���S��y�qld�)�+��\��F�������7�����#���� :�����0�- tm�����������E�>�ggS ���VMr���{���FtH� ������g�.pX�U���YYYM�����������=��
�n�U���v�������{U�%���&��7t���� ��S��=v� @�wE��
�}�Fl�*���/�f�����I����# �����+��?���u�~�W�a����
I��(}��o�����D�SS���?����[�$��R�G*Z5 t"%&��F��4 h��%��@�#��s��9y{�������C�!����Mj�F[� �PQ�Q��u�z��l�O(n8I�9��VM p9P��=v� �ea|���1��t{N��,f
Q;z �o�����@A�?)�x�����}\b�����F����Dg���z�DW�5��Y�[; ty > h�+?�����H���Jw����vix ��<��yK�E^ ��67��7�4Q�e�~�!�q�:�a�*u�!�qN���I ��(q���(s� \2��9�?n+v�Dz����w~Z@!c�?���r- � �M� j���w�vV�B��p����R �H��j��2 -��?�r ��^���{�*�
���D�zp*���C�Q�����������Ot���]n���-)kn0 tK�) ���9� ��YB��T 8@c����(��'SM��o~q��j���Lk?��[s�[|�>R���4 �-a� t
�������u���W~�%{n��x��8�I\�5k��?� ���o.-<�
E�S���X���%�G�*���?�U����u48ED^g��� �H�) ]�T^W� D��X�[jl���
!y��NI���>:�_��4b��2����Y�G��=7��m�at�jA�9D�a�{��6� Q�SX}��T��s� |����| P�� �.�~d�������~d��y������i�4^��IK{xN���P �������H?o8��22� n3������M��^�� ��� t�x�'1.~���M=9;�������io���'&&���+�L�;B���%��e���^�6
dT��4�2z��
\&��Ji��7 ��R��`�-/0��~O^s�[��:���I����t{N��,f
Q�����/2�W�NKK��gOlllsc����+�
�2}���nj�7 t,e��c� t��K���S��r����)2wg�������>^�V�:���{�}���ed�h�`��e��:��9���x � �(��_��\���j���D�L� \n\�����j���3ZF�jU��A�/��UF�g� �[�
�c������$�9|� �������%K�������s���F�1l/�9�3L������������S���8>R�� ����j ���� �r����<����~�m�JY��� � �o ��i��H���R��9y$��{�V�FJ"3��3,+]��� ?~s��5�fd\{�U�����f���Op���}9 ����j TK� =z\}��-��X, � �f5.��s��I �'[������F�����Nz���.\�{���32D��������r[�") � ���o�q�1���J��\�y�/��� \����>
�Ov}�j���!buz�F����v����{n��]C����D�\�Hy�# �X�x��u����4%��Qe�q � �N�� \ ��fG���s��}(|��g����}7����q�hhp���4G|@D6lx��'w��y��W�<[�\�P��T ����j��K �e��-�W�M�6
T���������w�qV/�y�T������U��D���V��\B��%b�:K2m�����q���g�����g����y�.�����)�3I�s����Bb���|o�OwJ��Q� � %&��F��4 �m�a8�e�;��YM���� %���r��v������{�L��q�O<�c��������a������ ���D|�,�| ��b ��]�C��j>���{O��{������ x�|����_qqqyyylll� =~���u��� 9r���iq -��%#�sR�/5����
�0: ����!s��O7m���������S����� 5��[��8�}W=�~�Jha����S�:�yc7*�������7 t�K��^�z���+V�0`�#�<����v�Z�����|�� ��T�%pJ&��&]�
��a���_Dw])��d�333 �����J��P��OR*��<������2G�,L���f�fX�RJ� �
<Y)%�0C�� �2���2����]������������n�i������H� ���_�E�>o��z9��}Q����#����(|����S]S��?���� ��}��W_� )k��O��1l/����:�����ef66��C �H���;��L�� tu�K�8�N��LD������[�H��iZ �m]C���Zt��k/�9�DC���h&�'����yK���a���K�h����������k��&�F��% W�[�O�q� �/ LD��t^k7�jn` ��<6J����^��#�� to�;�TUU%�P]]��_]e����^ tw�+��5'=?'��m���(C����&�������t{N��q�zd�}����7?��C��m�22��%^� �T���l�e����5�i')���3�+l�07�j}��������&b��9k��A� ��` ��Z��<x���{���"##��������EX tg�+��5)#�p��L�!�{U��LF���`�2�� kvJ`�^? _������1���yM��Y���_�U_3���H����9R�O�;������g�[�N����|r�k~GLsF.������ImT�\*��
�[��/-_����_�?��� ��������c��7 �Ux <���9������{ ���d �9(�W�.,J�r����������NfX/��_n8���� IDAT�p �{m����D�3�lH�e��H�@��c�5VF��*� �����������/��!!!O>��^�o� ��I���V'�T�Et_
8�X��������kL�
*��Y3�0s\H�A����2��X�� q �J+�G2��o�T��w��h ����� ���.)#c2�F���x.< .K�����kv
��>.��8����`GB�l��h�v�u��j�����-�H<=�6D{my
B ��z/^�v?��l���U�w=�i�V�� hA��/5)))��� @�$;7$���Em�g���FNCDW��II���d ��x������CKv�*|�����B}P|�z���U}� Z�N������O���;h1 ��4>7�u�a���x��� �� 7���+rcWY��~ZroN�� ���K$F������yg��-[�� 9r��y�S���.�����o�!a�����������O5.J'2��������S���86�vK 2���#`��L����_���KMM 0`���� �m��}������Z���/��0LP�% h�����sC��#"���7�S�MVk�������{�����e�O���������c���s�~����F�"���F�
��P��~�r�7;oH� ��)��>�5 <xP��%K�� ;v�����{����s������������ @��:��zpAs���V 6f�Tk�e���5T�Pdx�A�g������ ���7�(Er�[e��'��FH ��2_���)#ed������"?~����;~] �-��-�W�M�0\s�X�U��3���������u���W~:i�B6FK��\���~;{����LF��� :O��/��������Z tC��$�hR�:j�y4�.�E���d��02
�0 �+�e���w�?n��5w������ ��h��bgM��G�uS�P|k� @�.)z������#yyyf���W �P}��2�
#�;7�������/�4�u�����aI�������n�m/&�*o���9wF�����CD�����HRQa�&)i �x�k��l�������K�� �7o�K/�t ���/u 'q�k����;7��� �O�}���}�i�f��!�>A�4=?'f����O�lz1=?���}��O���9&�����s/��.ed$����?O9�Y`��}���Y�~6 ����:7���[�tiAA��f���^�ti���/��B��� �"W~����������w2��������n �xV�����;7�bBF����j��QF���6��+FN�j���������s�=��OBm�����s��u��
��0C$��� s���(y ]�2_���&�Q�_ ��K��i����s�����������4���9k�SV����e�t���� s���7�s�=���OHHL��#�p���V��Y\���_T n����~ ��R��}����~��o~�!�Q&e�� .��5�!����8��O����{i7���aT=�Y���>q�D
H���Fp��`XV� �'r�XXW'��iL����8U�9���w��Q�� ��wfg��}�@�D� �� �7��V� �V�_A@)�Uk��`� G�(�V�;J-!$ �������������&@"��~��y��w�Y�`���>O��G��� ��=�wzN555�w��z[V~.\���o����u#��_ ���yr��*�Ec� ���5O�[Ue��>�w��W_ lq�,�����kz F}hO'��c`p��t��5�;������
�.���}��/}��gO<���I��fs������ ����������F��8-<.4��|��#B#2Dd�����AS�����3k*�������K�� �������E���[�x�t���#��z0o�������s 8�-�� ���5����'�iY�j$��i�
�?����vJe��$gE�%�Ud��qa��3sQ �l7�� ���8�������k��[w�f�=u�5N ���O�~:���
1����EMU
��oz��DT����C�]��6���KzN~T����J�������16������`���jK�g�Ag�'��^`��o; �E��O{�S�#""< OX������ ��c}��%-'bDDZI��om�o�zQ�
��w�y+��5J�f���������L������kokp�+\l������Ed��{�~g�u)� �u)K��o-� p�:(:|���/�����E��'M��v��30���{�� �v�,� Jv�jX��V5��tS��.���=��9��Rdo��C9m��V�y�{�Gi J/��/�������os}�S �����Sz��J ���;%����( ����O�������k�����������{���� ���*X2�O�sE��wm]��>�h�k�*�iZ� �2 ��m#2D�C�"w��={ e\CR�%�q���V r#2D�Q��C���L��UDfX��k�� "��*g� �x&��h� ������^��s��5N�2��� g+w�
���\v��q��c&qf���n�� >T#2*�����' �(J-k�������O??�ce�K� R{w��\���uK�A�
Y)sC�*���}�� {��1�� �����2m#2D�l��.� �� {�����=�N��� ����cW�Tx��T�����M�[*��!�����������'L���+�\�\�����Vc�=����c
bXKL��)�&������3��D� �q� �9 ��9qZh~ 8?m��x�ea�~��v;Qs��6N�d�����@�H������l*�����O�81gZ���S,�ZAb�t���Srn� �h������'Ew��%��/m�
'/U?c�%�(jo ��a��*�G � :�}��v������GDD����8w��Q�F��� ��b[�g���YBJ�R��1�8�- zs��Kc�[ :q����� �7�����r�7�����K�%�L�E��� �nlU�
���~g �8 ��$��R�����s<@SbB;�N��`���
�nyG'6���*�][&M�� �d:�),,\�h���������j��~�������=Y�| 8����
��S ���y�L�AO�����]��m��~?1"N:���a�S{ _��
M�6-�~���y�v��x��|��+�c
�V����b75�Y �������wL�qmh<����I |* ��u.(�`��7�xC��QTT���Kg��������� @wW���sNM� �$�������S���IH����y+�W��h��?�|k�Q�&�������C&��9��$�%�&��}��Ts������\��^������:����?
�� E�3q�HD����&�y�f/�8%������y��-� I���� ���sA�������������tJ pvn����"�����fM�K��D�������8e q4����j����h�3p�����fhL�����d6gi��mzu�mH�6ueJ� ���l�1p�����o/�k��R\����4������ /����K��6/{Y}`�����6%�lr���T)���z�U&�M�` ���'&� (�[�w���2Gh�\���f��������S�=a��O�������=���%~g�`^u�C^���=_LO{#V[+2%N[���]SR�{���\�����=9�����l���y�X}�%���������j��~�����/��M���L�� ���e.����7��|�����]:% ����f������3V�����jc�fo�W��E�S�Q�1�2*�9Kn��6�����w(b�>������^�8.�v����Yu��$ ��T�"���������W����(*Zf=��]���nH\�Z���V�� �T����/s�1�� ������� N�8�����n ��>�����BB ��D�^����q�_�3��7y7y"����sBoI3F��N4�W�_��1�}�-E+l��4�u��o��D5�.f?�#"���w3�*{ulD�������G� �����'z ���s+e,�� l�Zg��1c��������[,�349 � �]���MW���e����a{�����=�S�Ih��e��q&I{�'6�:C�*�[����S���H��\�An���]����A����\��@Gw3�a`���/������bN ��{�L� ?
� ��O�1|J p^��_��'�n>gc���aL����G��?�-�K �KM�;|aBN~T�aC�/���j�]m�M�����������K�H�@D�������a���t^��[���o��z^�~n��o����������~v�u �l���=�\s������?� .+)�~���<`,�� ��$C���E�m��)�`��������� ��K��iZs
������]8�n�{�� t9{�m�"�ES-IoO��<�)�[<���?�/�l}���,~��8?��� y��5����7��8�E����^���� �����c��'rl���%�����G�b����$�����}�|�e���|�k���&W�����)���Z}�D�& 8�t�����)�������&Lx�����vO�8q�������T ������/��V�Tx�"o l\���-�<�M����� �h�������������`D��t���}�h�����-���}l��W�Y������7��@���r_\���7$�
M �1z>���A�8��@L���(b �L�\N��|�=�<������d�3g�� �� ���4�[�V�Y�����0LB���qC��. 9���� �`n��������I�%����%� ��:f���\� �����aU��������[�f���;> ��W��2 ~�� ���X���+�,((��Y ��@ �Kb��sf�G�� �<��(-��5L;*zl~fn����|����Od?�����v�{<D4�����OkyK�k-w�s>{��������2Z/�|m�c�qr�� ��F��/1�DQ k���]:% �yH�1F�[�[�1n�[ �\���D�TxV t����>���="f$Y.)��^�~�n��_\J�T��E*�F�
T�������'��s�M� 1
�6�F�����5���g���di��zth>��@�^�z�it
�� ����A�y�J 8��(�J�A�r���������V�Y��]]es��1����D���"���YL��^�\|X� Q��\�1��5t��LD �o)w[�!���L����K�is�+�ur�WZx(��Z��:< 5��8��". ?��e���V�\y�-�[��Y����e� p���mz��&���.�@S�M �1��)��v�c��[*j����U��S��TVV�^���G>�>3������c����^h����}WU���j���e9E���}X ��>�DP ~� ���={���~��� A���O��������� ����JA�])�����}Oa�b�
x.����g��+�S� �m)������ )�MR����V�0n�h��O=�H5���� ��~J�����������6��+�8�w3u�b� ]�h�m���~g�z�I�����i�Dt��������L�_/�
AM��a]}���^)S�7<�9M$���zj��N��������G �^�T����z��
1��Z�C��_7}r��C��,K�������P�jJ5F, 2�������>l V�I|��� �9�{����J���#G� ��S ��W���`D����[��[��5�15]|�Z{Sh�� L�t-�\��l��.��;�\�~�i�;���BJ�R�IJ �#�����C4�7������y<�8��D,Un��|�,F�����
�������q�_4��Y�������l�� s��^d �� ��~JP �I�29x�6���_.un ���da��D��-q���~�t-a���y���^�� g�f~Z���V\b=��Ok�� =X�H?(7�x�
���&r2"u�4f{Dy�$���j�%�G����Z_��G5�� ������� ���<��e��9��\4��s2���c^��3
��4
�� �g���;�-��QQ�w�}=�P� �*<x6��k� ����E���������HB���O��T��"��B������>���{y�� x\rqqh_u �������v�W �V_�8%�]���[n�3gN�[�/m;+��w�M�Z&���v�<��\�N� '�)�_����4���d� ��!k�O�4 �\�=� ����7�<���o��fNN��r��W��o'/; g��yu��r{}�C �-�j ��7����[Gd>_oJ-V[+2�tR�>����U���9M���AQ��\�~_���S���CD�/owV~������<�����������d0"CD\v:����� ��u.(�h��%K�DGG[������v��q�a?�?��= �3���^ms��J��]���_�<v����/� �z��������������� +��<��a ����&"lL��0w�\��]�Dmh���.i���^�/�6�9M
^-�nV ����@e��a� 8���??�s���K�:�q8 ����-�1����S����� �sX��eu��|��Z� .���y��e�k2������p�, ^5&���DD����x�G>��� "���� ]2���������@������!tMr���/�4���*��E�����S��X_\��Y��6��\�b �4������{.��\PF�����6� ��f�
�O} �W ��2 �y�C��:��n�=~� ���#��n���]so*���� ��z�����f��
b����������I���d�"xK��g�yo��6�D��< J��L ���� �[9�"/_�bI�
3X[���8��Z�(�+t��� �j���q���^�z[6o� �/ �����9�7���R� �L���=Q�����c�SnN��������� ��w5�+�;w��Me���X�_
0LB���q����� �����y��� �����3IS#M��-��L�$~��97:�eS�4|a��x+'_��\�%������("����o�[SVl�e=vht��_y���/@�_ ��:�y�����_�� � "EQ��[���/��gfn p�V�|;s{s����q��5������2��r9#5�%���^���bF'k���7��c�<C�4��0c�,3�g��_�8����VJ,q��a�V�}��� �����nv���45�w�oI^����U�\i����q�����/��8��^��%"������T��'zS ����� :��c�����W�^_|��?���O}���{�� ���k�<loL�m�;4������� [o������)��K��S�� 1=�������@�����rH�� b�� 3�4?�[#�wPD^j��'e���������������Q8Gq���Ts�BCf�zq����C�
Q�K?({}G� �y���}w�Sw�=�� :B��#��.*�����Tg�)y��!�X�vMDV����
/s�1"� ��p���m ��d�:1ca�N"J2��,�D�d�F��M��11No��_Uk�q7��c=L2E��5����_) �bF����Yg� �I���}��/ ��%�Ul� m�a� =-Zf�9�&��%t�P(mtU�F.��C�tY_$�V+HpE�9\��>+��>����A�[9I7�8V2�
�Z��Fd���Nu�����* ��'�� t���T^^�g����&"��w��>�r��331 � �3I���I�>�;>xZ���if���/ s����E�m�
CU��m�FcT��[�1Qtf�>Uc���8�2C�HMR�&)��72x���
���T(og-��]���MW����}���E�j��}����9�]�������
�l�����E�_ ��:�Y�t���3���������]w�u�|���o�y�& �iZV�������� �����|k�������i�
&�X0����;��+(�W��8���.��o-�Ma��z��� ����� �k/��� IDAT� z�T��G���c��M�����8B�A�� ������s�_eJ-n��`�(H
{���C=�T3��cw���.����=v������� �q������_o���b����Kw�q�e�]6|�������;�<C� �3'.w��?���DdJk���U_�aD9m{�F>dW �m=����s��
a�u� 7�3Q���VL�#N��(��8���Q�f���f����;��
��Q����oL��S�<� �8� �������M4���4 ��:�R��rY,"��u�����H���t�325 8mjI�#�F9��%�� . ^ 8�cMH�"5��8 �[������V����+-Ia��cm�n��7������O��ox��`���LD\� �oJi 2>r�b�9� ��� 9b�!�V���l�^ly�^dS�M'� ���s+e��G�Z�=z4������I @�[��S�O��o�����������a���������'��p��������r���Z5��2��N-w�+�����(������KY�m���bB �PKD���Q/�;��e���=�\���H���(<����%�����IJ�� ��c�X���R���������!��o���;:�9 � :������eKjjjJJ�������� � ���� w��T)!���^-�H�^'��j�`��SF]tO��/�~J���d���t+ ����oqJi��I}/`b�[�81F!��g"WwN�l�.����������k�c�I���w�u[������\���I�1D�.Y�t�^�8�(R9u�^w�V-� �=un���/����O�}��s��%����[n���j�* ��-���+J3�����v�,� �6t��S�|���m����i���������+��H��J�[ezv�>�q��5��WV�*Z�����x��?5F�1�?��d�f��_��.H����������Y� %�n��-���gn�j��> ��`��0
�)9 ���������.��9u7���� �?S�����r�3��jL����yl}p����E�7z��i�����&)Q���t�:���z��� P'j<�?� �(����+��?x�����b�z<���k�O�%���R;��[���5 �����j��k����6"�%�M�����=I`���|GO�7�����<�? �3���}��/�1������ @wQ0�:�!"�3��x��O'������t���c��D��qO�bU�����j(j 6`pY
}U��W��I�^�-O�����(D���H^ic�{��g��v� \Q|�K�"�w���=�����(L���������S��L�$�%�ab�z �zH����d]x�l �n���2 ��g
%��cD�AI�����8.�:3W��[�����>���d'�I����M���(����|���d�,���1R���:�R�u���� ����i���b���S�}���� ���gYy��etc��HM 1 �/�U� Wf�K��$6����di��z� ��%�� k���1w������ T'CD>f�������9 �Ot.�L�-Zt�� �i�V�9�k�7A h�� ��Rs���C
v���Nr{Y&��g��4f�(��lO�F����
��n���fNDr���3�I�~F�-�� D$�u���]����g���)��/�������W<��� �m�/^�b�/���w����������r� sk�����h��k�=b�� �n��-yg�d�B��E����l��SaR��s���-��?�� 8��`�����O ��d�F���~����1 T����GD�� �n�7������*���(5��Z�Z��DD.���� b���R?[p�5>�9fb�Ka�ripnhM%A�3��� �d�t���qE�KJ �Wn7
m�k�.k�:U�QTVU��&���D��R��xW���������4$�{��������j�A� �i���`�DC�� ��eEU� �!�����"q��(V[;��$�]
��HNw�-��}���ws���~ 5�� z�_ ���� � ���Q�^d
<*P� X��}��$����� �
p��W��E�U������a���J �$��?��e��[��(�Y�M�6���$i�� �8s�����5q��1�$(�zgN��:S�MK��{���Pz�M�6��G pfu((���;f����~��Ym������g ���ZIZ'x�K\�ebM���O�a�qYB�R&�<��l�d/��L_������Uh����8�n����U\&@��� ��#HM:}�3��E�������$�
��D��U^���$"�Xm��i�I`{myw�����}g��e& w�!���'�q�Eq����y���Y���Yy�z�h�Ag�'��^`��~� �n�CA���������v���x����]?/ ���\�'0F�'"?��� �����>q�^���U�E���v|� ��;��`�� 3}lzX=���iDS�@����*�m~B�(���T�����7CL���������|{�m�� ����9��C{M N?T��'" ��P���{�R[+_�s��2E�������F��G# g�ef��� X,��]���hs �/���zE�{��66�1��-���}��bF�A��wA��/�ni��qY[�a<<"C�V��)�MRO�D}�;���"�@e�K���X�MD����J�����<b!��'�J�I��"9F����\ �|DD ��>�
�4������ �E:W}��n8C� ���`n���&�]-e�\~��"����3{~ ,/MDLc\i�b8����m}u)�����������j�O����w����2#1&�_R �8ane������}{}�cbB�G��^�}��l
c'K��= �@�%�M���� ��Y��/��7�i�8o��.� t��L� B�=���dsWE>X ��&��R�rxo�� +�����+-<����=F�\�Gh"�[�dl��{�z7y���RW�`�` ����� ����W�F�� T4����/�;�F�$j�sI FmvHW��I�k����0a1�S��-Zf�9��Y.�R�� r��:� �\��$���CO� � s�d+���q���-��[K1MDT��2����bjr��YS�H����A.i������V"���})� k�
R �]c�h�I�I��4 ���h�� ���*C��$��At���p�=^�����u �r�T�E+l�fV��"� �m�YA��LkUT �[��� �i��e��9��\4��s2���a 7�S�F"�Hcf'��G�������� ����� �q!��x���������4�'5��f��J��*?oy!��o>=�}uN�����MR�(E)3��ZMUC��m��HL~��ooLY%5�|z�am��&����h��qS��� (�t�T�s�������y�� @w�A�E�������� �r������e'� Gl;fg��iD�3-�8o�g�_R�a��
k*��� m]p�4z]��E���=��8e�����L3&RFFN����Z�<DdJkH�TP����_n ��E�6�����!
��}�R�Q.O����l�cvY�vY���H��M���b\���������^>�Q r o�� �;�������O 8���jDF�e�c�<5(CD9�Qa�S�3s�3s� .�g���Hk��^� ����w?v�w�)����L��!q��QIF_����������2g��rWS7_���4�����u��{,
� ��5S[F�.!;���Z�,s�s��`n�=�\�����< �� �/ tw~gYx���)����[������olgbs�\�(rI�zl`��W�q�?"S���A�����q��O��1 "z�w����`��k�]
�(���J6OL���Y^���� %�n �Q�f}�K_;d��Jc��)����W9������ gT��2�f�:�����w�d � c��^������w����w>y��%9w}+D������U��>�d�4��~GuBK�DH V�|}��]U�[o�jnc������'����j�� Z^�fI���Fg6oM�os}Sxb���o �����2���������� ���PPf�� �/>������� �3Z`�yOp�&������������)��{W0��7����3DD�r�!�:�O%YVWD� �Oy��OD����
�#��=����0o?&Cr:�>��� ������_�{]���d1��v�c��;�(� o�63��<8
���i[k���R������ �D��2�e��'��u� �p��i�jKc�%�7y����frf'N�_���hdL2�!��eV5�o��OLz��G+=
����� ���T}7"��k^�"�>��VU��������O�����%�� ���%l><x��D��L��v�������}����q���Q��w<��]������O���8kw?�3k��*g���(�������:���\)o�w: �i
�L�4�L� NdE��w;��_����E������/K=���m�Y�w��������L$c|y�/��H���u���T�G��#2��*���CD=��.4�6
��~���'�#��.���%�|'��T��,�b�-�_���Z �Z��/ z����*����3�L�6-���\w�H/�{.<�� �#�j|�������w��� g��M��N����� �%�����O �'������z��f�^�hu9���o����OC����"��5I�b������ :�P�E��w��� b�J#h��I��� bB�+G����!���i�C�I�m;=�r���h9%���- ���q���n�b����[�TxV t6z�za���\;����������o� ��P��i����������=�����?��: 8���l-� )��*M.�x�E���Dd�o�G��$����"�Bz���1F�K���N�����q�Q�h5"CD���`D���(���y^ �Z�I��67��`9'��1�� |B���_p���6F��� ����@L(�8� �?�� ~�Y���sO����� t��e��3�^��{;��h������Q��'K������R�@<��mDRfFX�E��-��� zf�?�S|e���z]��m�>bJR�������p���T6�L���KDo��V��lz*Ljz�0=�BA� ���?~����=��uAPf��E�? ���h��d��cW��������S�����N���q&�y1�����������$MrR�72}��(Z)4L�!���k��v�%�;c����� z#�<�G���n� #G,���� i���#[�n ���U�3?v��qe�cw��������# �y� �2��o?�A � �_���JSZ� )����������$��e�. 99� - b�9Z�O ��DQ��h�F���i
o���+�� ���CVB��K�����o��&���b �!3?~ri���������a 5&���� ]�"H��� h�=�; ��:�SFQA�yU �t����&$/������7Q�In�����lj� v��1DD$����S���89��iUDI���~> a<#��a��)���� �#q����Z�������3�zc� �|��A�o7-�8�,XBKV`�� `�;� ��:���� 3f���??k���W������ 8���7+SN�V&T�?6F[�b���xdfcD��D5�/�����JEUKv^F��x����M~�mx%9�*C����������I��;���c&��D$
��x�U�������j� ��G�|���n|�.(1 p.�����O���?���c���aW
�~^ �.��U�n����N�m���tB��$o@��zza������< �0�^���)�B#:!����(,Y��)z�:�g_ ���'6��+m,y��������>�V���jD����U�9��M:]�WZx(qUU�F7� D��L� �d�A�r*�-1� D��2�g�V,����FD��Y �������}���Q��-�w��.w�2G� �Un��?����,9Z�6�� TL��z)#�g��d}�D�&�$��Bb�r,���zs�Ob����[ �i���b�����D��~�^{���������V�e�?[��~g�`^u����:��qE���]��K]� �J���p�
gh ��0��=�7�R��8}��������.Y�I��V\Q�z��+���W��m�����6���d�����R5id��|��������s���/�+z#5,�o/��IL�3����Rm+���\"r ��
�#u�{
Gn3�1������� ���S: ���e~����m�7o^M �|!�J���^ZVR8g��rWS�1b�� j~���DDd$y6��>�k[�Yn�ct���p��D!���J'z�K\�/� k�
�{v>q�x3�e�yO��>f�`z��`"���2l�s��"2����&r����y����O��# ��[�[KLL<�A ������������-��}�7;�iY�d�� ���WU'���\JS�J�WO��ci0�L@���C�����!�q1R}�����6����.�����D��qu��UH���H�n��G$qX-����aO��m��&��}�Z��� ��: �9r�HUU�?�<��C_}�U�M � �������Op�U���_��.�YQ�����!��4���LB���+����))V�O6����;R�!N����Z� ��K]hMk� c,��+���_UMD�Q#��:�O��8{�c��;��6���)�����!3���OD�6����)y�K�v����[�������)�;��9Q� �|�����9s���;l�0�ZV��7��g p��V�ym���5��$D���=����r�����3MR#2*���/���P#2*&�bf�3����w���q����5��hN����G�%�R�>Ll^/�ES�7C�����l �m�do��� �}����\��m5��(���9'z��i��[���4r?g����vc7 ���e>����;w�Fd �SV r�y�6"���� >����e���l�
�u-��% �R��G\o������}9[g���Cz����_Y-�����EU���-F)sDh����,z���F2Ny�z"��mZ����4� ��lj��$�Y��v�m+�s"�~~�mkr��m�& ��Y���������{wX�����u� 8�5����q�e%�4c�Ki
�*�d���������4.�[�x��X����1�~H��l�n2�`��������]�ovo���ib "���="F��(|���,�y��-�FQ#�x�8c�eTj�-�
�V�.�Z9�$� �C�[)��K/
8��k���t��u��M�>��' pnj���4�0F���� ������� L�����oo���/,3j��r*��'o?Z����1��Q�w���Y:�"�X^v��e 3������ OX��F9���CBH�S���L��TxV t6z�za�&���,�Nv��� �g� ��y���=����#Cw0��5��g p���mzu�5��� �����hZV��YU_N�2#i�Q�0�����G n�q���_'��J���P�����K:����,i���Lr�x��)5"�|����}��0�x����\���
�?����,�S�[�V�Y����Z���;��� �� N�q~���]z��_~����S���}J m��� ��\�cb�Q����_���-��E�j�+�7B]�BDJ@����Q����81F������A��SnX2D��Z?�i����j ����L�H^ic�����]+�ID��DM����7�q T����'����_��c�:m��N�86����\_�z�O�I��e��Q5&��%��� ?�����s+en���^x�� m�1c���K�tV ������"q7qU�����%��$��`7w��h���_.�4&|��b��$
� c^0(c�j���rk��"��������1�����{�����A/���_����?6���0t���H^�P�[����'
2)FA��L(�|� ��^u�.��F�$�������Qw<�L�$�w��q���T�D����!g IDAT �g� �������b��.�)((���9s�.�g8
�?w�2G� �U� 5��[m��TK���;��������u}�����n�M�L |^���L5G. 2�eq�2k0�1f�[��/�VR����B���#�A5��9?�� �)
#�y���6{o���%���������9�&���b;� ��t����[)s� w��??���� ��� �+�%+l;f�A�
����������� i��F��k�O����� )�����yD���p��
wnY�j�X8d�E�U���V������� :"b��__bG"�v�V{�c�z�@/2������ �7'l��R���vgKe @P��2m#2D��c�u�\ �!�A��������� ���-�@s$� �����gO�������%���]U[���� K�]��<Dd��CoD!�FXD������JIk��a�\T�8����J<Qeh;��z���9���K���� mu.( �w���jqE��q{����T�%��O4���~)�Z���W���)��^_����p� H�� ��~����&�7_���%HW ��4Gv�������3�I[��xS}X���� ��N�����_>e�W^y��' p�������*y���P �� y����=�vFQ���-B���.j�;�v[Q2���^�Ih�kA ����x�/~Y���6���g �h ����n��? "ZSs���cn� `����{j�E�.� �j�����? �u((�n��S�����O{2 ���Lj�����vo���|�{�vU5�c��G|�fV-�Q�i�C���<R&�8��#c���d!��@Uh#�t4,�t������7����N��U ��n��(��t5�o8�>s ?y�<b����
����1��[�=z���3�1 �a ����?���<y�}��u�| ���i��Z}I4�w�] �A`���zJV��3�������i��W]�r��4}f$Y��"�>����x�hp��T��G�x������ ��.�z��u����s�4j-gZ�����.s�B�������� ���PE�td�>}���|���Y7 �.���m;� ���x
��=q�����?����kG�1��e�.��a����U��4�ZK}�h���t� -o ��t� 2�i������? $��,6^��n�5��Y�����vp��k�*�����:�n� ��� �l�=�w�9u7��_ �:��e��9rS9q�D�%�-��?n&"�I�����i������dW&��j��?5���k���%r'�� V�����m;�)(���P��Py������ct���A �C������X������.��|Up�^d�\h��� p��?��������/ �����9�7���R� �L �9�7�;l�� Z��u�z�S�w^CD���K[�x�<��S�MNn�k�c&1�v�*�w�g�j��>W�3���a;�(��_�gy�P7�W�_`�)��"��BK^�#)a��S�/��5�. ���{����s�n��� ts+J��{���2��!"��'����t���wT���#�����x��e��mK7zC{�z���� �����i#��V��_{�� ����$���b� �g��O�U_ ���[�I0"CD^��Fd��Y>���B)��p�+W��V��� ��$.���o��+�.������<%���:�.c����$f�$P����� �M ( pF��.
��8�l8�$%�FF���s#�$4 �bJk��[9S^����%|U��<b���x��{��I�/����+�?}� ����B��O�A��[-���Q��v��EE��W� ��@P ��H3F��~# x�}&Ps�D��$��� ���L#d�Q ]��G5�����-^E_����?�<++Km1l��jj}����|��l�hj����P���if���/ s����E�Oi ����� 87-<������<���+��S�{��&���?��I�$%J���NK�c�� tc5Y������UQR�U���|J��>��� ��g�^�=���2u��@�HM�^�K�]=�7�R��8��N�86�s����a�#���QjP��q����y�� �� 8#��X�� ����z|/
(�y����&��5�IR� �x�mD����D������l��)�[i�����5%��cz��oV��U��^d��cV�e�.����D��U.��F�$����K���_[#"o�X�[�����? h
A �3�c{S���4�����#%S����D�c����%�ng��!���8����6�������7L�yO9�(0fA��x:���i��>����:��8�(#�� �ck��M5"���� �[aA����jDF���� �� ��� @�s�, H�������e&����������^��g �\��W�1������S��w��e.�a��[�i�o�����x
�{YF��C�R�X�6������$� pr� t1w�
����a���� ��9����c��������� ��MDd�I��c��*
���� �I3F���z�s� ��MB��/�^�ce<���� �����&��d�H��^L� �/�$V9[���5!( �eP} ���f������3V�yYIa�} {�r�)��r�%���4u�Wa��u�M��=��O%�Z�t���.�-�3Q�23��^ Z�z��q&IKD�<���-���� ��=���=��]��*F�(���q����4����3��)�&����G/��Wn �S�J ��ZQ����k�����8gn_�8M��
��w�����8W��p�� "�I� Jlk�/��g��E��E�����L��AS5���5s�9w���|�h��
��&�M*� �:�������������w��Lzo�AEEJ��E]Q��� �kw������W���Vd)*�,+ �X iI ��6��s�������d � ?����k��s���p��s����Z��Ue7jE���_ZDF���@T}@ #����u�d�<� �����}�:�V� N ��$�i�����pz���X;|J p����C��&#��c �V�^���`��.%Y��'�-�9�K���� ���"�G���0X\���������(���,Q^1���.����n��7���^T�Y��U�������s�N�C ���xK����m� ~�:�O�v8�4a���_ ��;��^)q��O��5�(��O��B���{����J5^����L;^�y@
T0�3�4p��15irM\v9w�
^���4f���"���
�oT�1�9�u��X"�5�L �����K#G�<���7on�� thZI���ts���D%�F��o� cb t\���TT� + �G.��O��"�������1
n�Q>pH��� �Q���<% <�* ��;>�V�
k�d�;~�|Y���>��O5���F�� 8�Z��x<�W�&"UUg����S�k��6::z���6lX�l��$ @�0���;67��!"����U�i�fV�]�zy��sD�'�Y��H������-8BD��������/�yD7� �81F�DDO��Ft��ng&����P\z����62���w�������V��X� 2�'�� �5�:�4g� �N����|��a���y��������c������x��3���� 7{���� �5x
�T���xK_�2�S�D������Q���h��q�����<*;���Q��uW��t�I��+��
y����S�X�Ox����jE����u��k&yr`�A�.~����� �PCW� G����+ �d�:�M[�c�'d��d jMZ��i��K� 8����� " ����g������mun������{/lp����v�L�� g��p�}�T.7������byS7�M�ye���������k^���!@D��=�F �e�8^)
�������E���UW��9�I��?��|f���"��(� � �X��,j�Y�����a�z�`&E����r� ��wZu��,��k ��%���� �m��?��V���t6 ���o�� t�]3�:^Lw��O�"r��Y;?k���G��"2Dt���[�g�5��g��[�d1�����_�tLdJ�t,U}���--����5��������p�
}�}+�=S�=i]^�_��^>F�|Zc�m[({�8�- p������=p�@�����Vk�n �, �����KZ�z��*�� ��cD���"��b�f���=�iL�t��.R
?���y����OPD1��� �u)��3���&"z�� ��:����fb��'x���n���k��� �ij[Pf����M��s�v�s��[n���'�< 8kt���Kz�9:l���KD���d��1�"D1,bT�P�jn �p�:�.IWu�RH�&&�RvV�2� I?(�Vs�}�.�1��Kf<�( #C��
3�&������T�� 8}m �ddd,_���7�����.���� .]�4''� m ����5�IMN9\� }�������D������nDt^D�N��M��^�v���+��e���g�}�4R��)urlL�L��8�&&�I���}��o��Ky-2�Sk���"�(�9\Q ��Fm���I��O�~�^c���5 p�Z�;T||� /�p&� �A�r��or��9��c���:���>;�|�zS���G�����E���F��=U+G+�R���?��=A�������,F�X� �u��D= �`M��+\�
��4���/>��~��HZ�h�� �6���3�g�� ���W���?8��4c��~�W#u�a=�����5��C�T��u�����L�H� ��������t*��r�. �o������48��D�d�|��^� �l�_�k��� ��d�}�4� pC� g���D�U�q+�b�%)��(W�N$�+�^*���<Z������n�f��OL_l8�;IVt��z�Iv�����v���:^��8�*H3���SEe��2��;%�Q9�M}�$�f��t_:Ur
��6w���y���? )B]]���3 |�6�At��Y � x
�8w�h����.���-e��q�����
Y-
��+��7��%'I]r�� H)F�� k�;���d��'b��{M"�i-^�����Z�</vk���(zCh�f���D�E����^z�_|����I������n�ua&s������}< ������m�2;v��?����CD6�m��I�g����?3� h7���'�m �<*�^�|pu0"C!�.��.%Y���D���~��������w'
x:���z�����p���j+�j+H0���;�<��J�o��5�DT����=�)�Kf�
n�����j��y��7� ���V�w��9�����!��������;� l ��9w��"2.���f�83"�I����:������Ed����7�o��1�#b��?�H�g9�(�}G?�>�#�,R�j[]��m�]���pL��~o����V��������6�(��]-D4>'D�������FL�3��C % ��m�2>�/111t$11����h> @�p
q��8������(
�RN�1���6��w�i�F�(�&d�������`�T�Xc��he�\��1� E���9'��P�?9� ��=&4�6h�+?��:��)R��/����_�{��:����,�S^wBv�����eG�>U)
� �( tX���/ �o���p l2bIoqf��(�|��c�R��&)� ��qE� ���BJJ0�T'�Z��"s_���������::F^��L��_��U��W� ��DHx��p=�k+���4X��1$�G�< ���pi������ ���R��&
���;!:�>�( ���-(��[�u��
�X���/����k{�
��E��c�rG��,����^R�c������4s����'����n���%Nv���NW��D9���5��9���w���)�K���
��Nn���k��!i��$�)� ��[+����/�|��� ��Yd��{�w�u���VF���o��y�>J��������j�Q��2$5<� ~Mm+>�p8F�5i�������>���7�|��>�Z�gl�g_�,� ���}Iv ����h����� ���,-�9u�
��p.� �"%���M3G��~6���?���>���+.xy�_/��/�p�&���+��:�n� ��� sR5�����������L����l:� �
[ [�A "����x ���}c��� ^��[�QdS/��!. � ��}�
�Z��5k��l�)S�L�2���v����vD �%�� ������Qo[�v+v�M����C;�h��O�"�)�*�KV�� ���1{�`�� 0��r^��{u^W�K����t�-�:W�@aQ�X/' ��Q�Z :�y��P:���k����$3�� p#2D�U���� �_S��2D�������+V�X���{�5 gb[ gHX��%�g�������D4��n�����[i�h�:�k]gjl�myrvr���+Jp����[j��_���@y�p�����3jR��&��^��n�%�N{� ~��E� %l��� ������G }t�����: g��^,�r��|��l��FD��� ���U]��������Ju���h �W�R� f0���Ja��W>��=1}�AhU�$"�V����w�u�gbxqF��X� ���J=��|�L�����Y0��h"�Y��s�Z�!���
W�(L�I<�d ��9(c��v���
�����s� �K�|��-��@��yb/.)��� ��4
��\@Dsz������ f���M �<"J7G11�7�6�%�**�>{����O��w D*����"S��� )z���x�D
y7����.���"�k���� ��'r���AY�X�d��s l��~\8��Y?`^b��&1�������5e�v��� �I��2_���3�����������l�] ������D&Q�=��+�SC:���i��h|N~�F5� ���#����l���1�h�����DTNV"��^�y����o�
&����h�������������d)�E"�[����c�.�mJ^�(����ND^��~��Y��9w11>2u��C�-
���Qh����� pV�-(������G111��&� � �u�� ����xf�_P�:�_��x:Qa�L��j��E �0DK&c��4�&��o��
m���|��X%����yuSw���G�.�����/ <T])�C��J}�$V| );K��htf_��H�DD�1��Y""�'"���L���$�����jDl ���m �����O>�����c��}� �� ������8������DD��2�OD+�[���cz�&"]r��%���_�_���i� ���/�3M�k����f4�����_�#��KL������k*]��t�J���W7��:�M1_q �ZM���o��c �]��?���}�`0����A����d ���u ���^�LF�Gw]J2W<��%���]�L��)��J=�3��A4J�eo�Lxvc�w��*�Py�[9`�{n���� ���&�o��=��gF�PK����,F�By�?���r�.Ka�R��� �a�-(��SO�v�maEdn���v� @{�LW���t�a�L��,""�:w�""���x��q���sB�1Q�8e}�W���L�[M�>�������?�2X�Gc��s��"�����.���d�{�u��'{�� �E|������������*� m���)�f�� ? �`Z����g���T�quu�$IQQ�E��o�^Uu.���c�8 @[U���^P� ����WD�
��I�?TU#p"z��M�G���$"N��`�V[ �n�������QaS2sH�ts�[=�>�W�\o��xa���H�+��k�����K������d��`=���lBR�h< �az���2DD�d�y@2]��� �9�c��oU�����5k���������� p��z�Y�&g��G����F4>���N;g,�
��R�3��g�$&������W�3`�Q��o���]���^Y�Z"����Ao��^�������Q����I�� IDAT�z�~I�)�l( ���uB��Z'l�u2c��� 8g�*Pt������=��x��L 3� �{&��5�S�[� ����;#o�ElH���"��(�p��(���!�gl_/��2�g}�����(�� .�KI��� ���*������d�~;b��L[�,�H��� ��rs�
� ��u������9ID�� ~]��K�m!q�@�����X��D�������k���l��|����X�z��s^��g�}�@1}~�_�Qoc����D�����
)$"CDRvV��&&0);K �0���N����3��MF,��~/ ������I�[��{������ ���u-"��{�A'��)\��1Cq��=4vVE����#�D��6������{A��`u�� �����,��m���G����5[ ���{�L��9jr�HD�9�-wp����d��5�� �#��}��O?
1 �>�l�� ��R�l��=<�K�����\���� ��?� f�oe�G�^F�`����������,B#�"D���g�E��_��bK}�UO�)$���+j� �h�������r�G
\����I�9j�"S���~ p�h[P����)((())iq2 �����f#'���53��BDRb�G"5�������� �os�����N� zN�������e��"�69� �����wW���T������7��X����u��j��Z������nSe�:�����- ��^�������LSd����� ~�N��c����z @;1F���{��3�
p =�Sf��g� ~��U�f�V�� �~-RcR����g�{��/��;#$9��7�m��������I?`^blN4=p�G+�l�*��� G�������c�s��I!�T�7JY��bvI���� �9��QU����]� ���
3���D��u5��$�+���U�3��� ������J��7�[S� ��8OJ��M��.��$���/���~+]}`�k���&���(�g:���5���� ��f���~+_�jZ�����XQ�+����>5����Z�Jp�"*q�*"e ~/Z�j���>�����[�papPU�5k�(����O�� �} �o ����%�-3��JeK�4`^b���`�^���?O;�#{���}FG
qUQ�����3w�l�0�N���b� �o�D�\3���������py�M���_�k|������a��� ��NQ)=�1�:5vU��[m����qr'D��D�7�8���
�^/4�2�IRT� r��_� \Z�Uc���n�!�'�� ��1��6S���>�x<����|�Ip�1��W���G��� ���������\*9��_M+�����^�LL�Y\������� ��_Fd�H�IY��$e]��9kiq7I �`"*��n|���Y?���C��1Q���|������)o\�}a���CDqT51�5������hQ��Q�� ��z�ls��5�X�X��I�m� W�V����,���8�!. p�j[�h���W�>s���:f8
����.��%���� �Xwt���U���nS���9�)�p��k��q�7�Y�{����4�&lYNt�G=M���<"�N� �| ���7\Z������+ ���j�|� �,��i���IvG"���yj0���G�S��� �������Ne�'�/6
�������wl� \<�"�pi�/�� ������m{:r�Hff���M��1�� �mZP�W��Y�: =1�t��tb�#?>�/b���=���� w{��%�zR�������=��v��2����{��W��w�}w�uW}��������/����{}$����^���4�"v�4]�?)%iF��� `1�K�m�- ���7�L] -������l�] ��$�;�[�� � u��� qO M��� 8�y
��7O��(���t]r�`P|r���@�1�5�g�3_M�u~��T� ��"�����j}��L��������������n������z���!'���y�\������&"G�������}[��� AGM6��!"�������t�p5 9�:"�cB�,�?�) � :�O{��S���m���#sss��{�����v� ���f6DdR�� ��Q�d�{t3�6d����L��W\�t�y��n ���,�{~� ���[`�z������e�����~�� a�oc����_�>'���6�W^�vY��6-"CD��[Tj,�"� raQ��"2D�B��O ����
]-F�1P���v�6��4 ���v���� ����e����� ���_e����_|� l ~���?�H�YLll;�DQ��""!%U�����$F�O'Bo��BNS��WM�����G���
�
�j��eW8��}W�Y�U9�, �j&Y.,�Jc��K7�F����Nw��w�;*'��]����������k�
k� � ���`���
�o������%n[A�qJOk�Y� ����9?;��{���;�g.��IM0�SzZ��/ �9����4�����Wcbk���������+ �]��3��D���Q m���)4XF0G&��HDK
w�������Mq7��Q{���gsND��
�\��� z���� ;��M�33��X_����#" �HP_
��G��#x���X����1��G�����l��j��<i��QD4$��u�~�P������x��(_ t����m�� �7�mA�����:����� ��E��c�r�]��efS�S�����5��/����.�R_����qE�.���g����
TT**��������s��������� 6�k�V{`���1�����^F�D���g���S�OB�|�;������m��J��elf����B_w|N��y�QI�,~��.������ ~����$�� c-T� ��L9��q��rQ���[0)�+�V��y��3I�����&�!�L�J��|�{f�m�cW��XW������KI�r��A�}~��ZL��{��_��[�jI��- 'U���_/�����G��7eO0eO���9������}fu��������b�!Rr����m���wk�,�����q�����{+��Ro. ��-(y���N�:G����`} ���E:��6���v������CZ2�\X$��kT���$}�����n�(��'��^P������U2���%`��lX��.�&3�3u��
w�y.zz���lL���(�/�!� ���|�)�W ��=5!;Bv~������?Z��1"�ua��$�#��.;��*�� �
_q��� ��A��2�?��m����Q���?~���V�Zuf� g��p�s� �]*��"��i9 gm3�DSEI����%`�5U6�����������O��Q�
��5�yLT����R��Q�
2uf�����1:�#b������1�� e�+�hQL�9����o�T�Q�Fj�� �9�mA��]������<���#GF� ��o��>��x�C� ��)\j�<MkG p�7O#�L9�Og����4s��>��rINbi��}��QE�b��DbL�Z���t�9� � �T"r��X��N�l�^=z}��L�iF��xZ���)-"x)��y �s����F�d�<����>g+\M�0���0 �s ���_DZ���#���������v�� ~����?:��vI���/^pi����V����DI���1�sZ�����C���R�=��h^icI��������)�d��)��q���(#�Z������}�h<Q� ��bZ����RyT��D${�t�=G���)f2��?�� ��&S����M�&l(����#x��(�;.@3l �v�1��-S�����+**BG z��o����� gM�u4|�]r: ����K���\;�ky�A.�I���Z�9��w2��X��+�k��������7/�h�\�q���d�����,�v���#_�x����kj�=? y��'��I�l���sW��xR-�c,J{�����!"������|��c��L�����K�7Y� ��-�����1Fal7��TDd ~���1c��������K��
k�] ���3g ��X�Og��n�����{�XD��2M�G����/$���7� ��/� ����i�5��� ~�� 4 )�O��d�18G1������x6��i���}�8cU�3A�`�����V�� �������K��:���7LP����J��?�z_�>t�Y��Z%g�{����� I5A ���mA��?�|��-��
p���5����� 1��k��,�a�~,�-"������fk�2sz���������nN��A��h��zt�b�&U�Z�,����E�r-~�KNj�9� pxf�� �1��5<~���6gk#��}c���A�j��Z������ED�����>�����Q�8�2���T��I�%���:� ����Rs'����}����w�y��� g�)g|��E���L�t��.2e�;��� �f���r�*�����[�MB!�%-J�������������N� �D�KI�zto�l� k���;���io���Fm�3����Vl�)��'(\��'���_����u �� 3�&Q���IE���7�(7wBtk�8 �N���MUUU��=����� ��������k F�� ��������-^���� �Z�O��:��R����"k8�����tB����_|`��k����7�9=jP��i2>�����HZ�hhR#��=�}�R^�`y`�j|�������h^��a������������Z� �W�1���������y��A���`z���{W p.�X�]�����&�/n�p9�B�i����(�6 V��(�3MM�2��%"K
��� e���"|9��9r�6� F<tK\Q��"C�,��� x�MZ�e>� ��- �S� �����K�<� h��e����� ��2 ����
�Z;�T>2uy�T���&�.�.o�"�]�����TNDd>�0��P�S�Y��n�D����ei���j��K� G rl�[�w�D�UL��I�cL��f!���X�F'����G��Cn�d��h������$�mr���w�6SZ_�
m���� prm����?���z x����)S�,^���7��t� ' ��o��c��dD����m]G��C��� �����N�$u��1pQ� $�~�;w�����.5 �32�|Z�i�O_^r�[:�O�UIT�sa8'�9 ����)����4!���h����Y1��w�$���K��hc��� ^��;�QdS/��!. �!t���m��Y�`Aff�_|�,�m���� � �Lb�K �5�'�|�J �~n&���*���G 3��_}Y�j�8%~���Q|�E��u��a1I���������M)v� +����KD���_��I������3W�.'"�&�+.�����SQ���RV[ �n��(��]-�|--(��W�+�� ��h[P��[o�={v����?�n� �s�
]-�6M �0�d:�`��R2f��un�|Z) �{W�����z�WA*Q
�����O�7���P�C��Sy���f��CO]���Bm 6Z�� ���`D���Z���{�s����(� ��n5���e��3J� %l��f# ����yD��������^:0v\�� �� �8��5�,��`����E�#&�������:�/���9Y����&��9��R&(LT���K���E��"��K �l�%��r�)_��7y�ov����~�Zw���$m<�$^�zZ��h��u�����1q���~YZ ����g������# ��h:��3 ��������!u�a=)
�"
X
�
�]GD���wyj;�����^����DtQ��[2���|�FVt��}T~���%z�+���D����i]���C��I�����6�z������� ��u���m;� ��XR�c������4s����gE��D��������\����
��
�E��7���@�M+B�h>���@v�#2D�r���e
���QL5��T�":�\�9���K��} N
A �p����i�K�Q��N0 l���D0����T�I_,�����M���
��
.��81}�A�j�>������j=S��B�]BD��%� 3�yUh`7Io� �t��T#�` @�!( �� �h�Lc� "���>�dO�EU��� �1�Z��L�w�y��� ����H�#S�GK�69vU���l����R�yQ�":K��p�}�4.���0��J��[��\���v|� �;�� ���/�i��-i��y����& q�Q'C^��a:A_+�%*Z\*�1�����1 v������#K]myt�at�$�8WEE�j��o��c�RfB�1��������f��6����^s��fr���/#^B��#�2���q����� ��)e �W����ie�JD�b�W������QM&qFD�E�=�q^�`� �D>�de&�G�N�L���i�K���!���J��0�x��c��#,��&}��,b�����e��Q4x������G��:��1��v��"�������v^������S ��#e �W�mf���'�>��'��*�u���x�q���,FjW�E�
Fd���QIK���;���:�8��U���^���&������I��������s�d� ^c�HD:sF�q0t���R��#�bg��M+��9�� �/�� ���Ge"�'�y�e/�I&�T����_p�b`C\��%��c���~������J����� u��f1���
��b�3J
��d&u��W����}���^ a�N3��zI������ �0k�����S���x3l�?�
Ns��Y;?CP ~e �W�!���^�/-"�1�?���t<(�����}nVe"��G�7��M��bB�q�9�����zE91�]��R��jb�k$�ojzH� ������^;���K�.k-�~Y�
e�4������u��M���lc�������.XE���J��v�P �w���%[����?�E ����pG��'�o>����%�;�h��[Y������ �-{�������� �����F~���!�@a��4 _"f2F5�z��{�E�'����]�A������kr ��P�����]��Y�:sF�~��:��n����OTN���'�)�a6-� E ����zV�c��m�����G�����w����:���) �v��u^��\�fb��:Y2m��K wN���#�d������cL��f X���`o��S�8��Z����A�}VHHS��m��� �8����!��t��<���v_�P2�'�����H ��6x�u}r7Y��D�e���-w-���Ca�e}06�&)u� P�;6��J��E��:��a ��:�O�� _�������;z�������� �� ~�d��n�B� "��
�m��1�tE+o���>%@������\�'O��Q�Q/�$"CDO������SL��C�u�>��oa�x��/o/������[_�{y�8��6t���C�>��H�L9������Y�j�����<2��t�V�s���� ��fl_[��O3G�16 ���e������ GFF�v�^{ ~����)��-q���f��2�<���zK�j�����XE�U������4a���W���g�i�w�x��Ss�}q���������t��|^���������L���y�^4� ��&������#�Hg��M"�� �V�^ �?�y�.9I��� z����%oe�w����0`^b�
���X Z�����b ����!C�o? p6q���HU����I�H:��Sn�f����%n[��G
���z�>���>����)�ru��Q� 'I�����C��O���jS��S����.����i�)��5����9�4��9��V1j����qQDT�iR�WN��v�e�HD�d�zt�9���O�b�W������(( �\��2���^��hl����t��f ��.&&p���H��&7g�7]�[:�ky�A.�I��=�Q������� �(WkJ��~�sr}$��1�|u��&��J�f�<�h��p�s����g���s[�_4]�����>�[�A_m\4����8���&��D����������8����� IDATX�j��H�YLt�|��sV�R+�e��y��s�y��0��eHj�oJ '������s�O�����������I������ ���[�u�7�`j����8b�=o�ET�(����Qe:�BD�D����V���2�x�~���� �H$R�
I?7�*���p�}�4.�����R��I\F40����i;���t)4\�hL��)clE��
]-���� o�d����b���,��hc�w������� �SA�2 pjm ��1"&&f��a��2��oo�] ��!���YY��~� ��dC�0�R������n�N+����g�(��f8?7�'�����[K�y�8������s��f�!�h�d�i���%����u
���K����s����4X�Wg�O�/��1\R'��)EM�ldz�eA��3
qT!��,t���I��t��� pyC�u{��� A h��e����g�y&l��nh�� ����p��M�.9O��H��(;�����v����&F��M���j�GS��?;���K�>����^']���ghEv�D��<���;-W�sYgk����Q����\��%M0/��x��j�����\�%�!��$KD��a��j >�=��&
q���K�4������*�QM� ��C�n�� z����1�� � ]V>y�Q�d�`�#���Q1ImZ���1�9��x�+�L�� ����Dy��@yE��y^�V"J��/\w�5'�=&7�����r��x^_�"k��e���N�����C��F�n ���MD�~]Z����,�?N�7 ��u���m�� r:���DEE�� � �]9������m���uK��gC�Rs&�����J�m����
_�C���ML���H���P��5�(a�{��ki�t�����������%��44�pRQ���J4F���j9�� �B�y������ ~����������6n�x&� gE�9:lD��fI���O��|8k�K
w�%�?<7�U����$���~R[�')N����6�1)�����]�D�V��U��2�������R���c}��%�g���$[�����?���4������X�Y���:e �u��)�c���������111Dd��&M�4{����� 5 ��lN�aS�Y�&Y�4�����p��M+��E��n����S0�$m��
��
�{j�!������h�m�v�o%��6i����D\Q�r+/��1 �ba����� [{��7t����Z�4Z� ����)3g���^{M��Qtt������
/� �Q}��������Q��"nzkP�-�asN�B����/����{��������_��&g�8 �B�4���.r��6�K ��b�F<�� ����DogH������ Ox��8&��=�bPP+�� ~��e��|��������D���� ���mfe������G�V��o�h1�$4}���o���Wi������f $���C�,/����� ����y��}����i����H>�����2]����H��U���A��Uek�� v����p�E�7O:�� m� � �9�yT ) �0G tT�������g|����v����+GS%] 75B�!�[2_ [0�T�*oHY�id��h��&��*�1� �0����������3#���[t1���UQ�)����|�h������4s��>��e�n�� m���L�n���[7t�����_~��k��� � ��`� ��t)�r�����,����F�����������dg�k�0�}�#"��=[+�%*B�����hi��`��@� ����W �/8�W���QE��_�#7m�Nhh�P�U��D��yw��'d��^[� �I�j���7����z����v��n��������2 ���s�t���:��nR�r�R�W����83Xw&��
3.�w��H�y�<��%RRE���(�KDn�^��Q���X�������m\��<4"CD~�����D��M���/\����R�?z����VEd :��e�V��5kl6��)S�L�R[[�z�j��z�6 ����Q�-J�v� �v�_�(5w\cA�m3+.5c������}��}u����Y�%��{�E��Cl��O<�I��6�n����'�����9r{�?A�b�?n��>�:���lxV#������i�n� w c�w����w{���{>N �o����=tt��S "z��;����[g��6��LqC�r���� 3����5�@M���H���Q���C����:��n���� ���K���z������OY�fI������ �,�e���� >}������c�Y�� �\�1�w�=u4 �_ ���N����$����]�?��z�k�q0tB�;��N
��=���I���I��~���SBK�lw~f������O?������{�xC�'�1l�~S����.Q]��������W~tx����E6�kA�2 ������ �4�����������c�� ����k��]�J�$Rn��<��u>*e�3������C�N-�]���u�z�:>��L�����G��'"�R���Eu7����$"����x�+�L��x��C+�h�U^f,��"��q�b;����uM��I�O�E|���vY ������ Zbo���� ��/w|��E���8tPv7������e���������
1�Q��]��/���� sHD���>)>�K�4�r��vy���m;E�i"*������R��&%v��U �2� s�������rU���w��n�|�Z�6R�l �}�*(�(� ��3� ��������3Ck��z�O�f���S�������
�� &�Rv��|�����$�K��A�T����L��jpM�|��<���nx������\*E'�:� ��&"�������(h�5.�{OmAZ����8�X�4S&�$��� '���L~~�W\��s�=�������gO{�
:�w�r���f�L��" �j���D��@p�)�v 3�HLI���iQf2���(��=�{s�G�&h�Z�'�q�Q/i�������e���o�?����j��2c�Z�ee �iUPf�����w m��y��Ea�n����� �����N�89���\� ���������D�����/�^����{�D����Ed4��D��U+�w�hN�N���PC��1�2lf�Z|�V D)���
���Qh����� gX��2��O� X����??����o 8����IrD���_pPg��|`m0(�da���[��<�(ra��Y F����p�+�,&������s���> 2N�/�KMJ��=�UW��[��*���}�0
I5�
6 ���V�����;C� �(���:���~E��XiE�4Sz���W�b���fy���'��c���J�~u��G�
'y-f2�
�**E�����>��HD����gmf�@�+�-�� �[u��l����3�� ���e���?5 �5kV;m :���������x��m[xG�.���Z9.,�r���IpQM�s�����%F�a:��E/�xqk��������-6
��91Fam)�s!������R��b-"�0I{���j��i��y}����}�> �3� Zb����� �6Y_�
-�r&��x��k���Hv��]��*�LLo �xU�����.S%���OLyY��J�2����DF ]v�F$/��jmrl�T�o��
���6�a��6lN�I���t{�] �_[;e �W�����x��*����T��>q��e��;���n~��9$4��!�IW�� ��L] -�����
sW�7tG��b�N -�+��?$���i��T�����z�_4V?�t��dny��h�����e�� &���; 8���)--��������������z������ ;� m Z���+�����
_q��.A����o�}�z�_����i�lj�47 N����~��`����$�2-��0���\�����b�*�k �7~��S��-����j�X(�;d���@�y/��A
'��� b&c���� g�p�)!/^<m�������y���^{�'�|������� @�j=�m�j���2���������H.,�J��\Q�C���d��x��[M�g�����?��M��/ <��7�u�@ lM�u����/����4>�*k��U]�w�}��� bPvTTf Y$Aeyt �}A@pE�q|u Fd�E�w ��%�N [g�N�]]u�:��IHc�����_��[�n� BN�=��C�*������$���_e*[��K�c�8��9L�zV����� F�D��Q%����t�t��E!��m��)��������d2���+��~��W_=t�� &����B!�~=�:6�~����"i��$dx�<vZ������*������{Q�k�u��� �^+��5fD�%
CX(�� O����L|���������'���M�����1����?z�w�z,nP= `|�|@�B��#�L��e2� `��=��� �V��h.��B!�����i��b�3,�c�3� ���)+�U~+�x��T���u}��{�^��������8���@����J�RQk�����1@ �E.� _�'}D��� ~K��U�)�%\�k��:���z{u>�����!�B(�e�V�5--Mz������B!�P�2���PW����2*:��O�=;t���!�� �'|��9�# ��:���Y�`���K��J ��oI�e� �X����[� �A'i5�!�3>Jx�����B�_�[\�����_!��C ��Rvv����SSSSR���&�brr��B!��/Z� �];=��T�#l��4���mg�
���R�jJ�GT�l���~����DR��j*D�������=�ev=�\����r���M�����+^X��� ,�����}����\B8�q���~�'���������!�B �eV�\9o�<���j�* (//��.��� �7�B����#2+���������~����!��������V|a�������� J ���3��Oz�I:
���JT�����UuvO���k
�eI���U��X���� �VF�.��J����'�4�6r�������i��J���F��e��2�j� ���P{���B!$!��s�j�������t:���^��{�j�E�B!��6� ��gq�Q
����
��X����� }\�[n�J�]�;9���V����b{�|�P�g3]B�$�/���:^/;��[�����!�{ $���[���e�5WG��X����
������c���q5��n���� ��KA�g���@+q-K��7���-�D�B>�aSLIo_�� �B���5������h :t�� �A!��)�e^��@D ��{]��@kI ��-�hku�r�r��Ll��� ���s�V�+ t�^��lx_��7�i�a�N���
�����Q�n�����*� ;&�@��~��s��5��t]�Wu���_�V�.��h��dVg�|��� �G�[O:o������&:m$6�D!�����%�B��~��}�Z^1����
6�'}8+�.����t�78
�
V�j�]���O�G��q�� � ��������*��'-e}
�x{J ����� w�>��Z9
Q�a G�6g���e�i�;��������:�1����j���~���3 +�1���v%�5=��������
��V�����p�1zJ�
�����
�B)��)SQQq�����& ��� |����7_��!�BHI�j)�p�q����j#o��4��~����7:�x�y�m�|������Qj�7�]�q�e����0R�
�8 P�F�������:[S i`�J�r}��i��( ���G�8z�<���KYMP1�����3ko}�C`�
����mcqax�����X�HL�� kJ����)qC���B!�:ExA�5k���?���S�����_&L������]��m!�Br��( x���d?�C�p�1G^�s���<�,�9�����s�i���B�� �����=�Ro ��
�S���o�R��`�]���x��D � ��!*c`T������ �0"��}��\p�x�]c�>��Ps S �xg��wo9�����q��E�>�e����l��m�iY2=����!�B������t�M�6m2�L����^��3g����0a�_|q��������B�����Z���p��mN�7-�A*��h�8�����
�,!'?*0�h�m��
��������^�j��'_� �%�GV
�2����o.@ dc�����=����~�Z�+���������
*PC�NwM��P#
���Vt�y����o�� ����p7m���Qz���?:v��
���z�C_e���} !�B����}x5e\.��d�={����� ��j5���C!�.�5Y�o���V�e�R ��d{
[P�q�[s�� ����@HN^s����H�t��goH��.K����pA��e����}��~g�n��%" NW����H���zE�CT�X���1U �R@o�,\�kV��`��FUZ"�3������b�����pwM�A��T� L�����i��4}�wb���@ B!�����2�����VkZZ��:##��7�BuIRM��T�(�e���R �Z)"���=�������\����E�����)J�|���� ��T,nJ�G,4~���'\;nP__���N� �� ������OQ%'����� 8����)��@���r���% �����T:6������7�Fp��3#�h�U�E=����g )Ed� ���F� ����t��z�jA!��E,��2����w�6��)))��(����`c!�P�Z��qx��-���
��m�\���Q�<"u�.u4��P�h��w�S������1(�� PF RF# �����[*�$_� .
�a<�ApM_ �Q���X��d{g�DdBT�*)"��nco*�d�5��^�y:~[KKl=�X��F<����-��.�/�Z����}O�'��!�B�� /Sf������s���V����� x���.�0{C!����\9�*;��B�V+:(��1���
:�$�K�����N�7�p�r6G�� ��`�v����_NZ�hB�/1\=�b ��V\�)���fZ��N��:2~��|��
WS����A1t�1P������4���� :u�I%C )�<�6O �n=� ���2!�����)����c��/��2++ ��� x��n����
!��ZT�t��!����F�D��>���9
^��2�� W�@�y��#� p =��� g� ����pm���tk��+�y�����{���� �r��T��G���x���VN��uv% �_�b_��G��c���^��a���Z�U����6� �B�����2
F�q����^zig�!��������2� ��K�o)N7F u �J������t�t��
����W����,~���F��z�G���3�9�D��-� �H���xU� C�]�g�� ��S��:���/� &��D�g���v^y��z<j���%Th��PA�����n9��$8��} M<���z����c���dx�5�� 45B�����x5u��L���j5!�B�k]�#TW�5�f!��MtV��_b�����[�����C�, ����;
��'|�D�jZ�^�LsH�:i��I�����w~�������'�7UF@wm���E��m��9��B�_Q�;v |�P�q��� W��V<z�V��*������S�~K�A�R�N�T��_o �`��JStO��k�� ��jS�����k����'B�6_��d^�<��B���5�;S����'N������{������!���J�� ?�$i�/~J�o����1 P�q���mcq��v9��"��8������^�����WW_7e���� ��)��}�LMi���{�q����Z"2 @����^�����w�����/P����� ����Q~K�{�^��_���
Dd �n��F����������/- ��t�1�6�k����x=�bH���3 #2!���W�����{����~��[o�U������>���`o!��E������� �=�� IDAT'�[t����\ p ^�
ny�J���O���G�e"q8��'7=���P��n������'}��u�Z���u�k� �;Qm)�[�������M�����T_�H��K�/ccS D�4w\NfTN~����?���#J���9�� �3:� 0B!��E&�L�g�yf����������������w�B]���M�}���R*�~G�m�|w��p�C]z ��&+���p,0^��J/��rUr��7�i�a�NsE��?92�^ny�� �N�R���TM�?2���a���K35���W� 7�q�x��|�x&��`@L�����L Q<����$�}lx���Sfd�k�
��2�<y� �B��UxA�����, !�xB�- �B�U�� % @y�����Vp�7�����h���q�4}sC�O�23+�z0L,���M�[~i){��*]�ri���|tFDKO�P �23TT�bC�WX6�� [�����%,���q���w05��e[�/iY23�4�$�._�Y%9���B��XxA��B Q��D!�.VE��2�^W��.��h�5���Ne!6�����m��&���_L�
�Z�;N4\B�mV`1��=����Txs �s�$tB4�5 X��������.]gd�v� V�A!�� YxA������O�G����@�_�B��V���k~���'��^��5��hc+�]����+�D_�� �yCW
���k���i���� ]V !���I��������� �WY ��3��*9I>`cR��� ������~?P�XnT��������::vT
FdB!��^P�������;��� |���O?�x�� �5�B��9��:�� �N����v��2\B8C��a� �uTI�����c��� ����g��O���#O�Q"z{u��� tQ�����xs ��v�$D��3�h19�=�=����7�T�F��� ���t�����;�;�!�B�b^P&;;�_���c�=v����' >|��?4 ��!���s����2�H�tYy���P{@�����1\ �z��>1��K�4y"��G���:/���O�[(��~���R�=4� ����Q�I�����F x��w��8�_ln��O��_�� DQ��� O��< �#�����j�eX���JB!��8J��k �%�B ���l�|���,��S�s��b����u�,o[���7'ir3�>(�aSMI����w��ox��F���V���rp��q����Z7b��)����o>1%I5H��\��G��3���9q5����,�����y2�w�t7�J�F=��#�!r�*]V^G��B��t�_�����Z�ee��#D!�.��$� -?:Uf����[�
L� iK��/����Bm;�l����:��~��eD&�@?�hB[Y����Y��y?Z�w�n�1!{Hc��*���MZ�����xU7���}�Q����\������<�<zQ!�B�����=s��e�����)D!�� r�"��� j��!��,!gF{�t���>�C��'p/��[:v�.+�/�J�I>H��vnYX����=�Ue`�����i��u��� _� �a�&k��]�� ��Yw���� ��55�O�rF���q���#��i���hK)��\f���n/*�B����2e�^����O<�D��!���r��f�����7�$''?���9@�>B�-��<0e4�z�:���+���6rv�3� ��� ��m��� �����7�;���������k]�_��������5�Wi�k�t\F�2��e�\���K�ET�S>M� �!����B!tQ /S&--����}��������%�B��w���� �������������2����s�1zJ=��T�#l��4��]:M���w�����#w�<}AQ#�R��q������H� BD��Qd�4���~�~�~hi�tyr�-�^gP���7m.�u�:R�����I�@z��O����x<�3.��W�B!��^Pf���3f�x���RRR.��B�?��u��v�����WT����s�vkN?���m������oY�(��z����O��1%*��~K�� ��Ok�d�O0�C�4=w^�����{����y��c��J[�1��8F������!F���|�+h#os5��`�� T:"������w�B!��^P����c�� 7n��O?M��4WWW_��!�B3c:w���+�e��dq���w� ��9WPfW��w����~)��WY�RE�%p����M#� {E��Fte��Bpq�
�v��|�_�[��4���-���DE� ��4����{��>���n��,!�B���2O>���)S���w���������]!�B��K� �?�o;tI6�%��������uU�>���L�� z���H�.�~KS��?�X�v������h�!��� v �w���|�[U *i��0b�6��?Zmh����> ��sT��*69I�_�tq���z$q, L�6����( Ds
�{����,��mP�w$�!�B��^P&::��{�Q ~����� �B�K 7W�-������ ����3��?��W�~G�m�|�D����E��H������G�.Br��M��6X�/���� ���3Y8�$��B�������8�9�:����`�Zk�/6ZU �� ||uQ��GG? ?���� 02UK l>�l��6l�������_�����; ������Q�@pU��B!���J[�v���������� ~8a��N�U�BHx�B���s���?�p5��#�
������r5��!:}���n�y,^U>MO��#��v��^�L=�o>����b<�|D��6f��S=�d�|�h�m��
�S\��{��M�����VV �_Iz\)D����x��H7 ��=�b����RQ�&��h�������~��4���H+=�B�+�y�m�\�;��s����BCQ!�B��k�j ^K��� \��BL������V�h�E���8������\�n����[_E#���~WY�-��W�x� �(SA.��;��a }c������ B:^K�K��[�
���
�W ����sY��K2�6����8y���� ~S+���Y�d�� �y�4�8�0� �; ���,B!���*����b�|���_}���j��
uM������{A!�kXX���oIKq��E�>�B�bj��Tx���d�!-�m�mTh9wL�T I ��1MY!�Q�����;G�j8��
B�-���$������3oY7<����K� � �(n�� �����XG>���y�7�E���V# �ro ��\1��PB!��y[�xq����z/� /(C)]�x���sKKK������'����hg]
= �2!�Q��)F�\V'�P �\�uE��^�QT�!��Ul������[*���DzM8�q�R����k��o�+{�|cAu���r���(R��?z\�;�g�k��&*7�sY3V�U-��l���qo.��G����Ts����K ��<��� �?��V���~������� *V�b���U\��@���>]�rk�(�B���x�������^Z^Pf��� ����������>��_ ?~|��Mfo!��o&]�li���|�8�)���t
���Kw>i�����O�6��/�� ����s��3s�}�e�}���Cj�T��yS]�
]� ����s�-)���'����m�={]_~����o� m����X|��DcDp
�Dl�&���c'k����� 5�E���7���X��9���KS��j�7U#����S��[W�B��s3�>���r�ij�A�6����)��0��Z��1\B���L�PB!�� Dxun����O>����-�y~�� w��Y
!�����p��m�LN�z���{6�9�fAvU����r�* dq���i>�D|�e�=������g~R��A# OGN� ���*5'/������n� �����G�� ���O�)S�����OLh.[�JN
�^ ��]_>�@���d��� R����>���I�l$�#�c�ko����U�{Y��8���������W
����+
���H������� ��Z/�U���OKQ$ PEd�O>�} !�B�����}x-���Gd ��8�B!t�������fd^�d�g��9� �gL.���J�����j��M ���c-�� u]s���.����LP{w�V��Y�Kz�#��
�
�q�7Q
����Is������ ��!/��a<��2�<���|���*Z�:��esO���xK`r�����W��N���� �N��p��N (3*E;*%� 3 `��x���<y��@F�.3 1!��# ���q.�K>b��A��k�v��B��Z~fn��'|�=_2����\ ����+L��H����2�-B�2��-��> ��i�zG���P���<7d�i}���L�@���t� ����#�Z��i�KXs�������x��$���7�z���x8F<�&�pS|�5/��3$jO`�
���� �u�%��[�j���.+/r�*c�p*c��aoH��B!�Px�2��_�����^{M�V��n�3g��w�-]}��wg����{D!��������Hi&��>��� $������� @�^� [�xiO��(��Y �c�
%p�4Gj�T�N�6��':�N5y����+��� s��)�7�WN;�0�#���zZaK�q�"� � R���u�=�s�����7K1��&��yG�+��On��E����+�hzP�O��g5I����<]��I����d��A!�jUxA������� 0`��q�\.�� ;���������� N�<ya6�Bu
���L�������� 1��4�C��;^ig������ 8� � ����!�o %�� ��P�� w~]J���9o-��f&�H��jx!F��)V];3�M�!�d5b ��-�g�� ������%��2T�f�x�� �S��*9�DX6:#�V��\z/�j ���lr�Pia�b�FS��.-J�f� �^�;��|�!�BX���:t���+������u���kVB!�������B0 �� �� n]z��3��`��*
�m^-��j�SC�%��+{?I��'Q�:��Kz��
�H��}��Xw����9Dt����YD��^�Ts�2��Y�3[zr���z1 ����r�� ��� M��/�������i�c{���B!������}x�2w�}���C�g����� �u�]��/�B����mX��=pPHJ3���j�_�
u�\��a�U�x��\(�4���K�
�1Z]�eI`cZ�L�6 @���+�h�N��-���G�����g��&�Gt!����J4��~�R?��c��P ��1 �S��2!�B�K�����R��" y��G����u�����NC!�YBc �L��|~�v�*�_
4{n��2K*����)�����������z=���P�D�5`��zxP��D�J��aS>>6�_�+Q��JN�w���� 9r�u��\���W��=3!�gJY���z��1���I�'�PQ$~�Au�k�� ��������F%�\ ���� �J�� !�B�W�5�oO ,��� L��L�`����X�5��!��R������N�j���s���5�
����z��]���?��s�������_�WH5e����2 D��~��]#��������'��5���������+�CV%%���
_�C��p��P������|����&m>�|�L2K�� |����<o�&�& �>���8�/������� h�"�@���1�~|B[ ����J�^�e���&��A!����k�j ���/��b������i�B��ha���� 8y��C��e�m���_�w� `/���_ �����e���������s����3����c_K�:��q�+x���k�iv " �O�W|#� �Z������Dd @����/��Z�+'�:b���I�JZ���:����� �����;3b��(�����_9��u8���!��c�m�O:����t���\�B]�� �dff~����_~�|p���3g���]!�B��r�M1R������RDF�w�U+�2�F�g�y��p�|���D�&��i6� +]-�~���d 8����K�#��-�A�%�$�8���E�b0�u ������KM�'�l��2���?uJQYF%T4��4A�cab���+� �� R�JoI ����5mQ�������Tx�����u k�� ��A!��E ���+��2`���n�I��w���A�B�S���S����i�H p�+�8�Z?���6o�������>��e)��oqYW� �tc:��#�3&+�'��'MH$��#/�w��T�^>\�D�Uz�kv�L{S�x tU��"�*� ���]���Z&������g��kgv{r�l�m����W�k��y+#��[�����4�g����\�B�.(������_|������O0=��C��+�B�W�������&�)�4w �9�)�|�1�k���� )��2Vz�n��-N7F���o����[�o���Z�`�=�b��z�'��o5>���!-EtZe5h ��3�5s�j���A �J�.P������M�zo��Y���F����� ��q��V���wNL������a<��2 ��H�����U��S��:gfM�_�k�B!�5��ijj�����2!�.yY��������� `���]�*���4�# � e @h?i*�Bp���M���^�v�� ��x�@��q��m������7L���S�:ZVa( a� ^A���6��}0�G�3����4*��_�x�g5y �kPl#Z��o>��+�� �_�6�_��~�������[_ym�,��HX1���ZJ@^����P�<�!�B�kb��}�-��X�B18g����B!�k���-������K�>����9��S5�S��7Rrf�r��w��W�7U^o���];�
L @= �L���1��v�y���^Z��u�?g �r]�����9�k�J3�b�#S�EN��*����b�������y�a8 ��;<�'�N
���{%
�) �^��R�Q��#u��DT�W*���-MB�b�W��L�6h��?�G�
B!�P�^G� =zt���d2��e <XSSs��Ut��Y!��r�V���w�j|���5������� ��O#�c�>�'��.������q=�w��j�bS���Ej�|��G o�/r4��eE&�1�0��������?������?;y��}����iI ��S03mM��S` :��j�}����$�">~�y(������f���n����%���Ft��K�����K!�������� _����/^�|���:m;!��/��J�R�$Y[ke7��xlk["2�\8��0���heA7zB��'�Q[\�����l������k�W����W �'����n5W����L�ZS6aiI��>e����G���z: ��;���H �Ibb�\��b�z ��"'EX�~�W ��/��}���� ���Y~�
�Y ���F�o3�N��}�:l������Lb��I��]20��~E�IezG�h1�B���y� � �������5�i!�Z��g��8�@odO���9m�w�7����|s|�p����tYy yVS�x
A�"���yZ^9X�U�xk���3 }�G�s�M$v 5�'{��*9I�V#
��c�G*/��NKjX�U�������,w �^��}/�~�����hG g�A1�:o����nM[ �y�y�~����>�=������>��J�uFG�v���r��}��C�RX_ �����F8��o��7B!����5�/S �����SOEDD0 ����p��#F��6�B�Wn+k�YR�-�����:q l<*�G����#jt7m�m�� �8��%,q@:�Qh9�ef�[51,sg�}_��������%�F��]�$*�{�I}��r5�3H��c���V�B}=�SU��~����?O��zI^�_��&\]��F E�H���� R,���h��@������z��� ��"�B�-��Laa�3�<�q����h �Z��f�Z�xqn�2k!��M���VG1�2��*1�w�+V���:�,m�m����2��� 0-��OW �B��L��}=J �Nz����u"��+Zk��j5��������|Gs���$�{�!�� ����ni���RS�z����K��-��_����Kb���� f�� IDAT��������O)�����l � \B����^G�U�{=������'��-�B!�.V�e�,Y���oJ ���Z�f�����o�~��B�M��������}�F�QT�V��>�����qr���qVO]�83��kx�W.�J)RcHo�|�;����-���U�6�� P���u�K����=�]�G���� 5��������T�}�4���&���V����� �.����������� �LJ�P"�je�� �
)
��[b#�B]�� �x�����C� >����!���,'/(
t_���� ��w�U��2�����
9,�Ww��{Ro����6�v���w>)�U���l��U��h�{��{O�Jb����zfI��S�~Ks�$�\"�)C�7� � �
M� B���JA���4Q���R�e{n9�QQm����m��iI
���5|���� /w�� ,��~�O�2BK���
��U���"�B�*��2!�P�1����k�PG����^W�u����������j��&�SS�iH�y�_+ ���c:��:��g#>��N1-M����!}�z�� �i����� �U���x�\hB��1���H�)�� c��z�kE�%tr��h�;V���+� ���bF
N=���?X ��|���{����_���6K
� !V�H�G��
5���-�.�B!�:.��L�^�v��9v����W_}������B!������[�xf����� 0�q���t�l��e�n �DfMhG�E�C
Ux8 �.�� >",�efHA �[��%N��M�6# ���P.��._?�� ������S1��BYb��3�� 9�0���RDF����� ��.3��3��^/B!�P�b�=Ef��e�=���u�l6��f{����y���K�^��!�B����N�t�K ������4Q>��D��igB;������q� ; �Z�N�y�8 R�!a�q$��f9_[����Y�k~��yc������.� *������[�n�c���3!q �j���)�����-��� ��"�B�r�e��L��?��_����9s `��� }��F��0{C!��F��!c���C��}iF����M��v(�
��t��W����� �*Y�;�� ��" ���e�����|�&�[$���$��&B���f�����y.���u�k�I��Z�Q�X��$F�=�+�7n2 ��w���m��aN�M���
1\����D�8b�m��(\��U�L�������d���B!�)����D ����B�v�,1��C���H%f����~l[����
�@@g���ghPX��)nq�u �����4#���ef��\��}�|����:�p�2�5����,�O���i�a�!�ti�/�5}.0��8����&�F>_����[Fzn{��^ ��}W$]���U��O�1&�:(@S�M
���B6�C���U��}s)�\��p����tYy���!�B�&�����hO�(666���v������!�P�x��A-�� ���w�>�~N7�B<Z��1����i2 {��3.V�j�]�}����O/��K�x�|�T^W��y�M#R����K���rT��@P�/���(��f���ko,��7~�� ���� ��KL�n ��D8�����g'�t�z������E�+� �&��!��H�������b����3������B!��Uu�_���)�|����{����[��=i����V:D �B� N7F u �J"D���1�Dd ��=��~���k��:���z{u>��)����
���_ _������Qj�7��� ��}�[�qGsuS� ��|� ��T- ,���>}T��� BT���cGG�8����)6Bza���P ������<B
Z"2 Py��xe�; (K�(KB#2 ��x �;��~W�9�#�B!�1�e�F��y�^~�e���`X�`��+.��B������7D�|fJ�@e����� $�v�J��������_n0&lZ��}��^G�V�}�����/��3��):m�^*��!4V]{{�:~����^�Qt@ ��$+�G�icz��)�1V^�U@��U�4��%�]g�� � ��9I����c��� �����mP��,����e ����?!�B����2 |�����7BZ�/\{�� ��]!�B��z��$�# �h��]W����-� '� �P1$ �{ P.3C��Z�rn�P����7����6~&��-n� �� ��0�a��q��t�[:�%M� ��A���r�>��1�
��kT�I\��D���������D T�FR��_��]5C�8p �Z�,�>E��4"�B!*��K�V��IiD��� !��(�l�zeB!Tx��Z���\(�M�G,4>?3W1!����6x�A�$��!VOz����I�OLCJsa`��H8��s�*�j��!���$��Q7% d L(�`=����+W�bj�I���.s��������\wqF���:�SD�Tz�_$�XP�7��c+ ���.+u.\Rq���H�4")hE��u��� T�!�B��vP�R*��A!���&-���o4���N 4��RG��������� a�ipk�J';�=
I�� ���JJ < U��b^�nK����oU���V���X��j���}�8�v���F�h�����#�����:��?��������]V��{�y���
�;
����f�^Q�A�4 ��dzvs��.3_�����2�iD �[�A!�jGx��F� �y�f���m�rs�HD!��� Sf7Y�{������go���H� ��\X����8y��C�+�h#o�kyOtk*s�������� xk �;�[���i�S�-��D� �
O
;u�j���8���! �JN��e�O�@��������:`F�1�������
� Vn)�1�4b��1�jxe��*���������H*=�7O�8�04^�*w\hj�Z!�B��&�L�� ~x��9� ?~<�0�}���c�6n�x�6�B��)$����c ��$b�P��0�H�j?�r�M1R��*F8�=� �Z*�6Nk�=,-f��-R�ieq_���F������
��=s�O� O ��Gc���7 �23 l���5� ���Y����O x5�m��{
nhk�~��N����)3�� T��s�?�?����
�����fd^��=+Q-����Ts>� �B�a�O���{������� <��k���O3u�f�!�Z��|h�����
���z.t<{�s��A� Y��SS ?�7� �?�A��i&+
Iq��R�~}� ���W��/���a��nS1|�5�6�L����5���oR������<���3?��E �!Q{&�l��l|LZ� N?6���.y
t|���1�>�$W��K� ���=B�5���a-�B!t�t�_����� >|������B��S�B
���lu|�e����-p��S/� ��'NKj��5��PG�Q��0 @tZ�oo�"@�2e$�3!q���
|l�&�u7!�G1?�k����K����
��vV���� �#XG@��}%#����l�� ����(�������vWz6�p6z�h-3��aTJ�g�|�O�\Q����V�j�B!����)�Buq����� �Hs �����) 9�#� �J��b�����,0m���� /r��1\= |X=�/��9]��n1N1���!���+���s����sj�A�������J �R�xKy�����
��#�Z��i�KXs���B
P�����@� ��ZeHkgK!�B�2!�.*��(�H����P����#W �����+�NKj����
�j9�z��^$g��{~J�U?,��F> (���Z ����^�%Q%�) XF]n�og�#S�s�����!����;���]e�O:=B�F<�z� m0\B��@� ��K��B!�:��K!�P�z
i��?w<�E��P��e]�'������b]{����L�/��7��# � �)%R����[8YM�=��v��h�RE+ �~��������n!����W��%��_SBj��Wo���NU�t��T��!�!d�V�"�e�������lU�mMF!�B�A�B��3�ij�l:|J����A��������}M�F��B� �[�e6�t�����K�(���s�<����W��)zc��z������(n�@6 �N�^Z�[bul�"L���x�y��j�}l� ��>������L -
��vj��SC���S�~K���{��M;O��e��[ !�B�� �2!�~�B3\�%��?���^P�Y���#�hNy^�
�����M��"�u:�i�c ?��b����{0�D4gm��V� S�>�I�l$�������v
����f2u{�Tq�z�S�
��������1 5OI�\���/������Py��R�H� �T������ �e��������Sd:^!�BeB��ylk���P���?0���� �=P6���G����B��=�0D*?R$e�8����W��e����t��&:����: ��a�}�O/yi��<���{��|I �'K~�7�G^���:���D�� ��!�X ;��aDHDcR��D��p�4L (�� ��{]�K����ZJ@ g }J@ � RU`y�!�B��� B��7*(O�$j����F(�e�;.(���y��F�n��
�B�x����^��B��E_��G�����������[,1 0�����U��� ���(���j.� �_r7 ������2 ���/�
��,��TJ+������9- �2�� �S�gW�����S�wy/�wV��T.q �mYZ�
�A�B�v`P!���a��P! ��pN����� ������r^���5�.y/�t�KYW���� � }�3���M�C��6���m��;(�jAP���h��%����� �0" t���yI%dkUt�X�x��R��T�G���'���W?����}��i�S�~:������N?}dx�{��=�C ���'�_ ����]�E>6<�Hf�4� �m���_�w� `/���_ ���E��M�U��B!�eB��9&R ��0j�6�6W��� ��<]� e.kGn�j9` ��}�������M�����X�6W�=c�= (Yx�t�� \f�������3b���Q��H4��tN����j�J�����+��@8�h�.��$ �L���*9�N�e�4����������[���>i��s�oSR�D Ycc�v /�L�O��Y� ��)BVK��) \T�VP&�
h�*0�uB!� � ����sLJ ���v��Q�8�� ��� :��f�
Q����V��� ����'����8���s2��X����}� lsF��~e�![+���� �
_?�P���X|��*W%%JG��>���V��C *�
� �i�P�&�+��X�����Oj� �Z�
wM���( d�:`. �+=gN8��J2 ���4��Jk������:�y�|G�r$@Q�����x� !�B�
�X�V�����~�v�B�"���Sy�I�O��h�m�������pG�:t `�y6vd�Q)���3�r^�,��q����OdUI�� ���ZMd�/,q:���� �K��������o����F,h'�����^8�c�����+��+Ny�/�|\?����o5������cC�2�s�S���4.t�$�����uB!��(���9s��e�0(�B�<��l���0j��������WL� ��� ��x��W�������CF�N�io���N�iu���I�uD��)��-��u�1��S7��m�\��O�(=cr��T�_�d��/�.��~������2) �i��h�M�[����O�k��������3����O��O�W�%�O^���g���������[Kw��N:�*I l��(���#*�$���2�8��7#�"*.��8����.�,�����8�
e !�u��Iz���� ���CH$���1O��[U���1���='g���R���'������z���MT�;�8 3dNr��
o��K{�B!�~��.��
>���n=!�~�2������t^Y����@Kca����]$�S��2�����mF�����W�jl�
K|�qU +
qU�#e�,2;�E:, ���{,�� ca���)�j�f���
� ���g�y4q|��]����1 �0�$���p�)�9A� �����Po�[�rg��l��/��Z��$
�` $�����w�c(��>rA��������a�������� n�eX��<cx!�B�g�k�2 ����v��o�=�KB!��4{�ul_��`���9�g�wsF�wY%S��=��[� � ����1�oIY{20!�'�f���+������v�w�����g�����I1oL��5�ot��dM�*�T�����(Uc 0D;A����4��M�&HAX������O��-=�0X��wx!�!B8S�S���<"]
�hf_�:�3������i?\��:!�B�\u-S��^�� �P]����B�Qan��azX�x��a�[0lBg��t`iYq��y�}����� ��qL�2�.*�M_;�0������W9��h�W }{;����X"��,�V�F. C�N�o�6��_ �R�> $��)���A�\�_�Rx����-� }�� P� }�wtY;w��= ��H^�����F�c?l�B�2u-S����G� ;v��y�i�A������B!EQN�9i�
�^N��-V6N��WjQ$y����/�y�SK�o�s���ss !�����H��=�#l��-aY.-M��BX�u8@��\�) !7����P�(5�������M���~�(�'#�;-Ak IH��Nj����u��I�p�%*z�{g�r ����� B!�P���L�'�|�b�<���� ?v���=��!��9��r���:������_��x���+��_�![ Ho��D�� � ����$�����<���+��^h��k���B| ��J
UW� ��"�&���b�����>�����"�DwU^<3n�E��+������� ���/�� \V�\B!����e����=��C��
6��� �B���r�bp��������W��r����������_ �{E�W?���n�oI�����TK���!�Wf��k{��8sD��J������=X2�2���4TQ<\ ����K��� }4G�It|�8�Gp��������`���e�;�X�?c�ts������3�n���g���))M�����'<G|"m���:My7!�B�8��Y�xq���i���bB���4�����cd�����oo��.���������G_>���"� �����U!���~��qK�
� !�g�X����"ug(���F��N����}�$��R <\�b� }z�3���A�$�����h^�{��c��n��4��2u�:pv��� L��X����[����kUJ������
��-i��y�E�Z ��i!�B�����%��=x�M7��� �B�5{�u� ��
|N�� a �����oLV�C y��Z�+iNd���F&k�����;�<s���!����������}��C��Y�>�x����^�KI6 3���|�������^�AO]�`�c�������~�U�\ZV��4���.��=�s� �� :���k�����#����B!����e� �B�Dan>%0c���������#R�t���M��
�J��G�`� ��;�fC��k�}p��t� >1�4|l�t�g�y@�� �X�V/F������A!����"q!C�&�oL�5��vb�L�t����G����z ����U��t3�u��:�t�{�fB!��%�Z� ���s���yyy��� ~��R_w!��������;�1^����{W3B�����T��A;���6Qwb��Y[����@���}��+���d62�d�l� Ng����EB��$�.����9 ��i#2p*�d������z��C� ���i�%����S��M������ L������3�5={�<mN�">��R�M�:9
!�B�s] �|��7O?���%K���
7�����<, !�P��+[Z�:����nu��l�\I���f2�;��T�� IDAT
�wPI���c�#eJ0qB�E���Y��8 3dN�`��I �H�����q���o^B�p��$���`k��@�|J"��/1u� �e�� 4M�%�|����zz��@���`�������k� 4�{��V3�7��=oN|���-�� #K�E\A���1w��m�� ���t���fB!��P��}��g�Y�pa\\����3��7�<��B!�M--+��{�`����?&���X��i@�)���,�07��Y��^���[��usFN�D)H,�� �a��*oK�9����J|2�S%�u_�le����D
�U��9%)��q `�
s��R�+���_�[
�&u@��$l�~���L
���K���Yszl����W�!W(�U��f pW�J l ���]�1�� ?�?����|Z����U�.P��J��k���i!�B���ZP��v���jG!���q�B?K���\F6� ��S���;�Be �('�V8����`�GYB�u1�k-��]s�� s7�����q���� ����y4 �-+��^�3-j�Vg"���� � �El>�.^���L$ XV !|���� 7�wQ����Q��f�����=b��5����5���Z�Dd���2>me,�� k�'nw
U&Wx]���H:�����B!������ ����D���B�glf�%X�
�\J2�3�
mi)uvm��3�^� @�6�Ys ;d*����0����r�w���t�[��ud�D�X�n� �����=lv|�R 1 �{�Z6� �����O�&��d�?t*[Ga>4�� ��V-q��e
s�k�y�k��d��6�o\�?xG���j ��( ql�3c���@������~��> �a�w��!�B� �kA�������_t�E���_~���A!��V�mV�����TB �$���0��:<�� y�U�c��D3��qo�� c c������|�I �� _�V�b�r�}e���OS";!�1����J�����%�����o��&<���3���O��T.T�<�35�"�L;�����e�y������� Oy&�#�D��������/CN�ts�J%"�02�_'�~�������� B!���u����O?}��� ?~ $I��������SO���!��f2������M� ,���\�E������1��g2��Dd ?��d�_Rh=�Ek�4����kv������eG�u�6 � �����e�w���?�����o�������_tA�,���t��) 8D}~�e����"2 9�6�=��;g��w��
yd����(��o�e1P��u�$=���`���9�:��B!�Pw��L�^�z��_�z���O�81~��������i��#���Q���
�F�G.p}1����; P�*R������" �����vZ�LRy�� ���oo%��� �����tNt��Nm�s�w������
L��4A`��cl���B�Z]�k����<��^U�9�����}/�+�f ���V� �&�+E��������c6-��^0G�&fI��3��O
!�Bu] � @�^��/_~>��B��+���f�^_���O�M�P�R�r �Pw� �[�kL�n��+v��y��K��
� =�N����%��2��Q('2�����4]9X�x��9%������~{� 2�0!�LJY a��j!{���~(s��W���CSJ��i�����MC�<l y% ��J���
�L��4���5�-s|�����!����� �|������n��k{0a�_�B���PJ�<�C ��M7�tNV�=r�B������M/m+��d��#o���.*�_�~)@`��,I�T 6� Ld�z�r�5"�����J���QZ��>$����`r�cJ��2���B�W��D{���y�*i������Im�a���%��>�h�`���
v-��%��S,����MT�;�8 3raz�d; ���eI� �������9u�����j�����w�~�mmK���V��q�B�Lt���]��q�\{������3g��;(�BH�7�2S�o]B�z�&�{�GDfiY������-j-�N��&���T5�QmC>Qb��p�5��bn~��Q�b6s�<�|�� �\�}a�c��a�w��m�|�����Z�Yz�'��]ol���5��wd � �b:XC��g�_��� @�74��W�?��M�?� Y����IJD L�>��[J
��u��tl��� �DW �`P!�B���ZP��/��1c���c�V�:�v����Buw�y4o }����l��m�����M���&N��Zg��d%�������(��:2:T��.���o�������w)�!9���?v�Z7'V������m��A�J�\Yv������� �]e,��M2 �'��X�F�����"5 s7@}\��i�� ����iq�e -���7%����Grf�>I7�5�B!����e�~���?�8..N;h2����B��u&ff�%"����Y{>�e|eK�"2�9��?��S������ 3N��f�Gf!�� �L���J:������T�Y�p$��:O�+���o%�SbS�� ���F/�+�|:i���� ^�� >s����G���k��9 � Q�Sd*~������(���� @�����)1�����T�����X�'"
�0��B!�B�P��2�A��i����� ��`�z�P�t2����%���:����r�w����_������vK!������Sn��n^Ql^Q�o�l���[SH�)�W C���o���Y�� �?>9����F�mn@>v���U�g��
l�{ �w4[��)H,�,�yR�!mRCrp�{K�������Vy8�l��� �������*�~�Z[w����� ����/���Ysz�������� h��u�b&�7����F���, #k�L�e9�K!�B�Nb�<E�h4z�^����kw2B��%Vr��$��+Yv��|�����i��� �=��F��>k~��y� [D���w� *z�{g�/}eK�Vg �/N�������?T'�D�%o��^7�g������������L�(�L|����'.-..��a��f[ZV��j�}������4���l���+�n�vi�����LA�����?��V7x�����u??"Mxmd���_O��)f�n��S�8�<eJ0qB�l, �Z ��7�}�N*H�`K4� C��}l��� B!��y���� �={��E��5e&L��z�������{�hF�s���g��6D���Lx���Hx����@��r�)�����������hM��?��;uB�;Hm��5�����$����M��WZ_��R�f� ���2
��H��������&C�]�cM���]LP�_���_�kLL���=�o[���,������=��?�}�6��y� \�
��?��"��p���s���-U�S��#CL����B!���=��wj�����������k���n�v�{����4�B?�Jo��pP������{���Q�9���^;� ��_���\��@8(��3C���S8K�r�K�1R����6��=���Y{
������ �� |@ �Pwg�b�Mk����; ����N��� ����j�`}] ������O#�kp��
��w�Z����k�>�CS���zE���
;�� a�T9���|�B!��t*(����n����{�m���� ��0�1 "v��gb4�����Mj�u0�d�k����*���_97)�(��7�=�:]>��������~/���He 8[[�� ���eW����<�������{����m
��$�e�5`�i^��dd��ZV}����'��� ��F�r�Bik}��yJ� ��� z ��������p.` �Z�3_�����R��&P��x���J�<�_����f"" d���.7�5u�$������+'&�~O�� �B�n�SA���_ �������� ��`f$����:,��z�8���&�,o�4�:����Xp��j $�����h�������p�� !���r����R������@��Z���������&�5g���T6JP,=Z_g�"v�����!@) P@,=��l|��]$Ed� �%� ���R���ej�����]�|����� �� u�����~���g �[�� �Mw~�����mEX��LO a�� bx��o~�����i��R�*���d����!�B����[������ !��-���� Q��z�����;���%�}2�����6�o\�?����G���z6��R�MO�\[uy1d>���k��Q7��R8{�D�s�����|�!�gB��YI�d������0�[[����r�>��c ����,���j/�^f�1��KQ���\"GT� ��N�>��QS��K��D���L��o2� �c��B�s�{~��ZK�h7n���OFF�9Y
B� �Du�&��t�7� ~�h��=�!����m�R��eP��?c�u����Cp��7&������c�����XP�uy�J��� a�>��,���e�9���gd�%�c��>������q��2'YQ�{����[ �;v�O[�7*U�]��89"�����d"�g��`|��� �~����l��� ��uS����i� !�B���k-������F�F�K/�t���B�� � !������=JDF0����{n�c��+������5�oxlC��q� >zZ�Y_�&�lg�~;Kd�4A�y��'� �M��~����Af�5�;�:�~��J�lu��pp'Yt�y�5��)� �
u,����G�� �E�$�u���1��d���M�pxH�k bY�v< �N��J���n��w��B!��OV��2/���n���������!���aJx@��M��?���8�~�� ����N�,�?R����������R6�1��'��2��������| �9s��/���Ze��4u���W�=�</�(V�nl�-j� �R9�}��!�1�Jq�1<H������ 8k���L�
� ��[��CA`���%�*��Q� d�����C�' �j���rBm2e�������q �%���9~�!�B������!�@���Buo�i�9��Uc�lq������v_���B;P��ir;�;��
^�{�MIo�d�$I5N65U�2�lPRRo�|���>����?OH��'���n��DZ��� C5��}^q�Sz�)�hK��w�~�m����{/��V �4��8�����^rB�Z�a8��n��k�(�s�I12���� !�B� ����,�UUUg��B�����y�����d[� (�(��N��w��u����k�c�� W�����&A�N���\��
k`��K!g-���r�� ��������l�����f�Z�j��M�t �G��&,��d�j���_X6QY����W/�o kj& ��jRo��4 �Ea�����R��\LN����x���Ea���2>me ��Z�(�����B!����lPf��=�@��r���C �ey��uc��9?kC!t~)A�pjL���w���9���X���!��������9�a�3+����]��)����d��s9C�Z���8{��z3rA���/�\�r��M)))��
�NFvCj��,���}{��Z�`�GM�! ���)Y�V;�};l�a��R�u��� �-� o�������,J����;3+�>r��u�b&��&�!�B����A�O?�������o��A $� 8��[o=?kC!t i�����/��W
�(��{6�P�3�����l��mm�9��Mq�V��1� ���� \�sVm^�}D�Pg����1�[����m\Cs0.Xc���/>�V�X�X�.��o
1����iH�P���� `|�>�V�]���W���� �T�%
���2������p`�U�q3��>�Sr&k�R�.P+{nN^��� �>�[�A!��9�lPf��� �k����z�<.!��y�&����q����� �L���P(�=C\fs�_[f��-�goP7L@�Q�M��5���m\�Ia��|�\b4����$��y? 8���L��6��b�G�*R�bM���8��Q�/S��� ax����-*��=9��@���e�V����w������U~�]��4aD��R��[�����B!��OH�j������i !�~�J�A_��,���% 18k�� e�V�����H��Jos�K?�c����X P;g�r e��H6� �V(!t�e����jae �2��S�\����c9}y]�3�L B{�H��
kM�C=�V�]q��V�����m[��EX}��B!������YYY����nnn�x2B��FC����w��;�.P ^W����XZV��j�������--+n��LsD��g�R����H�K��7/��{5��M
��� K��#�6"��s-JDFaa�{��(�-�M7�-�G�n���E�w�1�0(�+Y��v���a
��1��������L9�����=���li��l�����������!�B�'�kA ��}���_?u�����7c����u��XB��ACy�,�#i�
h��w�.P �vWkEG%���i�V s7�����Zv�����A�Yx (��UUq9���bD�����\qB����S 8[��������(� mA���gj��Gb�v��� 5nBiH9�M/Y�����`�D��`�{�Sw6S& ,����^'^�Q8"Gt]� �0\�9��I��ly��i!�1*�!�����|G���c"�B�n�k������y��e���������2e�SO=����:"�:�2����mq�U�qD�O�>�.�j���^�,zc�������O[�7�D����_5
P�01�qF��T�X'�����A��o�;������Q�Ue}�����!�.+kP'+ 89=c,��� �K� �8�`�q�Io���� ?�H��u�?��G�M��M\5���_�
���I4�h�1��;W�5�z���.��+� �@{�i���p:�{�L�y P���;��[���<�6[!�B��"4�G��|�� .LJJRGN�<9m��>��<��� �kO !���eG�u1���n�����m�;�}����$�W�
da���_;������~�����[����RxX� e���NH� D������'������$J���A�RBH�MC`�q�Y N�Z�����]b afl �2L����+"Qf8$�����r|E��[�V����jg���Vi����S*\���s���"G���T%���+��-)�� ��z@�1�.?��������p���q�b'�B��t���]�� �� $%%��W=D!����S������7v']��+C8�`d���}���>��j#2 ��_�3Gi��&�4>��*��� ��C�n����i�x��+lA�-�Zz����n=�2�`�Yh$lB�&]h���>�:�O���`ssS�Z�����d��R} >�Q�E-�;������F�Oi[�qOS��+O��6+��EG� ������`�����C y�{�����j $�� �Y����d��qG�����!�B�F��2!�~��C����t,z{N�4�DA 3��<��}*��
_$����O YZ�y�c�v0 ,��� +���N ��W� ��9�l�^�#Y��}��+��+����"���
zb��� �[�
>��pW�"��d �u'')�7'�T�� �oBr����e
��bg�1����j!��QfJ�a�w������+_� ��U�e�V��&�[��tp�����B!������������#_|�E�^����Bu#J����*J����.y���������� ���U����0����#���E;��|���S�� �M�f�� FF�?����O%�� A���o��p�,5` ���Fn�2q��R��:��H��3Y)�`b����/(�_7_^9��R�.�����1�f�c������$��#0�G{��7�!;�7�^3|����}0tV����S��a��[h ���� �Y{��.0�L��j� [k#�Bu]���;w�-��R]]}��7�G }��[o�� � ���X�R��'=U�%��rn�Z�DK������|Z��N�L���M��C Z�7M ����w{_h�� ���8���9M�79.3�T]�R��CN��,S����!������aD�B�����[t���y pu���P D� �n�'��(C` ���xS� ��x IDAT�����.�T�Fb�j��2���;���H
kN��O��G�0�R��������{�� LPX3����� kF���)���St��(�@�D*��3 �dF���sB!��y��L���n�:��u�}��w�}���k����l�iq!�� %��z,(�T���,k�����������sf��Q�<��Y��M$�]J ��'�� �>?�2��eQ$�������S ���� �����2' /^������~��}+�u��,S�e�1 \��`�u�:���yhSC����65l�� g����R�-W�~��/���.�s���v;R ���h�G�HM�����~�!��9K�o4ga��u���O<pP}������o���B!������2���� ~�����jB�W��������5� _�P�3F}K��D'�h�h+2��9
w� �C���� ����1���<���o^4��C��������s�5��5[SH���rGf"�Ug"���9������T;`[ �7����o��3���?�J;�i�'���1��2��=�%:�A�Y ��J���
�iJ�p�)|�>�e $��C�f� �8!�#�RF�?����|jd���>��(4���^� (��c���c?���Z!�&@�vT�!�B�h��/B�R���7l�_��r ��q��6T�c�:A�Z�5{�u�.�s���{}k& ��&����� ����w�� �P�����tWV�� ����Im����C�b�]�E�S ���� �K���nj�o��yn�
o���������������?3}�0�J�v;��� Q�^��:�)��UbX|�^�| ,��d+A y+�����y�h���==R/Z kht��5�w;��j�����v����x�svvo!�3������B!����;���n�g3s�����&_%�����%F
��5��m�:��kP���,-+�v|���GS��q��i����dY3S}�%� ���(�Sfj��TN2���L d��|����gz�%3��� �����2�����B���P�����v�4t,��#���IO�2-U`�D�xE��a�:Eo� -U �Xz�Yg������3���I�����)DTx�z?�� r1���FD=tV�j\�bc�<N0�~������U�7�k�Km}���� ]���v��.�uk�]�qm`�B������ 3eB����9���oo��.1 q�S�o�]��e�a������r� ����z�Lo���Wh���������.`dt��9����F ���B8�-�[C������0_�����kT�y��9%)���$���Qf��`�i�-�+l�.1h�}�F �!g �Q�g�)����V��'����8L��%aid�^�W�;R���;R��+{� ��M��'o�w\by���5y']8�����p�kN����� B!�P7�A����[���p{# ����?���������
7���7��Y�-�F=�Nh y����My�{�Y �5�L�>@��!(�q_�� �y���=�o[�������bX� �=�s�~� ��,���/!oV$ ���� la��f���I�,� ��no�����K�u��;��Z�_t*���+�Q������cA8}��F�~�V�OR��+[�����B���t��� !�B�h��} !��O��������b�h~0�Jy^����U`�T� �Z8�]2
�%�.�P8-� ��[j�`B�J��R[�����)a�^�i���(��� p:�� ��pU� �|b~L�v���o�mx�F!�4� ����1\�9�O�W�K�L�P��`�.�%H�l�Y}�n����T p��v�����!�B���L���(�X�z�Y�6��'Fo�Q��9��j���]����R�;�����}��#6U�8 (Y�����f� �QG��J�7W�-� �P0�#�lI!��q��p�Gb�R��%� Z��;��
�g P���~dz��x�z��##��)��+��I���+{�����@B��"�%�o�eYYSfR/�':�� !�B�$`� B�R��W��J� b����D�~7S�g��`Jk $�2�$�����KRZx�6��#1��������I��dQ���a9k������3k��v��R9 Q���O'�@L�����xYd����g�j 6�z����&�������� �v �iOj��7t����=-� 0�G�����RA�p� [����hf���L�>��t��B!��Of� �P�v�];p�ESE�EJ�\�&�{x�5g<�7�2S��Y�wY%S���e��GYfd�y
"�����=� �sfm�#S @,+��T[��'�OxN{kw�8�Q��� )B9Re5���tM" FC{�ea���� ����<m
�
W�_�h_� ���(��+�8��
E���i�L/ l��v��n}��}����W��rWz���N���4A������!�B�'�2!�}�ua�t�
�� ���
Fn��9y���H�i��
) �� w��� �q����fg}�I���%�0G�{���K� 0hz�x��R�<jd�zd����J����%!O�K���[ ���V(���q)+b�F��X����u�v�����/� ����i�����p�2� ��������fvP���� H Il��l'�iE �07���M����B!�P7�A����-���2���8� P��uz���8�Q7!C �����|si�&��5@(�/����� _�����cMuaY��+��
��'f��^ef� �����\�p (�P/�R"a�I�1>y!O o�������hk����h_+�plf��Q(ez��2��� ��������lj�D (����G+�����R!�B����2!�}Uz�qaW]��M��K�����:����"�� C������A�x��d��l)�}�u�|��}���;� �����Z�L�(L��7� '��S) ������Uz�{u�Qe���V�%s����� �(F`�����S��I\�Dd�VKu�o�/����c'�T��d���������#��XOt�^h�}���h}�IO;����pm��?���h}�C�6W��� �B� 3eB���4���F�����]wFfyT
1<#����8�)
�A;�3��t�P7���� ^�S���$� ���
��r �
�@��P��" `��
���#�Z�dYs^� � $�������V3��
P������I�9�-�e���MT�^�I���Y� ,��;� �5��&fNy���z[)�#���@������1
� � k x����
����| �)�ez!�}` ������B6g��|�N&� �B��
�2!�}�����,�������I&2��J�,�C�t�@��� Pd=�����8g�S*�(�N�x���r����"N
h��cc*���5�����.�7��|��|�,C^�],+c{�i��e����T.��e��9v8:��Qj} �w��JDFe��E�?����
� �������J�]o�|S�l���$��fO8�z%?Q����-�� ��3AT�*(�f���ug ��Yq��
(�>�A!��p�Bu_��� �M�a����5N-�r�� =������z� ��qC�K���� �0������� `���T0�KI���c�&Q����b�j�)����+�����!�-�rC�{�=[7� ������C��Y�>�x����x
�����_ b�a�I���R���^T��WkJ��(T��o�Q��nC�]^)���Gb��/G9fS-j0 ����M�e�+��\�f�a`xb�a� -or���i�k#��^����xot� m��N=��]��|��B!�B���2!������Cf'?������0���M����������>J��[V'��z�9&|�>�8���� 8�=+D��/�)r�"DN���]��e�����t�� ���&�"f/��d��&|N���X{q��j-�^�M��tO�(�;.��D��&�?w0Cxe � @���7���DO��b�m���xSDEd H6��Z_��|���y�-�B!�:�2!� �Wh�,�m�� �+�X�8�� f���'~����5|v�.V����A�g��/1_����(��_�`�T}>K��7m����+�����oO5��rI�T�H�� � &Dxk�v���'�<�� �L�� �j������/R�M�<�[��YL)^�)�:�=�K�#LWb�|oY�&�F!�B ��K!��R��d�����k���z�R���%����]R���V��h���#��[WX^hd�$��S�1W���g�U21%�������/��6�D�s��������?` 8G}����J���,/xHDNJ�X�Y��/M�������>���\�@���wH��}%?{`� �z(�)H�`K4�?���nrk�B!�1��A�_���q�
p�u�����l����~p�[ ���i�rnR^Q����o]B�����w����7���+o���JJ�[���1����<����:�M+���b��=\� ��!�mw���� H<o ��~��*�a����NPR;b b��� #���}{k� ��a�����L���n����m���=������H�:���[�i �-PE��]�
n�9#4���R�}N��Wo����?��GVD�Q]�[#�B�`P!�~Y��N�����E)w�����/oP;�(!e����JkV8y�������.�4��+��l�<2oe8 �������S��1 �}��� �tV���(7����gC����L���W0�YW��b�@S��,D�x��+6�� x��dl�����7igjk�*���#��fpC��!9x�z�� ��Y�y�vhk�t���C=Ub[(�4}B!�:;�} !��%z7���T�i�-�/�r~$JG������g����5�x*����i!����Q"2
� ����[��Y�vvI� Qw��K� mN���'|Q&'|�G�g���fO�p�� �J���6Ck�+����}���� ��= ��<D���U����uo�z��Q \��W��]*>������a@�w�/<r3�#7��|�[1�wE�C��N)�{0�Jy^W+�v�����U� oM�^5oiY��O@!�B?#�)���}_x]�(Arz]�� �M�.��~ E9����
��������{o���&���m=�M M�-i�F(� ��|3��������k�7,�(��m�J�c������{�����)���~�����q��������C�eENX��C?�Tw1#O ��2�ot����/�UO/��'
���!g-��9w�Z�c�6��h�����|J`���U���F�u��� ���8�7J=������]}�,/�3m�j��c��i�V
�����B!�������~������P��.Y���0l�-�� ���giY�����F�J���w��i`��#��$��r���g^^f���7�z,���������W<� gn��7��8|���p��D�R��M�I �'���d �����������"�')�n~qy5���
���������y�>���'*I����'P�k���e��Llx�QQ, l���8�i��q3��EWZe����Q��������0�?����=��1bK82�� ��!�B�(���=f� �~��T 5R��� ��xJ��.�3g�ZK�3k� � %�!�����ni��j�V�����8N{��?�U#2 `�����>�� @DD��~��eoO���\�eF�U�B"�"H�<� �� l�/�����r|k�E�9@�p��|Nv�� ���={������TM ��M�5n�J��'|kI �-tB������6�F*�� ��!�B����2�_(�&F�$\����O9�neI��,;���#3�7(�G ���) X�&|�����p��C�O�fs�# �B'���}���������o�oY������t}L!C��@D!��z���&�l�����F��7b��}*^�_�e
���� �_���RM�#��� t�3i�,}s�I �������X�����T�C y� ��L���
Q#�O�Y_�&����B!�~A0S!� %���6� .dKL��� �"�v�)]-8r�\ Sn!�.���T����0|
�����G�����k���k����Hb��D��=���k_r������#��=JY������&�B8F���~wK�K ��w��u'')�_����mqM'& ���#��Mm#�Z�dY��JI��)��:=QGTm�v��� � ��Q�R�������0'�Y\!�B�Da� B��7�R�@.� �o��B/*�9���A.�)�(qB������O|j����LLi����*���W �����Q}���yGSA�Q������O��(}� C�]{L��,��q2e�� ��?(k��t �����$�e��K�V�[����K�9�V=�Z�_����+�dZ���k�R����E`� R*��v*�J� rR�CN��������t��a����5n�� �s��uB!��Of� �~��i ��Z&���+��|u�LslikD�����r�\���\c�
� �e
����2dUu�����6t�}�n �Pyt��C�&��=����V�!&_�HX#�I��{`��W���i=F@ @Nv�]1Z"N�(U�/����2]2��E��/�2n���V ��m� �/�UG<�^ O h�g��� M���%�f�@}�����uWQN�Y�
B!�B?�A��E[d�.�Q;�����m)�U�?���r1��K��L�]X�/95���l�Q;�W��]<�9OU3I]ky|�p�2��i��Q���=3z� @['l
��A0�q���;�zrLV�����P�LK�^n��R$�b�� H'�Z3����OS>��J���IT�F!�B� eB�|�Y�a\�sF��T��$�(�NV�9c<���s1��P�� 9���_��p�:�+[��}=,������3 (q�� �&�-12~ �2�q��g�V��R]���fb���3���f�]]�9�;g�j��� ��?m|��X���a5;F���P ��r�Y�!s����U~Y�����1��H�����U%!�B�$t�6��M�lf���
?}��>��-���Gf��� #���<wAV%���6�-��� ��'�b��+[r���j9�>id2�Z �u��Dc;! �������%���~YPf�6^jk A�%������'�S����8��B �"�� �\���d�
�$��Y9w��Lx���������c��5]%�>�U�6�����$B!�BZ���=f� ����Dn��ugPu t�:��'�x���+��d~��Ws�>����z(��L���}2s������M*�����wd~
u�'�l���� �r/ �������2��D�l��t���<+�Yl������.m�$�� �� !@ � ��0��� @��"����Sn!����a����nM�8oe�4��FdB!�P�aP!���4���D�zu���6���i���� �O�I8����j�����PJ�/����_���2 "�~}�]%����l:�� |�{���;�G��e7$�����/0|��df^i��R �A����{Y~����]o��$��
���$�=s�� B��#�nh����w���n}�p��K[���N�W��D(����E����� 3Nz�DK:����"}c,8}!�B�,aKl�:_8.Y7R���; /-+�^5�����U���G_g��+%"�C�fm�`��(=�=};$���l�^} h�[���-�N����OBQ5�� �� o�Z���`��
A���g�v�E��m�]�.�7��hf �n����i��~}�I �aL�_ .5������Y������i|U������� �C6I���!������ ��Gy�uT��8��83��:�$(:3�"�;H@�-�}��=�^�����B���D�r� ��o��u���x��Lk���~([��iv���# �V��4��l�6�<`h�i_AA���*e�v����?������g�y�� |����Brss_z�%���� ���S��J � ��� ���mD�9IT��pQ'4��� ��\Bd%1P�O�������+��K%��:
Rk+ �|���q?�����*5)J�
x� J�� &TN8Q�"���M�A���w����'9� } IDAT��� k���r&G�31!#�6^�&F�� �����/-$m�w-����p������-��#�<& ��=���9�:��dr��P��oOt�-4��u�� O���S-i��d�������K���e���r�AAA~��*e(���w_MM�-��b���'l�����x��
_}�UAA������oA��O0E
}!�,U���$�V��s �I# �.�~�������)��p�sG3'������y`��5�6}���9�+��CC����/� q\|���* <L�.�$�tQ���
��.>N�U���L����
��+x �64�t�?�h([��� u����H�������,�L���;n �*�`��b�,��{��A�WWJf"�6�!b�\�l����K0, ����P�XC$��]��}��|e_�6���IIk:����:�������/!� � �=~S� !d���<�����NX�d����7������ohh�Z��UA�r5y�Fdy�>t��Q� ���� �sh� u���l{��b��i�5 �p��� *��<���G��2��R# �Q�����d �u�� �9p��e�p��$jC���&.N�����!,k�J���������������?��t���H�<Z+9 ���~��|V$�������k�Pm3+���f�%^ ����i��+�������YR �s���1K��3����=I��$�t�YQ��6��B��6�YQ� ����4:�,��hpK�v� AA�M��eN���NLLT>N�0���������[b^.��?�r�K�nQ��)j��`D5�&mc�0�3O_�p��� +2����m��y����Jo�
1w2 ��D���^�� ��x��CG�j������
�?qIB����$)K%��\Z � ������� eLl��������ops���������-���[2��������
�Z�����<1S+�[���:��������U��vF)��cb�z ��)������s���� ��|"rf��g��C�a|� � �����y������8����TWWs. �� 0�� ���!���r������#� �a����K �q, X���c �d����H6� ���G�O�.9���<�^������I�\N�zLz�v��-)���������@��@��-���zGW�)����G�-�?2I�hi>�����VR�����RI;T�e���=�����jd�� q��)|;4/���-�� Q���z"�Z���k�Z��v�? �-
�6���x�d�t�!�����:�@��gJ�������vFl�^6���g9�U���X��}�| cT}��O���_3z���� ��F����K� � ��� ���~�]�g�(#�F�1 n�;�s)�'�� ����6����[�]|��a, \P���z��)`QC_��x���J��I���%t\n����S��%������������N 0��Wr�� JB��.����k�= I<��O��H���v ��n����x��#V�i:��B�AK @ � �9�����ze&EBN���7��c���zh��s�Q������# +7[�ft������i J�[2|h����(:��{WL�zHG� �6 �X��[�1\�\�����E�;�TT������Q��A��
���h�X�1z���� �k��d AA$,X�W�D_��,gQ��� ��ra������hM@ZC{F�f�MZwl���gV>0e���� �/j���yK���Z"x��O�����vo< W�;�sdEF�o'� ����G���� f^wg�3����! ������z��ns��?��"��]��m��������"�T�T��
2 ���7�r�A�����J�G�����4�*@,!\�0����w���������}������}�c��r�;�3f��Y�6���Z2%�� ��l��a*�AA9K8�*e�z� ��K�������R�A��%�����3b�)v�_���!� ��VHxCp�y��A�\��W���Q�o�?�{�Ej�f�]9E#F���L������
fVz"�qwW��Q�
�����3�M���.��������]��{��e# ���+>�4 yE�������]CI���Ub��KX��kf{���l � ���d#����J�6����������� �L��� |���%Lj��x���9�3�1���k�M��:�� �,i�o�
F�AAA���J �8������7l�0rd�A�_}%@����.�{z�i��{$�' IS���_� �����=ab
K�0�&��{�}�O� �����D �wt���N'�D+���������is=7?_9m�`s�ffx�iG^������l������+�X��~�J]�6�?A�
`��^���s ���$R���9�t�.�)�~�������+���Ut���o0,��u�~��Q+gt0�������_;� l��XF�wK��>� � ���YT) <������z�j�e?���������_zS��(VU������EN�&ff����o �����l�Gt� ��������`�W�Aw�H�n�;�4�w�z�!�fG�E�����c ���R��s������.|�}����F�QX>*:6E�� ��� \�U������^ h��^}�6�\�T�l,W�K�S�9�]���&�)3t � ��5R�� =���&��� _2�P�������V�a�rV��)��8+��K���:u����yl�`��f���+���U�L1��v��b�� �v�� ��)������>� � ���[���$i��Y]]]��������� ��O���+s��Y�t�R�e333�,Y���O�,!���� ���R��Fn� ������m�>#!�G�>v�W���t)1��-1lBH����)S����>>�^�O����&86��-��������f^����33���,�,� @����x�c\��a��� ���"��b���[4�Np~�h_ �=��=�5s�8�\�8��l�wovV� ������M},��O[�M�C�� ��Dt< �z��'�E�a@��v������
M�G��r�N�c4#Jl��#�b=|�P�m�p��e� Mds�K�@t�k]��;z�r �(��&�7���������z���� ������ � ��= �W���������!�o�U{��V��+��� O7�x�-�(z�s����]���b�� ���M��K�\SY9g����j�`;��SX�?�/i� �����O���1KV���U� ?���r�C6����u]�u��?�*���{%���
�~�t�z �)o������_���KS����[�gz;� �>�Z��m��������0}��-Q��C�����^y�%��gf�NN_�6w�����
�#
�]�:�oItX�mJ�M����.�* .$3��������[�|�����_.���� � ��m �W���} A�����)� �����e�M��l>�FI�`����~��
No�(��$S������+_B.����U:L�a�N�stjFj `I�;���� �S�" �+���?��� ��� ��r�cW�f����VDF8��E��7r�� `nF�Z� ��?��|�V�c)P�mj\�z�'W�K6�|�4���7��k����i: t0���J�m0�����������R���0�����B�l��W�����j *l��^�?� AAA~Y�.�_A }�AR4r�:�������i_= l�^k�s�6��_;��T��������?���L�H�V&�9���n89��
���� �b9s�/j�����'���+�yt�.� @]Zo�� w��;B_��%F0$Do����������D�6����+.������W����F~w� ,I�����)x���VO�H�����r��z�� �c�K/������}��8 ��K1W�L���s�$g�������UT��U��g;�W�c� � � e���K������ie��x'X�����a�Nu ���d�c��B���4��!� �� |��zo��& X&��+^�,�{�x���}�����53l����T ���T��� �[k�[�����<:C�� ��hRI**�#y���~$�t�3l���u�7~^9m���I8����Q �g���kf��kyj��i ��� D#�� � ��]������t��(���7kj�7���icRD�S��T�1h �s��F����u����{��iC��-�F� � ����(� ���4��O��E�$�Uy$���<��"?��O���C+���R���::^�_������^d{�� �?+���m���`ueo�?�w����������Sei)��4����������*�����8P?��]�^���e�-�M�w����ey��
j�#��P����L�0|�2g����cG�.�X�� ��v�� ��[G�pu�um��q��S=>f�f��1��JyK�n�t�c3��|�+s�j���D]Z�c%O�?��T-q�k�{ ���?�1� CbL� ���E�_AAd ��2��1N/�zfF %��?R�����d����r��K��7�_��,
@���wdS���~���Q0���?E�N+
y �~��Tln����#���5� Q������Waz�:;I� 5=����
P'a (% GC������`# �m�+?�3��(c;�f���<[un�����(���5p�R���l{A�����^Kc_�.��GM��/������ ��i�s -Ly��3;��*�WE����)=������Uc�:�#�j ��b�� �[)#��v��Q�Z����w0 ��O�gN�q ���`k�����FAA��e9c��e�F�8)�uZ���Z���V�9�������[�It
��F�������=����u��s�"�Z���R�;:���cC��D<n (R�{A�5�)���.��Tl!l�� ������w��Y��6���5})2T���$����a���f���=H�������UD�I���H�Ue���pq�����&�/� ��� s�6��������hO����G<��|5[�]�O�f7;���m��f#� � ���� ��GZ��%���$����J��)!86�-��y����L���mn�j�
��~.'�/U����!�J)�����%����5/ ��.=r����k���^>��M��mkR$�za���q��@����p�fm�zM�� �Te�$� D����"��q1�5��N��.D^�ml���Q��� ���
����
]��4gE��%��� �u�[��4mL�g�/�g| �?j��HK�������u�������lj���m�BAA� V� r�8�tj��Q��0��z��� �b7�Y�t0qf��gb
a7;���v0�A��$��+G�+ z' �=Krk)C>m�������uAQl����W��y��6����f�9��+,/R����fV
x��Q+v� �Su��_��f�\a �iV��@����tn�� ~.�� �K2,�J�]\zs���]�!�
���r�y���/O����9S���� 7M����!:�q�u�k9V (1f�T��6���{� �u{ ���%w��uh���c���AA���JA���N-� JeG0���DY3�x�����.����L���lM�]
� ����1�1��SMj�]��J��u��OG���)��aSL����N���~���H��
G
^�~���{>�$���� ��z����C��E���+(�,��R������
zhr��fO����������3��#N���W���<� �\7j�����(������jn�������@]...�X�o��bcA>e��7_3g���aou����������%���S��AAA�� +e9e�=P�����? $H������Bm�o��T��� �/�+S�����������t��iR��9�Z��~���= �~u�w���)�P#�M�k�7�1^/4����e �~���v�S���M��%{]���'W~� ��d� Q�i��^0'��b�A��M&RS��-<��9���u�
����V���N�C����N=�����.�ty����Q7"������(���\����i� ��/;�Sx�&D
�[�/NEQlk�s��N7�h�s� G��s�|�P�tA� 833�� 6v��U�R��FQ[�����AAAN��A����� -L�[8jR�)���5o��`R�O���j=��4u:�o��i�J�Q�k�� �:��? �h���������gh��� �hAU�����2�!��8���������4�nZ��[����2+�0�7��F�#1�7�~��%�����
��7GJ5,"��Q��~S���^�5���&��lb0�zg������.{}k?"���������8�ll���|k[(�N�p�0�� &,����'j���/������ liy���7�CK]�����AAAN��A���?b�4R� C}+;N��WC�Z�V�
@@��0�)9Q�d�?��8S��v���a������rff������c�z�ky�0}�\��1�9��x��@��?�dI�\����#�
�������s�q����b� �������*�ZG���Y�YqS��nM�u��?���?n�o������s��v ����x �yt�1c���aE�j�Fy����}��n ���hvB�>v�����}�{�db���+�G������%��l�S��AAA��+e95��X��l�/ a7Q��g�+86��d��v���i3����+Yx�M�D�Mg,#�o .>��F� �����n��.����#����>��c���4�N���\9k�R�!����@R�3v��sy��J�$�V��s ��������T[\#�*
!R������x��;�JT�� �����
Ey���w�{�
��\T�wl���6-��68uUd��������4���. 0\l?O��C'���S�#K]���AAA��+e����J�?�� ~�� �/L�K����/5Q���HQ#� ��d���
�}��
�����~���/T��3����y|z��
� � gs�9������y� ���N�+y%�N/]y1����:���}/L.S����� z�1�DI����j �/�*)��M��{���8�������>�����f,�'z u{�c�^k��d��t���H�K� �m��|f&���| �\ �����R�X%���s"TT��C{ ��7N����FX�<�z/����R� kjAA���� ���~�]����*wD�|J����RM����iJ�����i$����cS ?�{�Yo����&����L���_|�{��WK��>+���467�7~~��7z�����@P�� �g��.]���os]h�z��j���9�5%4�.2� LY� { �i�~{5�S����U
��Xi�AM{Q�3r�G� ���6SG�Cdz�����Q!����u��0����Ro]��(#T�CG��� c'j�C���C^kS7�\gy�Z�| �H$� ���B2c�;��(�.���W����=m_z}�`lb��8�����F[�� ���� ~����a���YQ���vg�?u�\� ����s�P3*,� � g-�� +e95��X��l� �_L���Mwdz�O� 933���������h�16�xM��p�M"�a���T�������I����/$E?��la�� w ^/��������~"��eU��%��tB�p������e�����X]����5�H��k)���vV��ZE ���n��e����FW�D7!|�P ���D��A��v���u4@���Q��X��}��� ���.k�Ct�G0��W�!�*�Dr�kp�9�m�����9������t �h}k�����:,uAAA2Q(h L9
A,}f�Al��$m�QBH�O��{�e����*��T|:moG�C���_�6�}��83�������z�t��~ uLqE���� L;� *a#� ���;��S������%~6�������� ���_E���Gn5��(���]�4DL ����c�M�\ @��l4=U �Gt u8�[���A^�I���6u�������yk5�\h x�|����=c?�l�cfH�����_�U$�3������� �W� IDAT��+������ � �/��|��JA�0��y?�+����� .����/F�����J���]'[�,>VM���9�4��:����4�5N.��+����&�����������U�����K���W"f xcxg���Z���7�����]�`�����&go�Q�%�E 8��4���!�!vE��&�}bk7�� W��4�Z��Q ���������J b���h�~�~�u��6v��S)�!��2�����o�� ��_����lj�[�f5k��\$���O���M�d�[�g�� D]��3�AA 0} A�_�&�d ]��|��������ie���H+������9�/�]U���34���KN�$�#�f�wa��U"�����].|��W�%;a���wr��&U���8��2�%FC���&�]����:��X����W��%��J�W�:%�_EF���*�C�����A�����\MN�,�PN+���" �o��9� �C��
O���-/FG?B�� #y�FY����Z�D��mL������) id�3��,�,���9Kj������)Y��NX>� ��G�'4;D�D� b�;J�7g��3���&�{ �vi���UH�|c��H�����Z{lAA�,+e�9�-���LN�t���?�i9:�X=GS���V��n �Y�VI 4�?>+�M`����p"��Z�}��B�d��|����T��� ��p���,]$o2��x%����s�b���V�&k V ���y�iw%��<D z����F������� ���Rk3�N~Q� ;�v����ltz [�%)����"\��/`;����o
�� ���W%���^hL/T/nQ�.��w'?yJa d�2�!���j�:+q���yb^�~��W����'B]��3R��8�|�WM
� � ��EA~r�I���`��5���[2���Qg����e������{:
�h�� =�f���M <L�A� �����X�T)e�u.^c�"�4�;�&%g��=T��� \[v�T�>�Z���� �1�k�C�F<���^v��x���'�IUk#R����� ���v.!N�: P �*����k ��@| z%k� "�� :=u �����9��#6X�=�2�#:�t`�37�X�����$�������o�k�e�D��w��ei�n�1�S��t�@e,�|�1��-K pf��
� � ��EA~rN�UtR�
M$���,���tH�Y���6R��S�um��b���`�L�G�Ji�e�M.��%��S�.2����� jW��m�����
N@$�P�C���C��S����s�^\�[K )i� 8��P ������'���P ��k���`���� P��)
�"#���N(���km�nJ���' �{L���O��{ ���C�����;;
���90�HN� ��)����c�� �<�T�Q9m�}_�Z}������w U>�e�>~f.(��tg�" rf��gb� ^�AAA~��(� �ON�kE�����.QrS��!5��m�w���N+����u����3�8&@��&�ZgN�����X�!P8 h,�o�8����.xJ"�D�
i�}� ��wuL������dU�/��b�u������p���+�0�3+=���#� L�C���q�f����%R�-��,��c uC�Pg�������D��n�p��km c���y���� ��W� �t�7%/U��Z�����G��S��C� �C��+Z �s���U��g3|�$���p�$ �6���{��D���Bf�]5�G����q?���a@���8#pGR��
� � ��g &B
4fn����O��q%#������e�f �� �4��le��]w�EU�n��� \�����9��> :~1��������1���3�7��vQ���g$$zA��[7�$��bEf�[+o�*(�E�}F-���4�E���) 4DRU��)�/�F7��(\����� �az�:��������xy +(�OT������
��8>{(a��D��o�U0���+�m�m��Y�^�Z�����e �g�^���0@ m�0Z���? ��n�w� �O^���nri���W����i\��9�� �3��K� � ??��~ �i�10�s���I����*�!�>s4nY�����/ �K������t���#��f����������z�������!��u��_RC���������A�S�a�3||��
M[�1�0�+�a�cn��^�������i�Y� o3�Fw��TZpP����������FN�kSJo6������g���$J�C���Ve�
���� p|�w�@�B H<!���"���F]� � ��1�dT���m�w%�/ ���P�%�o������ L�tz�}�%��>�,yu{�z:���"��;A���Y���Yo����&����tAA�l``��c�����s��]3�B�X���k������h���4����7^j�T�.)u^r_�Q90�5�{�M���v��2'����!d�+Ew��)(j�� X}���3�4 A�P'������h��;^�Di��$Fj����y�"���j���D��<^���D/ L�o_�S�� c���� �2|z�Z�Qn���K���n1�J$��qS?��������La �w��|�_;��B �S�K���z4�7� ��$D���g[l�����'� �2u����G
�� ����=AAA~9��OAd � =���s�^3+=��x���� !��� ��������g
n������ S!c��CH6���4B����W����ie���H+++�P�:� S1��59�g�J����e�u� z�Ih���h���������~H�q���8us$K������,� ���!��L?�L�� ��XcA����-c���
���u���;�&#�� 2����.��C]n� !��PQI�^;^*��U6 �1Q� �R�M{�Y�(������d �����47{�4�� AA��(� �2� ��I�w�M2���[ "+��y���!���K�rH
�B �.9QM�|�J �7[F<pe�.�V��*����z��oGW��i[kL��HU����{s�yQ-Vd��-F�N�T!A �������=����%l�����#�o��X��@�H&�<H����E�!�Cp?�.7 �� #F0 o��1��u��)���(9�_%��[����b�5oE���o�U8x�:] I��r �,���3ps�e�����"� �� �u�[�1)"�����>{�v����_-U\Q�V�l����Z��A���OuK��<^��v�� ���YQ��6��B��6�YQ�S_AA�M��K�+#�p����P_��zW��s8 �� �[ ������oM���V������������-W9��z���i&�?2��� R[����=�R�����)�[MW~����[�����������:'WT������#��'&<B�*6� `Ar���FW��W`�]d�+�-#C���a������
�����S2��i�m������m/�6Z�RM�R�� ������0 �2����i����& ���/��� 3�&r��C�� N����`����+��2���mcA��$� ��\�a�_��k.���3��bU����!�� �3f��EAA~{�(� �2� ��Ii?8���w���.2����_B �d��� ;�e�TJ~�z����B_����Ki�i���g4���#}����\S*[� H�����1G�L������� )i� Zc��o�"�Gs!�0�B���$��d B�Rm���� I����XQa��U:��"q�7���2F�F��R2�@2 �i��)����Qd �R��m�5��"�C}B�%�2� !�q)��M}V�`�1]\r��� �{x�1���f\�a\��`��|�4���.�������|[��Q�����h���a���=St�{R�� *����GQAA9UP�A�_�9xhF���s��3+r���|�Kl�kF�Z#d}�����W��g��� �\�f� 86��!��!�^�������~r���!����hK�� q���O���y����"=#=;�^�Y ���x�j,`d�����N�������!���4&I�{��Y����sr5�H��,���*BN�X���`�����>���Z0�����>%�_Ev�h�����������.\ �}��t
�����������R�20n ��5�Jy�����x�x��:�����S�UE^�+�$����:�'� � � ��E��Q�����`+t� ���U�;�h����'j: @�)l�cM�� �]�5�I#�� a��B z���!����n��!�^���=M[tw� `�e"�� ����g��[�����,)$�&�I~#����(#��t�A��8>{���������a"��@�p��������\��7�#~[0�p���]�����F�f ������stR��� 7���h�=��j�Z��#��$��p�d����H��� `�~V`�&< �g�k{cN}�'�$� � 2�A�_AN
Kr�����G������C�]�U �ct��H���NW�;����(^�lFFd�\K����p������R
u.�I�b�LqL=�jF��S����=<Cy�������������h&6�q|7B0 1b������L�E;W2����F�� G�=P
\\�i��]
���
�� 6sj����W�����/I ����?�E��M���:ZF�n l�?��e��?AA�W�2���rf ���~����V L�%bi���#�F�?FG����jM� ������#�/�o��(��je�.Q; ��� w`z\��(�E�v�����vGR]�H����|'���x>=�'�e�ND��O�,�9��C�;@5
aY]n������\��:��L~�$����.5��4 ��d����+y���?���S+N��yK��6��������������3�1cf����%�0<gI
�Tm ��i4�AA�,�Pz
�g'��SB ���-[���N������I����������C��� g�5s5����;�T:�o�LyU.+�X����q�:" ql��MV����Tw�+��������$���[eK^�h
a�� d�E��C)%
�uoO�� � %� (L!� ��q�f�!���!�P�D���g��N�Z�v�3�1��c|lk��r
��C��@L�~&QQ �6Z������loS���\��#'�� lL� �0�h� ��|#�t����q\����} P��n����������H:�/"�~a���g|�K�AA�,d`���=
4�wA�c���N�����}��~���Ry�� �_�r��MsNj.�N�^8j�8��/}���;jA>t���C����a�[�SEb"��q�ff���0o?��=�+�� k�����$"���@���T8��v�p�^Q��NT\�_$��k����A��L�aA���4�H��X��o!�#� ����$C����i���9S�%o! �hY%�H�����!��!/��� �k�o���'�zv���Jr� /��N��D\� ������4�]�����
{�3����DE������!� � ?��~ �i�ATo_���_�`�/������������9;>nq� �`^r����yrYM��]# �����)�`�����\wU��;������3U�Y4���sr�P������S ���J��[p0�_'�E���M����� L��}F.� �6�f*�d~WAx>������e5Q aQ�i�CE��Pm�t��!�)FO_+��D����� ! a�
!�������5��� vg��� � 2U |v���U=�:�� ���$e�:7��[l�F��+ZM�C����1S��5_��bU��;|
�F��S�[c�� � ���� �����P AO���'@EA�qu����@ ��U{o����m� ��e�}�����Y3��/Mh�����H��$E��]M��]����!�!7����� �d�!D�"���l\�B �.�R�Pb���tx�K�����A���;BU �5p��E2��AJ&�����a�t�@)!$># �o�m�4�9 ��**��k+��(����
���d��}������7&��������oO���u�(
����(�z2@�W?�O�sWC����O s����o��7���� � � ���� ���_z/��HlANu�P����>�Y�����E �Y����w����� o_}Q����4b|�^���aZ�^ �������?I&����/��[��N����Ll� ����r��t��I����,�
j�[�ZvtZJ"���-C�b6��5B��R����3�4��5��E�W5�� � ![�F'�� ����B�� �V_a��Oj���-OE
�hJ�� �����'��D�������U�N������_��c�qf�;n��
�`���C��C�<u{�c�bkw�����^0^�v9C��ac��� ������B�AA���� � ���������KQ���[�����>�J$��V8:� uS|���E7���|�:��_�b�s>�1�5/ W�J~.���l
?fkQw ��=~-�x��*{{Z����33�(���p���MSg%-�3>1v��{,���|�A��W��5�yF��3 Oo,i�0���3-����k��%M�N�R �#� M1z���z �u ��
����Md�'��?X �� >@�������pu����!��c=���5��\L6Yi������%q"�I�������s��
1�n�* �
_���.��ywb�-gH#kJ ��)�$�d���OO�&�Kp~��xE.��^���S������_B
gJ�v?�J]� � ���} A� �+J�J����hZ�������4�c���6$����hP7=1:A�4�k#%���F�|t��=������n$���}���+�{v%�B�e�����?�_��xwL�����K��� #B����Y�0=/� ��mc�����#QF��H����s ��!1�5������y�# Pd���r����7����]�Q1�tN'���Z& C�3�����>���^�B4k�� �~�y/�?P���B��Q��S^qCb[���0FCdn���f�)M9���}��>��� �9��.|�#��o��{���I*+����;�BRu�S�L����w7��� j��1S+O�7W�\�������s��]�����O� � �c@���30-�����i�?������O�N������/��+�Ko��F}�����e�O����Vf�$���?������'(���k��Z{F) �0o��u)M���F�Og����s�j����9;?>�<��F��v�������� ��^�����Z�E�:�T���O�J���o���k��b��:��!:��)=���?���������.����7�:��g��C%��ur�����0r���[g%-S���N���o��H���v�-+������=\�di�%���� ����������U?���&H@����.��gH_BAA~q��=�/!��i���<M���z��A��:� t�dk*4����&��� M��*���}co 2X7z���D�� �-c �0��>������^!��T�l-�V����I< [GY&rLO��W� ����i��t�O���xRE ���i�(#7%�Y�����%��� _P
� �PPz���� �ac���M��W���� _����/=�x�qEp���������F���l���� �9\�l �:au���������mj�� 0D w�
:��O��'� ���ae�:7�n�������z�u]����%B'$��.�tE��1T�� �~q���(� � � �(� ��i��;f�������������Vq�%�xn���)��#{L������y�2�/w��������t��]��'sff����a ���G�?���P����>}�������`�z
�+$\glr������4�z�w�])���q&&�!����\�>�E�����.e�d�_���b0 ��^^� �fn_�G�ysH�^P�� Q$m�a�m ���?�l%����~PI"���G����g���[�m IDAT�� �fl��s��NH3��NMX�OK�2 �� �;��L� ��r��E�� �������B���p��W�L2�whd�x}��[��������b����D �n��L-� � � ?'�)� @ ��$SX_��33���O�j��"�l�={�����n�
�f �g����O;R�X��,I-N���-�F�_� +2 P4r���5 �X������uy%cw;T�E/�1N�;G�
YU������b|�j\?iYZr��OZ������;�>*����M��"^�_���t:V�� M2�oHh[�Y�� ��.��a�����5��\:b��������K���nK����p�>�^�~���������}����O��w~�|��7/�:T�K~��D�\#� �*��WeEF��;��yM�Zv��#��
>�_2z�uQ�����=�Z �7���usC��P?��KM��3\ ! ����������:� � ����2�+ `��ES�"+ �ny_3Mi *L��oXzSr����3� $z �%[]ts�{CH\� �L� ��C�������Wf������7>#}��PmWgx����������m[����l �W-�����\��-z{jdh�����#�0�@��,TT��C�f����iG/af��Q���s`yS��VE2� �DRR ^��%{C�9�Y�8 N�\�RR������7�{L��)��t���f�Up��t �y4f�c���: �pV�1��$=� =�@"
{�� ��i%3FYA��+G&�y��W����9D�zL|����os� � ���6B��:�dx��3AAA~��(� M��[����q��:�MqE����.��M2��
�S����@ )��|��1�����(f �����&E� F���������0�C���� �����C�a %R�.z�����Tc� ��+e|� D��W��!L���f���X"0IA^|�������&���Jo�S���* hw���^��y����O�!\R�������O� �SOmZ��m���Gak� [� !����������G��j���rO��{.|��^k^����(H.�=����a��n��=6$U7z� ������Mw�W����]/����xG$=?� ���L�""-9j?
QAAA~`�� t4=Dn������D�6��S��.J��Jz�6�G���� j>�i����$�������F�> �.)���������+_O�x���#4S
U��7~����(�H��HU� ����CM.��4�DE�|}��3�N�o�u$�zY+5��k=��0c^Z���cs$K�5�V[b�����H+[��I� �\�$�y����r���*��
�����u�v>�t����������l��z�c'���V ���XtN�~���T��������T�����><�6��6�����oAAA� �RAN�����,x�����������rF��Y���%�k ������Oo��Z���:�f��u� ���G?��m��6����}��y�A#W����i!�L�o}j�5^/�\����r���. %� �G�����������Rgf�I'��# ��} �k�rH���}���?��T5
� G���ZJHIc��H�r���u�p�95y[r^=�VG���'�8z?���$�� ��Z��CF���7��1 ,�����=�M�����<��I��� ��.��Tl!l�!���1K��h� �B��n owU���@�1c� �tdW$-,���5�:g��U�
C��qF �(#k���0�� AAA�bL�@c`��#�8�*����Dm�����?R ����)r�������(
,v�y�pj��e?f}���������\7=������ ����1;3�&��~%�+}X�n��� � XY^�x�� �Bx$f���in������J��?�����c����$ )�i����5-3$������ :�_f��|��"Q�L�.4t� );d�L�����5�-^J��>���u����� ������*���#J,7����ov���t�5P�s0�C� ����N�y;��33�>� �~]�pd��#i5���*W_� �����zdk����1 ``� ��$����}os����-�$� ��$ q� � � ?5�� ���4�+�����!�����1����>� ���K�jk�[�^?�Us�_� �����x�����y|���9�)��3���LC�z��������Vj���d��T����?(�������4O���a ',�r�_z�����%��)F�P��wZB���x���jn�y�l�}�#�6 ����G���K��9B�vN��x x�J h�� N<�o;����������.�s�low�$����Z8 n�w��} OBRu��&D���Ud�'H�k��\���������D���O^�]r[�_�t�,yhrt�����y|T����s��;3���l��- R�l���nU�EAA�B��Vi]PT������V��k] YZ�REQv�@Q�,�m�L���w9� w2s��I�5����G��3��{�������~�����(AA�a��Kr����@�L���C$�����`�w���&��TA�)�j�D �E����>��0F��ZG���U������i�� ��;�������>�?��� ��,�pL�C��N���[��m 8+�
����1M��H�Q}r�I�_�L����K���&��D�D��_�,�Q<R��m@#���.
%��:w�3H�FZ�.^��Q�bf��dP��RC�� �IL�E��,P�e���L�j��$�V#��P�t
V�!�6�" hK��*X��w�5�Q���F�_������w��r������C�
������Z�� � � ?*�RA���
L� f�VM_����;t�8.�tW����R�(�i��P1����==&���ut%VVS��,�_����}������ �� U�g��
�VB�o�����E�T�L��J��'������q&u��U#��#F1Q���qmA���� V�aw�?�@��}��)��5�K`��5�q�TQI�/ �D3�n�k��s; p�L��q����,*�e3��<���y��F6����2�z9���S1J:����'�UEW���_�lUe 6]�Hgm�Z��3�%�-n�R��T�S�� ���!*z������3�W��iC~����
���0� � ��}P�A�3��d��!q.��&�����TtZP��1JS� cE ��VU�e�Yt�'����B����:��w�Z]�{4����c���N�|t��aC���kH�����(VVQ9Rd!y����<.��S�8���D�pQ�9�H��&�P��R=(����P���IR���N�C�% `}}���1�OX�TR�
��� #�� s&ePzE��
���[jv_�m���p;_<�����Q �j���8W�E���������������ej�����~����~�c��\�;��6���_�%3�������pt����\��dQET��}����� � ����� �%=2��� ��'�-���F2q4�F ��}n���
bi����7i�}o�?������^�ST�?���t����vm�)>��q��A �% ���y�6
�
K����;�
��(���I ���)��� _�[�s�" ����n��� b���i\�kR�_f�X��~����!��E4!��d;�E�s�s�<wn5KbU�b�����t�� ! �"���3YA ;���0�=# Q� #�!�\�*Z_��h���
|�@�����bo�lKz�m�ny����W�w�s��0mv���-Pk�xZ�J��� d�%@W��XYei|F��� 'r��' 2� � � }��h>������eUe����?Xp��v��h�rth���3}6����=d~|��Ad�Us������2��K�^�P���;�E^�Q�5�!����e�+~��x��W��������j��B���9���]M�C��N;q7���)#u2Z�
�����'$8F
*[��K9��mr]�C��@
Z� �u������� 6�� ����� ��B���<��[�8�^ �����IQ�Bv�����{�Y�U�n�s;=�=S�� (q�[�E �����q&f������� r�Ykv����AAA~���G{l_B���Q����z���] Qd *.G����0cZ�� ��7��l��/�)2 0��[���@Q q�Q+O���
E _�-`O�q^���y������,;t���{�����B��:�?A>�%�<(����}�=u~��o�aBj�����nz��pl����8Y�/�����I���A5��������������8!^}!���FT���BP���/k\����� �b��_K���WE�}�m{Z�(����d)��8�yFH�};vy?���}G��'}�w��W�w7yeI�j=�����*�U�f� FAA�>�2rv�s����u��� �7W)b:��\jVa�/�
=gS B�����?��O"�c8�5�.mcTL|�k�;��n����c��m��O��?n��IL|�Ek��g >UtB�}8rO���&�6��2��W5��ZH�������7�` �������s�R�7�LD�03���8�@�kuT ��K����J� EV��H�|l"
� H
0�_�������A;C�9|�64���o�.5��(�P�_���auF���f`
p��u���������l������������X
f�x�#{��|������aAA���� �� ��� ����Yzs����Bj�v ������ �S�:v*%���=(�9��7�p>�� ����SVU�=���:�1����<�v=Of���.��$�2��1���/��&w��[�9�x���� ���{!��8V�Z%0\���xg��]�xK��O�aw- t����j�����#���1��[�y�W-l�H�~�@`�%R�~���2)� ��7�B�6 �S�|CAFQ@���h (�W�
���>j�Y�L.R6$ QY>Y�x?�x3 �~"O���`��� |�S�P�7��"sCC���#��c�m DW� �����mq1�A��3��v���,������eo k��+}��=� � � }��A���5�{}����T��Q>[���A��?:���G�.�j�U�9Z�X|"�u��M��xI[��f��������"�d(jS�x���f8\ �������l�6K����z-K�_bf�����kV�7KV����Oae}����Te�}����z������� 5D#�"*��N8�HM�X^��d��� ��YNE����^��&x�\[����n>v�
U�~y ��gR�5������~`c�o��&(z�� 5>��0�m����y��A�o���`�x���� ff|v�>5c�vLe������3��R�{��`v��*��`��*KAO&�� � �w@QA� *{�8���@g��:��q��>�H���X����<b��z���"R����y�GW��T�cv����-1�<��{�mJ+Jj�$x ��� J^g�n��F�UX�d \k`����G�KO�-���R�/\���a�V������r=�A�^&Q-l"I�lAA�+$�66�������g� 8��d�w����Tw����RA~�#���/�����q ~�7��s�����_��� ����F�����S.4� ��(���K��������<�)� �y�X���s����]/#�w%�$�Z� ��H���mAA���K��Q���w�S���\
q9���5
z�I����p��g 5�� ���Tl�M�I�H�ul�7l�����$��?Y��M�N��cc��� $���������~�_0����?\�����g��JG� �^YN`�_��� �x����
��Rj' ������ s��pq$}I���R��S����%hNR����� !sw3 9���d��I+54��/����� V���c��
� @ ����`���������t7@$|��uqq�"� `�q]t�j��c}�,����| �L�>��w����� n����r���!x(�2mQ�����sd{���s7~F/q/O��v�x?�������&EAAA~��(� �UT����i ��o�nE}��2j�w #��s6f��L �]Pj\��V/�t��Fr�I���p�}���^A��&�����c�mW�wV]��tII�@��������V�����kM����h����qC���K������u
) �LA��p
P������F������1���K����� `�z�Gf�;b���f�9��TR���%5;,7f������� �}���<�!�����?PCsH����-���e�REQD �/��zc��\K��q�L�Wq��}a�mP���O������-%!�x�2z$a����4�&f �[W�<w�&/0 $����*_#7�j� o����62![�cnz��t��21K�} 6� � �� ���^A�X�x������]du]�Rv-*��g��F`�������3]�|����M@a�ry�HNhd���%�y��Be����� oZZ���v�5��WK<b���?)���
�
�������\������U�u��Y�>�����D{-:[3�%D74�7��;E&� �E@* ���������_� �������������H��?������x��i*)�����*����qO� �� ���/&�?~�>sJH��r���,��H���^ �M�)J����1�l��r�zc�����0`d�%6&� ccG�]�#H�m�<����e��3@����?�&od�� M�����21K�����.M����(� � � ��/?��{�
�=� @�%�����{���a
���^;mVaT�J�
�+�-Q#�i��-�nM��Y���/.��%6 �����O�n��I.��KG�.�l��/f�����FE��z%�����\Qp����W�+������N�j}��:,P�� �jO�/e����E�uW��H���I1w��r��;�$)T!C"�2j��QK Y�6��NV���b ��h}��:�: �
���' �N<� ])CA�X������e(%g�/o�/��z� L����M�ff��c�}����x P(�
����
�������'���;�����~q�-� ]�yL�z��`�O��[�&��M)���/dlk~��� ��%�a��)*2�}�� AAA�,}�� ����AW�����E����~y��F�^p���X�����4�m�yWi�c#Oa|j���O��P������7����r��.�1F+Av��E ��E�
���Pa�0F�u�����BU��&Q��� +2 ����W����m�W��7: *{/ KZ[��4G��H�I/y������v'Q���O��qu�� �[����l`������2�P�@�%���u&���H��K�U�wd��%�6Va����6B�q���D�%q����� p���Z8�X�x
j)�zf�=e�� _�nfCP@,|�$��[���I� u� ���J��_�oD�z�T O� � ��������B�}�m{Z���D��Z�CU��
@��[v��64��J��j'��OjUe���k�
Q�,m��_[�b�.�8CG# +��m-a�u'k����C�R�n/����}�T �,�Y'O�L��3��� ]������_V�n�)Pt��xd���stG��IF�E%��g�] �D�/C*� <t$�����~Z�E� ;,��}u���x��_�j^x
Y�����"�b��Uk�}usK0]�D����>����OJ��u'o ����y X��|���7" d.�><��AA��+e��!R�����d
9e��_��u=Q��Of��1? K���Y��j�R������a��v���;L[j�SS����QQ����]��5��r��A7�u7>��0��Qb�s*7o?/������%�.Ac4��?M��O�z��f������h�pK �X5^�� ���b�}o���;��c�� �������(W�hbj+�>�#� ��-�� �����sx�BL���T���!5�!�B"��c��9�]r6�C�8��4�8�B�1\�7��F��mB�����
i��&���������7=��7�5��� P/� %ZI����~�gm�����`��e Z�_�vSnk� IDAT>!���H�� �����[&gV��"i^bJ8����D>!� � �� �2���d�%���.�� ����e�
K��B9��/.����P}��N=1�*�}R�^ \ �L ��L~����S���j��Z�&����M���}�]V��
��;��7� Tj��o� �ZQ�o�� KX ��a�O���S� ���(t�5��0��� YKD_!,�� \|U��ec _�Y �e��*:q�!��C�������fj��. ��k ��X�i<�$��
��(��yd�lH����B�l�Br6|?� /��^�
�����>��������d_��7�����S�.7�{�����S.����6.�*�+w �l]��mf���o��4����3~#��d�V��%��.6� p��'AA9K��%�v�UX��G��d���o��,3�
��e���cK6��SO��}R�n �M�/ ����%5y���U*�����N��\���~�����S#�F/}��om����s ���pvA�������O�rI;J.i�|p
R'��x�����?{��Z����w��i�P3N8_I�&�l����|� �� Z" �+ ��Zzn�I7I��7qA�|�0a g����� �z\Iuw��d$Fh�.������`g�n���$R��"Cx��p�t�5�Aw(������F���t� �7���?�������%�}�Rp�q�^����M�^�������6�zk��z?�� �� � � }
��A�o���>=G���]r&\0�YkeCE1 �y������%�/���u�|%3�� j��QG0� ��G65�<��;
��e g'��f:�\=<���������{�'����V�l�
�=d�&Z �(w���� ��Gy���� OlYz�� AT������gX��bw���|���T-
Z�a�9-�Z�3��l��X`LC���������[���gZ^nW��!`��wF�;����gC�_L=�;���:65!e��w\�`����js ��Ec��=�};�o���Lj{�� ��?z${�5��e��=��^����������$ ��A65��g�k��*M�� ��
+�r�e
M^y�~7��o��� N@AA� (� H z��a�� 1x����stj��mS-c��m����{>������{��[����d��bM� J
U�]J���S����u~~���UX�`�;^������ ��*�6 �#3�Od���2����AH���p��U�G�����]�N WAWB�V�Q��M8Sd��L����}��/�,j��!)�����!Y������7�L,�����h�2�?b6�@����������@ ��6�zxG_<�0]E7&�TR �H���R���W1��< �Bkn�_a8��R ���4�z�_Tm�u^�~��V���P&d =�t���7B�.C��������H+�_���y��pXRNhW ����v"� � H E���������d�%7l_mlXi�������jS2Z��D'=?n���N>UtB�w�X?�/��2c� �l���8����VN�t]�B�������M(w�E�����u ��x�#��zE�M������ ���!�)f��� ������_R�vS�Q3�OtS������� S���e"��9�]����1��f~�U`��� �������d��'��
&��rS?�����|A~D�QaWPp����>u�������L\A������>����^��dP�M;���L�T]r*
;
�l�id���(a��3_ �����[|2�?(�� w0�A+d<|&wAAA�*�)� zb�`����J!��� �k��MS���x`��u�6�0F�B���w,8�_�3�
��� � �; ���@,����W ��"l��L^�L�fB)dJ�}��r78��9Z V=kx��/ �9B� ^����� a����� ��d]CJ]��}���><t�������!V�R��|�N���������.�����h� �� 5~���?�g����Wb@a�(vg@C ��?r�<����c�������4����*���a�[
��&� ����(���}-��o�=v��q
A0����� Q��l%�'' .��@����� K�_r W�uZ(���X�����7����X��o
Z���f+2�o�Z�.�}K�[� � � g)X)� zzn������ �*�H7�OL���u<]R����f�(D��a��� � �� �/�x�w�3�����i�GW��y07}��a��m�hk�J�s%' �_�v���E &�uh�E�^SiW��N
!,K
E�I5}i3 �yq�9�v�[� "����,��<K �0K��D XX��}��V��]�3�����V����j1N�-& �,�m�����f ~�J���� ��1+��'�\� ������-JRBV,��7�� �f{ �}m�q�7h��R,, �*,=��/-��G����}�,�� Vs a�� �����) Kf�@���` 6� � ��E�c����e}g�j �j��q������VV����[&w��:�zpd�} �j
9��Y�
��gL�aw��/S�����\�u^��Q��GTn�?�l���fI}���T/ \�b������]x�����@����/�B4���>��*X���{sc��+hikK|�5�(�����l�O ���3������
�F}�Z��.�� #^���*SqT%�yH�f_��OL�-e�Z����&��7�-�����Y._�����J�� @����k;�D�{��$��^be_4���*�beU���5 �����7�%$�����t0��x����l��:���Q�� � � ?0��[��w�������wow0�x������w�T��m�z���N[�����C���r��� J[k��� "���w�K 0���GF\�s�
E���I��o���)��+����;�j���U;���3$R �Gf�:�v{i�� ��u+ �/��]r�;-Cw�J`� �%�B�j��2�D�D ���(� ��(\&3���LI�mW� @�� *������P��������� ���_X���E��[�5��(�����48��if�J2a9`#��, :"9�QD���M ��
�)s7�M�H��sf�=&� ��i4L< ���g?����.B
p H�=��_� ��������� �cb�.} �� �����P�>54J_���4u�s������#� � �wD�|���{�k��O�N9e�R�
�+�Qi����~�=���w�f�"�� s-Z�S`��x�k1�R����I_�����`"l�N�9����(NrY��\����2��������Z������K��Z�#���}}���P���(���xd��C����.�b�.��4]FR�-��p0?Z� �����wi( 0��z��jN/���>�n)!��eJ�DV�B��Y�P3��t0&|j�k�\?*���{�"��cQn�Zt�Ix�*������� ��r�c���N �
��Na��T��]�%]`�B�]���_U�h�4�� I� ��~n���J��bu����2 &�-q������sv��_�nV���� �VCAA�o���h������� ��"�� GI f���&��W �e������X����-�\*�W� ��1yi*G[M�
BPa2�� ����ci�n�p����fD5���s�=�����>J���7 ��� is&����._�0�B����]��� %���y0/�m�rr��]�~��g�L��1 �[����;N^��Cw?J���)��N� � �1cs��O�&�����tA�.���A���1�W��K�*>���y��o�Qm_"ck��NuL�� �mN��g�5=�� ]�v|�K��������|������9Z���}fRj��� � ��.}���/����7?9��e���t�3�������� �aw- t2[ �&���[� �������!R%����Y���qF� H`��`������m(@���V��b�?5� �b���� ���kG���+`��1
"�P��[����L���V� 3����!�9C]��r{X ��)���]�����)�[Y��-{c}�)��( ���t��'h���
�C����� &sa?U���[�,��qI��P��ko��5~f�3��",��������.{1� YBi�������rO���m��}�k ��D������� 2�]r��<�[������l�t��'=u�-�?������nx��}�<�_����SAA1�7 �1A��%�/����KoZZ��\kRoY��2|�����g�R5�Z�
@l!���k�] g'�c�D�I$�c8��c�_��_^���Q_a��R4=���v[�����EQ�J�!�#3MA^w�:�fD����G�)���qqn�Cx�_�H��7�5�*��t�9Z�TrB����Hj�v��aK����jU���� ��@Rx�/3����������M����$��Y�/��������������K�E �!���+/ @X�/� � _;nf�Wfn � ���u��)G�q(�D��0
:��zu�G6����_
��!������!��/�ke$�6y����;����;��+���U3����?9�,y���wn�W�> TD�> �:�~����I1Db#� � ��e�L�UX���i���y������i� �)5Ft)N����5b�Mw��_S��z���������;6�����V X_�<�@n�OF5�(�e���������c�������
���� �]R~�V��&CD�z'�� 0jw�`t��Gf^���Bd� ��u��Wg�r�Y�%��np&���t���!'l\��E�F X_ � 2%
�H5I"e�3�x��V���p~����T �����Q����y�e�^�ZT��(���-��}� 'p�82��s�b�tB�n��q@7�e���G;���'� ���� �FY�D>\�H~3#����6� AAA~T�(� g������ �s ���Pu+5��MT�j�
�m�a�B�����dm�������& ��$����9������
�AH�[v|���f ������o��T�L�BT�F�j�$��5����!����(w�v�C��QdJd
�"����^\CE`�M�e�,Zz��;��� +3����JB~��9ZW���|#7�M����A�fFi�~�lp�!P��tD{�0�)��A�� S�P#m]��u��F9��X�+����p���79���� �I:��e���n���u[5����\>�fHc �F� U�'��v�2p������ � ��G��SA� � Jg�FB�� ?���1W�r�s�I:c�<[r���������u���U��F�.�lQ��Y� ������:_�z?��Q��Ovf@g���q�
��Z��k ^���y��6�������&��;rO�z4������h]Q\�U�B=2��"k}C�Qx�=��@�6}iCC2 X�_j�"���_<��Md3�Q�2�RD�1���D*n( ����+��:o��:5\)���/��p������,���Qw�B}��X�����SZ���2P��Fy�TV�(
7��������?�G(�p��'Sp�@@��({���L�_�>~�MLa)�4��X�p�ja��{M�
gc�8�����4��� ` 6� � ��E���UXJ � �
K�_�n ��Y�`��5�{��g�To����7��"F~ZptU���Imb����J�6��+��tU�l����H�I�[x��F� �(,����_� �|^Y�E/e�y���:����}�bo����`m���D�z�$���p� �������]2�s��WN�K�Em�^�
��]1Oy� ^ �YV�����L�K�����������z�����c< CL�>
@�nQ�Bz���e��Z��1��l�ef�����rr�r
B��,�� s�?s� NN2 �u�,�Ndl8�>���g���\u~�z�*�2��s�d��z{]�������Qj�� � � ?f���p_�oZ4#? ~�g���vR�c����Dq�����r��e�Zx ���������,J���]�������6�O��>���D��b�%��K3����7d�D�3i��Wm�Xh��| l�����_�u� ?��
_u��]Ts�]��~���K l �gaC�^���PNX �$Fah�%��a���]��D� _�io�T8�,���(/\ ����d�B��#�N��o�z}be�i�>�J7��s�)�2���Y��~<����{P;��i�����Wb�/�*��v���!� 1��L~��� c���SN_�>��V� v�Xr�������������� � � HL���=V� ���{��C�Ls�>6��BJ!�"�(�� � ��`�:h793�������������F'@����d��@ ����Wd��O2��4��e������� `�u�y| q���� ��ON�� .�lY9����9�\�]������8{f���2be�� [�%q
+1��1�v��xd���yj�J�<�)����4����eHM����)2 @N[�!���.��XY����N���6��� Z�����R�Im������nO�����" ����$-����� ���eFK*�OSR�Y�_��[�u� (BaZ� � � g'(� �������� ����)�[,\��
(�,��7� ?�>JE�G� �@m����d�H@�W:�.���������`{Y�#V �50�w�Vm�u^=�1�Uwj[� ���MM���(@�%�����<sCH� v�h���&���?��#� j� e����|k�v ��:Q�3�]i��&�Y�@���H
�� p���a �7�����B��=}����[�BA����=D���E��y��
�� /@Kj�"x�Qa�U� $O�n�n �����'f ���E���@�(� � ���`��|��Z#F�� �H�6������,/QEV�����vmX]Q�o�K�� 0���p�����Na�}3�6����_���
�G�lQ#e��}T�W��Vm ��� ]u�%�V\�8r��|A~��HG�%��!5��Z�P����AB���<k]C
�P$S�%�3�������� �� � �+K�6n�A�U�NM�n2 �����a?���eb�k��F���,V ��MUdT<bp����;g�������o�1���0� � � g (� ��U�e����^�/��U�e�������P�B������(�' �.C_� 9���� �C�3�Qd ��p�v��a?�6����-j$l�n�K!��w�����-��0z�\�> ||��7�cQ��Vp�7�( =q�e��:Nb�����' ��JN���������J���'Y/�h�`B���_zB0(J}�o�HE�b������-&�/.�N�h�<�2a\X�ad������lE�}��������������B}~���`�acd���%L��%�Dm/�O�� �!�����R
��h� � ����E���F�tB�_D���e*�B�R�e�7��)��_S��� k��>E�<��c�� _��X��S�<ZT�b�]�"r� ��p�te��� ?.qQ5f�k��^���R�5�q�U��X3��������,�PB�N+�� �����u
)F�`Ph��;��?4���=D)�k�o������^�(��{� T�A���TQ
�� �� �G��b�t��x������������5�_�K�� �������q�����Q��G'N���wUg�������=J�m�E��j�M/���]�������b�� ��U0� � �#�o>�c� ��6��#o����@A�T��(R���i��awo���?�� P����^]QvtM{�;���mXUw� �������yH����g N�kn�G7-��($��&������l 0�+6�@\�[&+���L� ��Sr��%5��Z��=����x������{�
C����ec�O�Z��t����R���+��t�J���H~i������5��O��� \f����-�!��� ~{���� ����7l��� �� `:��+K
U vw�EE�[
f�O���#���w#�?����N� ���C���,��t+{�PTdAA��4�E�����2P�~����������L� IDAT������T6���F�ak��tJ�}��OrG�a-� ?ZstA
g �_�|��s$�i�?�-��wx�����������r�P�\�+6��������K3��iO?O/�����!���$����e5������D�*����=P*.??�����Aa9a�
G������ ��'Y�=l�P�����u X��%;M��~w�2 c`�+ 8#<��r��O�q�)�{ K�N �c �\? �x��w�� ����I{���� Wg�K�]mb��
�M���x���{:��U0� � �� �R�W�./^�}��������M���������,�_�mQKK�j�m����"!�C�F&�BR��~���� ?����k�o �p�{#S?������KT1�e���?' y�;�����\�O���FO]��'-�}��-�E_�R`
<[|�/3rP
{���1���BHwF����Y}sR������ a�~�HlC��� ��z?�������#��� �53��%�y�k�'V��Jk����� � ����~� ����F�V�/�k������jK{j�\���]PZ5}ap��U�%nt���^J55�DN55]��b��0�}������-��� �?u7��[�lAA�G���������^b��L��]�(� �� �k<M�2M������� k����)P�M�c]�WEa)/�W�3�������%� �-6��$�ey�9�-�Js�{yguEYg���O� ]HZ����%��'m ��j�� <,�W�� ��' �m�< !��0�`r����� 1L�^$B(���q���������/�k7�?�y�
~}L�A� ��D�W ��~����>�j���I�0�@*���F�iS-�U���L�e������g���l�ii��N��U"}�T��Cp{���[[f�����-���1� � � ?B/^ ~����e�����"�IH�Y;�a����b���0KKF�40����j�[D��_Z���Y�����kx�4���?{f��/���x)�Jc�M �����P����uJL�`��\��������� c8�/ L,0 cn*�g�C��p�te�����C��PaK4���Y�f�e����3���~��.�G_z?���x{��!m���l\N�����C�+�0#S��Y���>�����;��S�UX���i���y������i� ��EG�� R_L
hg��~wf7� � �gAO9mx�d+�-��M�RC����'+sV�4�04gUeY���G�&�������( �{|�O��:\A~���u
)w ��aw={n���E�%��s%' �_��1G�����}�S������-�r���"Nb�:_\07!�����c�n��m��C��~l]��J�m�'
�[)h�E}�[1<f��xm9����8k�|L)|������.� )&���~ �.�B8G&��R� X2s�
����5�� ��Qj���4��>O�~������l�}�.�Cw�����;�rbTNM������6��P`vA���S�-�R��f�����}�15���<�0& ��t�1�t�EAA�P�A�3��N�������������5�{o�d��YQ&���i�����+���+��E�����s�>��J�5�P�hQ(|�J�����V5mL��)m��x�k-�8�0}�q���� ���}]�qR������5���!�?Ta�����v�_� .�3p����u0��) �i� �y�I����1}���;��T[��f�<�����9{&�����1���A���Q ���u��������S�C��R�Kc&d N����`0��]r��<�/��/��OR�M����b�U��jw����mO[ �Ze6�F������ X-�AA��`������F{��Y�� /����� ����D�T�L��npF,u:��3 l�TmY�q�c�� �}���������k�R�*C`RJ�V����� S��� ��=j�������/�.����vtc� �X ����.�}���Sxj�����9��N� (�4� ��)�n�[�njQ��CY�*����: � CjX��w����O&��'I�f�hM�+����;>���1�������>-e!/\zW��y��U���g���t�������N�pvk�]�����`
��N�� v�;�R #� � �7�J�����tU[R��1�C;9 1��:� X��J>��k���O<�0�� ��1��o�����>�Y��j �����q�S91����G p]b���V{+j!B�5�q���������O���j �8��� 8���JcJ���
a!d����������M����Q��a�E�2�
3)�a���,��L� � �b�6V=e��|���~�(K�����'�m�he��V&��:� F�� 8;i��^)q�u�����9ewo�\�
����s-n)���K�>���W�pS�� AAA�>(� @��1��_�y��y���;l��]�� ����� �_������}�$x -���2�{�$6�U>�q���� ]����?��|��f�H=o���0�[!��x����i���E5VN��ac��y-���a�_�f�%a�E��� ��4<�P��A���u� M� �e�( "��I��k����7�������[��%))�xd[<��-����%��t����f @(!@���L�����7 �%�* K��_��f[ qiw�2��N�Y����~y��R`�m�7��L�(��G<�~%Y`f���BCAA�� E���"� �����{
(�]]���Mq�f� ����?���ka� 0��C�UEF��*��f.��H^���. Vh��eNZ:jP)�* �y�"oL�E7
��fN H���67 �C����6��:��u\� /����# �j�yB����I���qF� ��������;W,92"�����(T���r�t��D9s
e��x����K�~b��}E�c�P�@)���g��rz����~��1�{�C�%������`��zy��
�����CH���CM� |���[]{� AAA�>�2� Fe�����7-X���^��t���Q� �5�����c������
u)I.�U��H��>�I��5�n����&a���Wm�<����Q[����5u)��w8�bC���� �*�y� ��4�D����w&��H
� �H�U
g�a �r�tG� ������4��A�BH/<z �����_4m��5��<Z��C{�Y����F.3LQ�E&F��������*2�=��=��vo @�N����Q_�7l)�Nm�Y�_���FM
����D6V�����>YM^�)��)fQj��I�w�=��"� � H�E���B��|��.9�)�JKz�sv
#� @\[�Q������ ��o���{$��k���H
�s������A�Q�]��K)�(DjpC I~��K?�|w����S�_�\�Ln�� k� &P�1 RXAVd��R�����sw��=S\mc���/J�g�����A *K��y�T�%�a�K!>�M;�i�����9�|� �*V��� �$Gz^+q���eJ�����m ������x�,�Q�8��i��(�K `���&��6x�T�� |1AA9K��%��FB������e���p�R�,�}�> �l�������lA����T
8;)>�f*G �,�g�~9zI&g��2z��!BhC���1�(�o��i��/�M����[U�����9�x5 �T �� �a�� OmJww�!�G�lF������]�5 ��Sd p�� �9����o�b� b�4Q>,�Zvf%��a����3M%E�"�Q� ����� w��s�g�� �D�TE"P}����`cx�*a�zZQ��/���@�;
��L `������S�AA��� ���|3z���1?��9��5�St��a���Y\�����5!�f�3y���_�V8����i���g%�[n��b�����t�����:�r-�g�Ts� c F`D���-X��������������M�����M4�k;S'�G������Ze� *���{p�3l���!Yu����� �c[��B��� $� \kh HL|�Ki�y&�[��7����� �52� $� 54�Jrz�.��&�<��1��]L��������#�o^��P uA�\�?f��`vxZ� U FMYH�3�+�7�r�Q������� � � }e���*���rwT���y��\��1S��q��� J�T���k�G4?{� l����)�
�&����-u����M# ��{�������D)�e,9��z��iw|VX�������� � ��A�~Ua��}�kp� �$o�9��w=�-��5��"ds��?��[Tg� r�$����\U���8V6w�PY��������e����G^��Tuu���d�B�/��� �d�Vw4I�/
l��s>GC� �@S��[�K�,ud� ���w/��M�]mb��
3K�.��t�/��kjk�=r���#.�e�bs��e��r�F&f ����TlAA�,�� ���w !x� ��e����C�"V�5�Hj�A�� � )�_��ay�� f�]���$
��w��4^ �w��Y�g�=�(4���W�sR]��o����; ������\;�K(L�y�~��(}>����O�kU����p���� OU�{� ����� �E�_NC�Ph�BU^�� ��f8\����$
A1x����h91�z�>�A���f������kL�tY 9E������������7v�����f� ��������~�1�g��Q��� �d ������=`�g�Xh��|`��R�}����^AA��@�|��J�-�
K)��|����;��(����?s�� �� H��PnT��� (�(j=h���z�X��z�U���X��j+���%j QnH��dsg�����������lmkc}?�qv�33�����y�~�^��g���#�����.�� N� 07���!��'K�=����-_��^|c���{�� ~���)-�I=B�1�U���)�P$l���-9Y�_�G,�~z�9v ����l����=)T��� ���/LP��S[��
��B�i�d�^{9 ��_��� e�$4�1@E1Q��_f>hJ�7y��B��f�R 6����LZ:�y���;��b�J��l^��"��� �EQ��d� X
�l �W}��U���b��E��,�'�d>����C��4E i�d]0fy��j$����R����<A)X����.9P�: ��V�-W=��AAA�^�2�
(/U^
�����#� BvG�U�`����������T�}�zg������xT-N��:q.������7��z�
�A�j[��<w,�p��t? 8J �����@ ~h{�� �*j1�}S�V�|��}�5U(���DW�����cc�w\Y����i[M�J����W$��:���;�_9�dH���� W�GE���������DE1r���WX;N��h������I��|���Z�%y�\Kk:f> �M=iJ�a�0���(2#�K?i���
u'5*I���l�]���e���SU
�>���_#FG�O� ���4d�?[$JP
V���\������w�"}�� � �|�@QA�G � �8�< ^��O v���'^ �G^��re�� )�.]���jm��s�\L� ��M��N� �"������ �����M�g�1V �G� �pZ���[�}[v��!�Q{:&��p��EN{�s�X����G���x�gS��W_���k���-Ju�2���2I�\�����(4ZC�@b�� cO�[�2 3�(��� 8��U������Z��f�w��P�>R�0|i P*y(e�� 'm��w� ��"S!������'M"@J ����=���K�z��g�������g��L�$b��l ��YT�t��)�H;ND�,�B�X�-e���$3�� J��wk�_ @E�o�2eAA��)(� H7��%j� �<��G����ig���������������g�,)6:���QR�jm�^�Qi� ,�*��/x�w���n��J� ��-Gv[���e nM
h0.�v����e�uwTx�L�m6� gWB� ���#9&����F��-�5����u�Z�D�;_���1|�LBi|��8�>R���n�
2����3*,��~P �.��@ +��Po>q��!�����_asr�P-��q�S ���<�b H�Wh{Z}b9�}
�'t��A�0��W ��t� �:�6���- �'���iU6`
T��}�]�+�-�O$�|#� �[*2� AI����-W� � ��E X]�[3�Y1z��� !�s��l �����|xE�%�4':� �!�T �@%O��.X;�
" �2#�q
����T�Q/�uH]�� ��:�E�U�R��w��;������@z���K~<�d�:R{������F� ��� �#��hi��Z�'�1�?/I��r�z���{Nf�/� �t!����8[��l{���
�T�J�Aez�*�)M
�'}iczj=���B-�����bc9&V�BXB�*67KS��: !�����/<��;!��� �e��b��
�2�?��� ���S��d������z=�w������X��@�%�Ydt���\!� � �� e:#�# p���h�z��@�=��)�%�4����+���D�W�����#QB)x������ SR\�,*��K�!tA����q�� ��]�T�i���9�������XB'�������w���LAYI�xA{�MU���be�e ��%�U������Q������
�g�������m��oPvI
�1�W��b���N�4��sbXZu���U�*IS5B��0��C���I��]~ @c ���l5\\�dJ��vh����dt� X��P��;�W���q�1S�]��� 3� �������V��]8X#� � ��+�M��q�����k ����>�8Q�3|�Fw ]* |���J$&k�����f�?b������z�V,��8B� Y���^98���I �~������.w�~;,K .�m�7y������2 q��)+�������;���k/��/���C����kc�:�x�
�@Qh0$�:�-�Y�-Z2e�3��a�B�]�_6����>Og�Y"��vB��,S�9���e��|�;l�0���p��)2 C����P�������5s�P����_��S�� CHL�+Li�i��E�bQr��^c��p���E�R�ru��b� �������]�D��.���VK~="��)��� � R������s�~)�Wj�?� � ��+e$Y��5-�Z�� ��}h-���h�YgU^�/����������H��7w J�E.��XC�����������}��q��7������\�9����8mK������������%��4a�s�<�b��)R�9��Eu(z�����S��~��� ��d`��)}v0L���%���Q 0*y�~�vV��2���!D�<�<���3���t b��T���K�
O4�K 5H�I�P��_i W Ms��';������o���
�G
6��m������m�� ���7�F�T P� u��$*���()Wc�AA�����������?��� ��K��lH�w� �c�e W5�R�>4�����g���� ���)hc:��4?p
C,����}��!
�'tTi�e ��}d�(@`�����%+Xg��P���\�_���G m
)��?�/��������Cg?.�7�� ����r�a��1��OE����P�$0�E�@0�}�"�>��`�����g�M`xx(J����*���:��rX&��G��S�=�������fe���c'r.�*+W�����Mz&n�� P� vQ�LM IT�b`{M���:B2��el�|"n&I��� ��}BAA����� +e��������&�����@���I9W y�[���G,ys��^&����#�GWn)���(L���7���5�&���-� �cy��� �nMU�@��@���H�� =��;��=���q1 j>:��%o�c��{k����������@&@���j�rw�@@ 8V
��[- �������.��� IDAT��W��� �>�s� ��
:rG������9(�/%v�Q�;q^y]E���I���/����
���+�xw.*? �C� ���~�H��� ��} �����u�RZ�^� F#!� � Hb��A`A�����s�� ����;���}U�m��3�gW�{p�����eb
�� �S��n���\&��#<<��#C��8"<C�8"� �^���F��4 ��K� ��nm�"���|�01���K��yM�8d�zE 8P Ln ?dq�0j��� �,t�����F��;@0�Ey��� �G�=>ee~R?��� �z���\)�?�$�S ��#��G�m��7������ }����5�\ p����8#i2Cb�<!��z�+{
�RZ�^���H� � �X)� �%��1�GH��>#�}��]Y~t�fFS��h :b�&WQ������&^W��b�e�O��* =f2�� "J���! d_��k��zCE x��^��"����Q\���1���[be����'��� )�����!�;�rT��B����xlEY�4���c3��
aY( �����������<�����l�tG���r�~Fmhr� �O���/�M����f���`�j����@
�,p�Z�()�� �E�� � � ]���� ���.�*�Ed�������$B��D�)@�d�j� �
�e��c^�Q������8�^�����ySWT�-���J`�(Ju ���%�@�9o�<Z� ���� BY��x�z�QEI�xo�@��:�����]8�z~�� 0����;9 ����' �F��;���`�sq�����z�u; lv- @���������Gd��} �jB���0���7�i��
����$�NPET�|��_��*�9k���}(F#!� � H@QAz
a��k���k2n�:��CE������S��;k]�9�X[L�P"���� I3������Q� < be�D���[ ��d��� �T�� �A}�1���H( \^.?�l|T�!�KG�3}k��~f�O~6�n�C� �������O Vw�UL�
a���7�'OvN;�1i�#7y�����
�� -�?��)"0 ��M%���6��
at6%�G,'|L�QKZ����� �2�=Q����lR�p���,'�1�T��;F}�����\�->y���o_\� � � ���(� =EH�
�.��8 �KS
�� }�_=� UYa���[��� � >��^���_���|A�-�w,����c�O�`�_�JT���K-��D�m8&
��� ���b���_�
�����/�6G�/�� �y�����F{`�R� �PY+� �/)��[6��PEa� �
�:"<<w�#���@���B�CO�����r ��~����X�Wk�l���s"�|�/�m��u����-�<��9{���mL��csPB� � ��������,'{����&�_s��j�;9_xbJ���Y��\[�C�YR����uAA�=e����iNJC �Q����B�5�c* ���>U~q�������: 0��� W�N���v�}p � ���e���v�>o���G,����R�� r}
���9�D����������\�[:qp�!0�N:�(�U��-�������i����ii��f~�_������a�$>o��"��������5q(T��x�3�i��q:���i^`��_�*�,8a�o���C���I���V��b�oW�����o��5w���v��E�r��mV��T��ML?R[���g���� ���&� ]�`s�"��X��x+_-�IE-��I�� � �� AQA,���/������|mYE5=�|5���A,��|4W~ �f���/�c�8gx�-���|W�kg�k��q�#=.��G
��N]"�!����x��S��$rL�!��R
AOJ��� � t�� �F����������� ~b�[=��AKr��Q/&���/�v^��6�`w+�v+�����k�g�e�� �m�� @in%N�e���i�<��p[�W�OzpTx�����p���@�8��r��T�.�?��+�� �)��K�6���y�W>cgB �ak W� �����j�� 9J�d�������u0��|[L���n�� � � �2bF ~h{h �� �=�$�wL3�TMO�TGF
!%�d�,1���;������o-�i������PK�g�9%?j�yv�Vx����u��u��;INK���F�a*��K��.��=��W8g�v7 (r����n2�a����7 0�E !��v~���Y�M�P�����}�� 3�#��u Y ��~�(�( /[�Gu�Js+q9- ��������%��a�b���YS�gj��h�zBaA��1�s�oT��Q dT~�s������%�R],x[mA� ��@h|�R��wkmh�a�dR��n�� � ���� � Y��B���l�_(>�Q����D����y�� ���Y��T��k�i�[m��Z���aX��U�g����8w?�����x��� ��]���f�a�;Fe�m�8};�i�b������� V�8+�h���v# ��`S'�����E�>�j/�C�3���"�M� mu "�c���-�����R�" ��VdD��3�H��#���s��G������������
�J $p��@���C��m�? e Rf&��pL2# i|�a� 4%���8X��'7�QR�5�*wa$kn���\]��&�j_Gc���m�� � � �
�2G�x�,��8QD�q"BXc �lP��#���
��qE����wLM�mO���M�>�| R������}�������7��e� ��}� �' O\���#UaQQ���C@��� |��,D)� ��)�W�"��p\<�Q���JF{`P(m�2iz���,������!��,����3����"_ �|} x�� ��CA�P��1 PY� =&�d-�JCe.tr�o�L� �y��h0�B�����b]=��RD�K���������9g����RT�^�
�"f���\��R-��� 0Y� H�� � � �
�2��]/�cu
�cL��6E�k�,�=�rL�P���C!�����j���\����YC���H�%��+ w�j� �<����^�5��7��|%���`(��J���\����"2�����H�t������u@v�2�Lk9�}����esY�#=��/�n� -��'+:��M�����~�`��!�A�*4"V�B4 R���w�B�;��~{�k���n������_�7���[v���Pc@^�e� |�!%M�����~nX��1��x���� W���tq�]`���xbJF���� � �E� A:���{��sA���N�}���t)Kq��P @�+�-�y ��K-�: �����:��{��;S����er$����P%�e�l��'l��]
�m\9���f��H���m�j_��-�m;���Q���GOm/�� ��h����Z)�#��8>��J�?�m���F$�]��UY���@ �(����nb��'��+=�IM5���`p��8���*�.���Q��!�,�Vf�'F�6�
��5�Zm���2��S�6G�d�6G�^:u���Ys��1�G]��l�
����|s��[�����a2�A+AA������������b
��J��}H�q��c����F;����o�r��:y��3�^ �m�U��c�\�H��d�Z �'$|�}�����.2�����*��5�����]��HS+([�E�rkZ�8b z���QzV|������N�T?�| ��������Tvur��gL=���=){ �Q���h)��6-��# ��}y{�A<V9y� �8g�{���I��m����De�0 _����S���P�uZ�2��Y5�+�F���>��a�JxB��C!��C�@����y�� �������j���mRt_��0�=����^Z�U�����W�� �A��
p=�9 �JAA� �{��ZiE��e�RA`�� ��[�\L��?N�Y�����W�N��Vq�
^�E���<u�W����i� &�T^_���@A
(Zo�gw��q�����WE�� @������ �')
.�����N4�t�9/�-IS$�������/�>4���?�6~�a��: 3��>��u��fJ%J|lI��E����� ��o" 8$�
]N�o������X��P�(�O1�<�pD u&w����q�q/48.�G��w���*6�� xX��a ���*_Ox�<����-����i>�� �1!��z�� 0�@�nx�jd�V2� � �C��t !x���9������5��� �"��<��r�e��j4���Ly�@�Q�V�� g�v��vD)�jS�>4��� W -��
�� �%�(���t��N�|���WZ��^u?��D pL o�C��w�����:g���\I����D�4��Ug��&v��b��MB^[]�f��,{\ �"� �,�? ��4V&El\�O����#�?���q�O���, <,�y�K���.�#��^�)D� �5$����F��0��� ���Y�,�u 6/7�
��a��k� ���P�m�`�k�>��g� D] ���m�\i��$b�-���1���e�I*zL`��Y�
����� �� L|��x�k��I�p���4G�+67��z��K����s� � �{�����qN�����!��zM�r���,� G�f1R���]�.Z{ �4��y��cl�(�CR��1�1VI�����t���]��\����r�
/GNh�K�iS�]3
|�E] ���E���;���.cKa#^ �gW�}p���H;����� a`o�>L�S�" ����/-��u�w�B�<N�6�d
g�;�%$#�gt�;l�^<�4�W��
�� ����*��$��..��{��l H S���TmA �0��@
����U� ����d l�
�=�o
)w6�M��Z���~���R���9�
'���u������r��;^�dJG']�u�,�~X���� 5{�qOE����yq�N�+w�U��&�^�L^1z�:OAA�%��W{��A�n(+O�
gT��k�;Q� D��#�����l�
�,��������+����8%�bG� R������j���{����k�N �I�6���C���8A�����K��DN ��H~���� t��M�\��#m������g��D"�+m $�� ��(;r� j�����������;N��PST� ���6dP����)�����T������lk� ��K-7��P$O}��bc�,[6%Y�a�����6�suyTx�<�b�@!x��_�k[��e}�!������� �K toV;��/��Z�e{W� ����o�����8�������2�Z�����h�z����k�_�� �����;CAA� �2�����[Ud@�?NP�!�w� Qz���R�v�� b����B�
�� @�9���C�<�������e'����N+w(�s\^���� �1�p�<���� �9��7��^E���o~����L�,\Q���D��m�\
7��(g<z�������[�2>h�m��D�8���D
�(1\����t�=��& �f�+=P�����7��b;c0_Zb���� H��@�^�PS[ � ���O�ywzf_M Q j~ ��-�z�������aO�����?�������f��K�[����k/�L����/ 8�d���;��'��Z��j=���UYG���7g� }G��E�@H���� � � ?xP�A�o��?�-bt����t�#k�[�J~%#scp[���
h�t�82k/z�"�W� ���=���p���/�x�� ����.��� ;$��K0� )���mu�������
� �\C����)��j���e"(tV�� ����(� ��(�� cK�� s�:�� [# 214�(�z�-�L�����E !�F"�(��J�Z�����|[�{�y�L��� q|�=1�x�B$B
J@����5Y:m��
H`��j�_��WUs>f�o�# @V�XO(,(�v]6������<�
m��+�< ��2Q+x)r� �h
��n6M:}���|ar~B3 U�����j�<�2� � bI/x�A�^����������������������6�1:b0"��M�e��h������]�� � KH��~��_����MH�2�����;���;/ �(2 j-r+�����V �" �uE PE� �?"M������1���������� ]��� �
�i>[���q���>:������ ����Y?���{�"=#��%���
���p;B�� �� �i������|p��� �gt (�\o)rm��_&� ���h��XYE���f�M_m�������#���I���`�
o%vv�Y��]���av��� �L�'9��I�G^��m���%%�P}�X�"��GI�)�� � ���J��YF
x������/�;�I�����r[]
<�����_���� �����������z�4� bG�| �m��M�=�o�gW�6��.���"u>{�� $F�� p��_0��u�'��w��Oi�i2����b:=5�� v��~��V�Q �c�J��W2�|C��|���`pk@�hh��]\Z���
���Y{�������<�\n_ZBQ
� �Z7�0�-�K6���/�_�r�[��[��{2�c M���Ns�H�a@��R� ���|aA�(J����{�k�K#�mk�������+uQ& �G^p���Z�=�i"�>E AAA=��|���;-�������?��N�U��� ����Jj�P��0q# �I�� m�a���m���
( P4n�����{�(0|R�o�����~�xz��w�����UP�k(�
)c�h�[Xa8�a�3x�8�i����� ��(��@n@��E }��{� ���Eb�r �A<�&J��Ot=]$.���f�%DD�2�"�y�� "f �(rm]�Q� ����;.�94������`�j��%�4�����mrD��z�+�[ �W@��/�p�����]P:j�������1�s��G@����KF6���&���v-�g����U��DD��|��f�����vC�<� � �������X)� �`i�Q�����3> _����m���gA� ��$R]��B~���-7��
�����3|��,��� 6�O?^oU���
k��k��&��W< ���/�v�^���3��:��ra3��kL�L44���eK2��L| M:�=�@ (W��lf#[1��XY��(�f ���@��\P�m��H ����rS3[�������(��j�!lzzp��� �0\A����3w�����br������7�K+�K���w �</�������hj5a `fJ�.8��+ 7�� �-"�fdxF�{�k�}�q�uX���\]f_Q�~MS�����ka�?�u9������Z�)/�� Y��i]��,���AA��Lo�z�SNC�3��XG\�NfC��K� �#�� ��Ft������Jm �����1N�����,�' a���I�l �p1����<���{��cq�&L��� %���,
j�\}���Dh;Z��(4���^��K�.�M�q� ��"- I�+����G�;���~k�!2�X 4��:c�Y��*�Ph���������K �*
�e���^J��~
�7 �� ���� \�/}v���WO�VnZ9���)��K���� Z��.��/9��y2�z�$o�ki��R ��t|��z*x@�{s�Z r��*�s�����j�K� ��`�N�y������n�eAA���;_���A�A�@�>jf����H����_D�.�*s�]���ry������"+��5h��O,l��"zE �p�w�"[������� � �"�\�������R�w��sQ�����"X�Zj�jF��(zEF���i� �G���6d�G,���a�F��� ��m;{�Jq�0�s�# ���eL|��?��t]kU���V������4�y�����! ���H� �0��"�
��}�5�� z�k� I:�~ +�\�tk�r��b�rjR���$l���
|vv�=���)%������G �2.y�%�7�� ���ufE ����� � Sh ���ul� Y~ � �[�FAA�[��K�
�jM�=[q�����+��}FF IDAT���e+�+����|fu��,Q���%!�� *�#����_�z���C���|�h�T�{�_��~5�,�N���� ���\��+%Je&����s�������"[�!e R � Z�?���p��2����S�*7�wQ������Z��g�R����6+]*��&�'�����lq{?� B(�T��2����Q[��$���~<=~�Q�W����~W�
qb���I�� R�8���|VU�}W�D ��Y� @����e�Z*2$�����G�)v�-l�t�� #�
Y���]
j���e��l&AAA ��zB��qBz�o.P�4����6���9p��o�y����N��z��=��.�����5)��D?Yt���=������zgD�r�&���P7�hv�#�K$�AT2�M���L��F2����g9��;�<Y6�s�;�^y����`� $ 8���qy��!�@��Q���C@��.�����< ���> 5"�e��@���� �����#�G�����[� �\M��g�V�oQ�L�y�
��/�'?�������b�|^����������(������k�$xP�~�.([��2� � ����j������� �_'X����z�� ��{CP���ES|>U��AN���������%E���������>�.��.�{k���j�Qr������?�q�3����T��q�<�&��|&��� � ����=� �n��P(�"�m��� TJ!A,���@��A��&n
��\l��� (��k#�����������Z�e�F�d=Q�K�G/�����[�or-�� ������d?mg�d��"�x����I���z�����)��#n<3�p�������'���aI ��W !�j#��q��$��/�?�= �����v���..AAA����W{��A�o�o�������**�����mB�Bx�{�}�G�YI����(��5E��W��a�x����;�� 83k��!���ex����YO���)JNm�9��/) ^l�O�\D��U�|���}7�+�*�
Y�% ���(�b�� {�*n"� ! K��9 �K�����I�\��uL���� ���v�-������];�Lua�s���c�����WREB�1)2 4JWN��- � ��{��t���H�B��
e�#Y/����� �a��}���9yk]�Rx�����I�oDr�����d��4�@X{� ���;�V�m�NK�|���� AAA��AO��� ��61rC+�Ns}���N)CXk9��B����A&�����c !,� �>G6�|�����k7�{k�U���FX�h�&����^o������jT�%�z���������f'Zu
pB�m\��#VV�s�-`��G�Dw���
�YJ�A�
�`�u��7����� ^�C�� B�����n�0��Ru3�GKH$b��[u�B�3��,Ii� V& �kX3(_z4;�O<\!�Q��]m��=}^=�' ���
T������;��d+� M���.����*e�����AAAzX)� ��Y����Ci�A��A\~k�� b���� ��L[5�,�'�S&=��0�5���f�*>=��3�����xW�����_�&"��
v?�����
U0��@K�~�������l����2�����w�?O�s�� �Eg9D���<��C4�. ���<��� ��R��5
�OxcG��+S��P��f��W [ �u���}������Y�C��C�9Q�R���������_�~6 � �P��z����9�+�?����/��9!� � �e�[� ��/-1�� ��{���������H{I��(�y k7��(b�*� ���[G�U��������U� #b���]3@0U�h�
>� ����0qZ�a�����8����iO�� ��kP�s�i�i�<�D�����j?D"&���'�0� � n 5)>�n�(�!@��2P���PY�pT!�Y�~���#������p+S����s5��k>>��C���d;���Y�Iw���
�%�Xk|�
p������-H��2?����IR?����ZN� � � z�} A�����pD�.��7[*7�\ }:TQ�����_
p��w��t��,J�ku~����G����W�8����
%���������%
4R��~�i���w��s����\�Y��6&_]P]c/��>��"�
�����R�=�q�Wb��P�s�X�L9��<�zy�B�LH�zMV�"5 �(��x��w���������#�*Z� Ts��y��ZTv��� 4�g��8aEx��r[����oe���q�!,�ik �4��%�
���/��?��yI��Iw�q���~�?���M��������M4e*�L*���� j����SV��1�� � �-��|���;-��� C��]�s~D�v���ogC+�+�� ���MA��e��d*���G? ����Q��Ggfv���m;~��a���s��V�i��(��[
3(� [�>d����q������K�5
����k�0���Z��9{�7�����U��[���!d�'�a�"s��#�8 �s'n��Do�$��. ��q ��p��LF���{R(C)�Lg���DtMFz$b_�uJ��� �0��@
���
^d�@v�&��^{��3:�a��UmQ��r]��2�>����k-��g4��q� y����g���tL��+ ���4�3�f�1����6AAAz����7����;�9���qc���~���L�!�N]�8�i��]�l�Y/[ � �-B����3~�Y� ��QRn>���aXG4$[l%ZyAX{���9JX��%��a=��?u�S���)t�p���i]S2�} �r�X�����;�?z}f��#�\ 9��\q� vn���_���`��%B����q�2��6�*H����h �S/a�.f��"��gH����f����h��;��K.]��lf��L��.
>��h�K���xiz�~���hf��� �� ����WY���1lAR� �\� �K ����M�;�R/C�AA�=��W{��A��9�����#��j����C���.���r��7Iu�y�^Kt �%
+� ��gg�#�#�����i �DX�i��d+�^*�}{�A{`���,�'�m�R�B� �.�-.�<���?�����w[�rJ����w��� H��GF�-���/_���y��$�2},t� �>$k � ��E
��TgB���"K,
k���8� �����CdY����6�7_?�������������v&b}�1��{n�|'��C����G��HQj_=���/<1%#��j�#|C�w��)������iY�j_�!-AAA=(� H7 Y������D$��k���
y��I9����I�Y-y������4R�L)��p~�@./�D�p�~��jY��"��� �q�8J����%%� ��l~�w�� �|rI��v�6�i������t'� �|{�%s��M�5�ly��/&�E�(�H6*s�@n=\n�c��0���U�j A���������X����q=G�R�$�g�q(Bl�K�%}2���R[�a{�8�������w� �����N"�E1R)�Yw?�������� ����hc���$�f�fvq
K !��<C2}��7HvGAA� (� H7|vw��W �������N���#��>��T��&hc ej=�����,_R�?K���Ys��H[�dl� � �\ ���a[�'v��D��Or>��l0lR�nS� '�U��G
� �B����2�����"�E�8�9n�/�o��4�����r!D=�m�P�y��/��H��0�rq{��0�����3�c_'&�)L�������������>��[�y��-�� ����p��� ��o��g����oju9�64�V`:�$���F5�=b����m���]+�R��k�"!��D�O���HUj���� � � � L_B�n�����T: �����`K)p�Z�(� �Y)���jV&��!��j� @�� �Z�P�X����� 6�� ! �f<�g��E�>��N �(] �����}��%� E����JR��O2��s�`P n�(r��_Q�p� �<*�q\x
�N�[�Ta�]2kU�X�"�����j.=��1'%���eQq�3�_'+]�h7������q�D�!��%e�BrF �*�1gr�J\.8�.�nx�S_����R��t\�`�a&��-m��e�J�'��s�
�5�c���P�sTn"l��t
����+�a~<��<���BAAD�7���6z������#�cq5 )��+�����(��Efk�D�C�+#2{�����_5d]}�U�S%�)�
3��L=D ~h}D�`��, <,�y��R����~z�>�)e�3�����/��{3��/�{�C��/�(��c5���,K��
��z�J�: Bc]���������@��C�v�B�.P�8�9��i����5 ���T�&����]�������(\?E8��-f1�Nv�@���k�~M � g�9� %���� * ����"40�����,���? �dx<"��J� 8��0,�\�� � � �=��� +e��,��rC��� ��s_��E�K���OU�z����47��`~�K���FB��,�'k�#��<���E����)��\��U��&�^�L^1z�>i�� �)4@!"���J�z�!�Q2��r�@i��n;�_�AK�����+�rn�r7����1gD�"e��u�p� H�zm���Q"�`adc����(�nB 74��\�WU ��t�m��(�m��U�'_����o����9���=-l&Ws @ ���Zz[�| P�� ���u�4k ;J�� ��U��J�@��NBx�����. I{<�����������K� � �e������'~J]�%��'7��wm����K$� ����� ������dn���g�������=�zs��{�CZc��?5vL`��p�*b�����W
M��lgR@�X��NBaA��.#? �=�Y&C(�S��#iH?��\�/Mp+ %���r���mJ%�%�7~.��_l��PW�E8w��X�V�,�I0G�Z�(��o��
4�K�M;�^'��1r�0 ��
��4 �Z���}��i���E��������� �Q���|�m��h�<�b ������� M�!���&�
p=�UGH�i|�a�n�g?L�!���cZ��+������M�b���? ��7q��# `P�wCAA� 0��z���;k����`�����zy�=�(] ��K����GP�Ic��/KuF�� ���\t�0r"�G�^�� �H�]? OL�*�l��u��HJ���s���:/$V�c w&% � O����6?��= ��r��Id\��+2�����7n ��>�it�-9�C.���RZ���������^-{�����Z��$��$�����JF���,�U���=��*�;��u?<=�P�|"n0/%=Q!���e�aS[�"��D��u������H�����v��E � [r����B ~�k�#�6�����,�h6 !� � ���W{��A��0�P����Le�q3J���6�����'�7���hsN�N�"d&�y�Q2�'-K�����$b���HS�� c�b�T�
E=6Zl#���i����Y��e��?��3(���������D��253 �=�sO��)�.)v�P�������%n>m����� ����&���Um��L*���$�e�|:S�qD���55 Z�"��������4�� ���]\��g��#}]seXy\�)�2Z��������9M� ;�(��'�NFQAAA�s�(� ]aV^�|h���! Q����oI���(�ja� J q��W���� ��D�T�Q�n4\l�����|�(-��jB<@Tn���" ����������� ,��Qx*x� ��)~_�����9w �8����*�&��$%�u V�d�TatB�6.�*��Dpy�k�,��� �l��%`<0r�p���T������/����h���.����$=Ns����_o�E_��$��$��+���0/����gl��L�"���dX��7c�5� � ��G��w`A:��E�5�Bu�=b9�]�z��(�HG"�����">c�pB��H�;�J @X�P�A#@�#aY��8�8�r�n�k���m5!���**�^��2���y� ���|z�@������ ��s���@����r�����\~� ��a�f6N
Q���8��s�7p����K�C�(2��x��x�"�����U�����{����~oU����n��.�y���`c Q������~�gL��o�A��{=�Q����^<umK$��!g��ZP���Yo `��.�a� kZ�t �FAA��((� HW������eG����+9w?����_��������^%�-��V�)�G��r �\E"�O������-�nE ���;��V '*��y�0��:$�W��
��<rU+����� 7�/Z_��n����pp�H�D�����lflF�`��Q*)��Z�����wM�H"��a$F0�G=����"���<���8 ��# 8U/�D���q����B��]Xh���� �����%��$nR � �@��u������� ������=$G��]m�� ���}����=R�v>�N��8������.PH�J��
+�[
��
p%� � � ���2�f�E�E GIy��������U��r�i�~E��+�L}��)� _����0�X������q���D_a.jhs�Vf�|D�YTh�fi�Y�4)gur�;#�����~) ��N�G�L6��}y�$)B���)� ������$$\n�c����$d����S%�z��z" ��dawc��V�l�@�0m
!�UW �l��A�E��_
�� |1�/~�����M�|��f8���� ��������� k ��kB��u��{�����Y�Z���@ � ��i7���P�9�����G��p"� � � ���h>�����H/�q� �� �\�u��`�����$�Z/���ZZ!�Xr�������3>Isal�B�5��^
Kjc���X�%"@�������M�{��x��=m^��� 5{�q�J����y��%5A�b��|<#����J��+$bo>��U�%�I}�2���(%6 �]������P�����&������y|���������+ � �9DA�
A��[j=B���������XD��V��m���z�^��`�r�AD�B�� ��5��3�� ��dvfsx�������������������~�(���K��[qU����{����VP��� ���b����h�����g����xX��P�����d�1��0v<���^o�r��D�1�����"�s�i��C)�_(PbT��������� � � ���=�"���k����7����y6�:X�|�_����
.�j�t���R�P&��55��?���y��c pwe����h�(��������!��e��
W��O�v� P6���gm�E� �@�-�-� @��g
G���[,�R��'_��k|Q�g�)�� on%�����#����hL>z��"���P�&K������Vd "q��EH�ZTQ��&>��T�dn� 46
��*%���T*��2��l��[G���)���b�J-��@P�wr���#Q1��Y��E��5L��c�� ����Z} �< ��>C�� +)�O�Bk/q��f����!eAA�E������j�k����� =J� �Z,eSEL�1j4"u>/�� ��78_X�jq���?�a_J�EJ��`Hk����� <&���{t��ke�:��) j 9��'< b��'�h���i��I37�kS���*�����~��T� c�-WfR�����M���""�ssb� ����G �s��V�T�Vq)�xEa�_%���3���e_�e��'����Z}��6N�$�B/�P0���\W��)�� *!� � ��f� @�f��bx� V_��H�����>e&��1e�5�Jx�|O`�I#�J���l�S�u���'�� ��#Y�� ����R� g3
��s D0g��&.C�1U����+�h��9V�^�b�ON�<�����f��P��9�u{�;��Z IDAT��Y�� /��w ���H)������� ��bi���vn�B�����&������
p�������3zX �.����9���<B[�I���h
���P AAAN �)� �y�s�"&�����;%�������� ,4���,9g9�ul�(�t�����!|�?����<q����^Z�=���� ���={6�������^��Dp�O{Z�g������v $�~�� ��v���=B�[�e
����ST_8ycF�����Wwm� P�u� Zp(
UU���M�s|�n���v����K�?2����s ��V^�) ��^ ��g B�n�C��F�1�b��]��%�q:dO�i��!]�����>�D=AAANN���S�S3
��� �:��bq��*=�M�5���!�3�����#G� w�h��3�c�]�������������zB %�O�Y�0 �8�?����,������7��R���(�5}��)��n�H!�t��!� �w�GS.I�����������Qa�������&
��qSJ'Q?pB�����=�����(��JL i��g �1�,w=��afG��s����m V��w�O�m�'7�w�9 jJ��k&��]3�e��V�w.m ���|����d|�K@AA�S�S�� �2��������&.�o� �v@�Pp�2&��� �J�g��J�L�}�WTd� ���,� MJ6g�:2~�p:�~~���`�$��Pkx7��l��v w_A���(F�;KJ��RK���,�������4'H�XZ�>��C��.N���� �{�zUU�Y�j<@(P�A�)���e��B'X 0GU��Ds��/���qT�F�� ��(�����4�`^�JC����D��"3YZ�w��v3��}���y�fX���?�����U�1J_9:3���$-]� ��<V��1 p����E
��
��X�� �x~z�'� � � �7�T�N5��xe��e��{���� �z����� z���SO���Nw[8{H��\�C��������+aK�T~0���� -��V~�
�NV������=7���1 �%�B\�)�ly����������b�fl�,oM���){����r��E3��j�p�>�Qy��;��>���H[1��E� ]N��[i/�����^�{=��EQ �ek;��sq�? �d�J��U�����Y
u-� (����
������>�E_��g^t��,�L������}D��#GO��_� ��]��]3 ���:���<9�P��%�!�qj ���\:�qA���yi����1���������=[AA9����{�/_��=e����8!�"��/��S��|U�]��������p|h�����w~����f=���\ L�i�(* ��h� M!a��mi�����g���-��N�����>��1W�� � ���YP��f�A�Q���
p�������OOZ#������W%���e��3�-�2�i ZtW-{=C 3���W���6�gL3&��#�m;z� �� ���)��(� ������,����p&~|���i �6�~������^`��x*X8��=�x�� �;�o3�2����j���#��e��P�I����-��~�AAA�+������� ���d� �&O&d������@w�����v ������B���zi���/�(4Q����R���X+�vH\�����^� ?=��w��2�����+5�,�^Q��U�6����'�N��e����15ck��"�X�{/l���� Yc� c���S�`5-�?������T�n�js���!.�9�^����c��H=��B����(���6� �=���]�e`�0�~Z����a�~�N�I� � �G�]����+���n
��e�� I�p��\�-� � �|�`� � ��owMW��y$ �|���M����;/fo�'d��o�R~8d�t,�\�E�0T��A���v�aO= g�)/q�e�e���`�����+� ��{��}�</�z���B�K����NT �Z} �< ��>C�]J����s� ����:��e*IN?����7H�vw f3S ��!�r;�.��t0N]R��Nu��tjl����ic���4L@U%�K��olJ�Y3�K���E���� ��!9�X�d�s�@)���1��*�
� !���'?����8� ���uu�����s���g�-u� p"��cM ��<���z�) '7����AAA���� ���`�����0I���Y'v����] Z��di�l�C�u�'ly�S �-h7�j-���P�*��
����2�`#ns���O������A_��>R��t�X����5����[�q�%�zQf����]����l|���b��_]�� }����W��=��6�p��S�� Xc�&�7����m�M ?���9 �����- ���T>\���� �9J k\�ZZs� F@AiK�b��fM��c��N��-��� �cF'e�� �.��@y�p_U��;��@��[���������/I\��.�b�+���d��� iOw��C��k�-��~������I�KW}�R~�=
AA�.�� ���`������b������?�vK�J`v�I��=�x��E@��~����Z,��\Q�x`x�z���+s���}��9 %N.�Sw$N�"�� � <BO������_�H(����C��;f���QY>r�����s hW� ��'�(��` 8�K��g����/�� z���0H3T� ��4������ %|O&1z���~g
_~dE���C�enJ,E����tv���f�5����
"
���E� I�� ��:�!o;���]���+Uj�p�5e�R *Y8��<G��nY�3�k�� ������yf����ft����g#. pr��s�� �*G��tao�+���]!��AA��((� �7"R�:�g��� ��I+� T�p ��9�M���+��%-;�r%0;�����CITu}tW��Z�Q� �O �B0�J
T-jh+����)P}�����3���mIr�_�h�/=d�a�Z_{��B|��w�n�n���}��* /� �e<v(�c��� ��/�,�V����u�9,%���lb����.Y9 �����x�I���� <'�����7��J ��
�M�f3�4�];��S^K[�+>�����bk��N���b~�[���I� Ed����mQS<�f���B��N��� �<h���)ff�����0�f���3��N b��O��:�����]�BI~_C[�-v����^AwN�2� � ���2�������c�m�k��e�x��n��V�I���g,i#�u%x�|SH�P
�U��������]������C��U�g&�>(��X�����g��0��BB�������������m� ��a5���k{
E��fMt0_E������r�����
����i��� ����t.'$�+A,44�kXr�5I�����f
.�a�����������;�d}G��{��R�� |�l�s�` 0\i��Yj���Y��^;�)����(���Ptr���]%eWI `qr��r>�R�)&%��6���h���[��Y�
/ �E�~�4�,Z�v��T9M� � r��� ��OJ; ���i �C�������
��� jO��k���~c�(������\ �Gr������|qR��Yb��N�� ���:�����
�]>����3o��0��5}���)C���K��>T���2MPb u�����o�k$Bd�q�Lt0_����8�� ���L�b ������BZ�b����>J�y[~[�i���E��!vq��f*+������p�P b��v�1��C���a��*
1����0���2�����(��� / R��n]y���QJ�w��2������i����/�����i�P��-� � � �|@QA�!� �c��C�
�4���O�;�����R e�������~���q��u5{�VW�R��/ZP4I����P(F�F���]��']m���@MI�6��*���#��;��`��<��Da���O��� �l�vO yh���b�v� @ ��:1�9&(PE�]�-�6o=7�B���|��_�_��0���e�C�c�U��[h4q��m��W��/��^�� Q�gWc�� �v��������Ebw^ �xH�y���a����!<�JE����|�������2�[�(�� ��Ea�h �7c�iFc�S�r�d�N���z���r� b� ��|�����tQ��f�P�6H3����X���Gp����F�$�2�,���� � � �|GAQA��l��U�nU�I
5;#,��[�KW�Re��J<�n���8�Ym� j� ��W
�G$t�a�c[�e�2������r;5P�#���dI�� (����m�B@M~��t$��=�B �b������0������-�� ���X�oo��Z�5���=N�f6��tVl �>D���*��=�@A�9�I>d� �F�^��E���H������~MUi����������h��� �����QF? ���b���� $'�zh���������j$- v�� �un�v��+1e���v��'N�����~6s� ��*�w�M���,~����v����fcy|o���vy��A��-8�����1�V�1�<��f�r��o� � � ��g�� ��C<t���DNJN
����`Z6�Tf3�������H��C����� [����(c0,��\��
f��C �����JA�IJ��;�`����&�B���&
6�S��3z4�a0�X����bQt��3Zb�Q���<���-������ �<5��f�
��l{�4��HSL� *IZFL�8�� �8�0��� ��0T��Fwi��'��~
6c���F� ���12)��{��Ek�����]C��KU
����#?uJk��V I��OD��B)(� �� �z���}]Z�����c]�}z}����n�BAA�cI�b� � ���������� H*XT�[\���=i/��$_�q��m�� ,$(�; |�����K�t{U��J_s�8�T�z�����R�zd��
�AZ�� [���n ���8����- �r�7Np��~t�y����'^G��x�(����
M��>�KU��M������� ���i"VR#-��3��,��X��h
<���K[y*��l����C��Z}�qL1����5�����(�g'����K%���a�yJ( ��b_ ����F�������� c�R;YZg�0 �<�;m� ,4zYh����/R����X������/(2J����|q�i� ��3�f��� �u �O� � �|[�SA��5qy���� &"8���Z?� 2q��|�^4���(�q
(@ �����*��\���lZ'Gt���0�L�����[��(��U+9����UTeO�3z)N���]���SF� ��q���,G�R��6�2�8��� V����~���si�1�N�0` ��o�f�Id������NL5|x����<c%�$�s V���z���C� ��-��%� �����1�������r����Qn����vKo%3 2��E[���'AAA�u�(� �` 1 ��/���fT����j(�w�}DB�1� Q �s}��Z��#N pr�g�/p�� 0��}a�3Lw`���s�2 D39[��M0�9 @�����W�v�~nv��4�($�FM��u{ ������ �[3�w�c���J b��f����u������y���,u$����48�`�U-�?U6����]����=v�l�b�I����g_����K��ch���� ���/Q *����������4��o����� �u���= 0����o��2?�B�3��TY�)����nAA�vAQA��^�@S4 ���� ��'����T���3BL��R���!���ri����I?��H)���t���C�����R"On ���/n����,06�nN�n&�h�Q�l4q�u[r��<��TG�H�J�B �bJ������ ���l tH>C���A[GO�;O[;d�?����i��%N���g�s��.���: t��A.W�<��* (�W~�e����47M
6�C�(q��[$�6�bw�����&)� ��]���{��C� ����Op ������&�������b��`}F���5����f�; P)D�=�Vgp�AAA���(� _�Y �oi��{6���T���i��|^�f������1Hh7 hkK�q]��97� c��9��d6o�|:� Z*�xy�4H/|��%���&��� �w����o��p��Wj���� .+G��?&@x�� g?�r���e~�o�~����z9 l�$z�����^*��4H��4)��N��= �]���n�*4VS��
a=GK����*��e:I�V���D�f�J��X���FB�u�haD�|��9Q > /���>��a�e�(:J���u�_���x���H�����)�B�����ee�0rx�x�*����� � �qAA9E@QA����H ��O��"�(����L�57i6."�1��w�-������������r�Q<�� ���������>�@"�j� N���,[�����������K���ro��{�� �*����8i��� m3�!8zb������@����8(gl\�q���sm��B�����x� c�d�Xa�g�zBU����?���v,�9rRh:,��F�����'�|B�6p�46Q�P�7vBQ�g��
�/K9�m ���[P� %�����R��&�����#�0���1� �����1!��|�j� � ��R�(� _����1��eij����v �i�i��Xb,���~&E�����t����V����4'HTy�7C�cI���/�D s���������I����{��k��������n����������� $ ���!2��|'���D��� ?���6���d����k�����{��A ��~:���g� ']7� �v�.g�w=�7��?=�_�!�?']����EAc�6��D39 ��@�|�)2 hD�| EAA9�AQA�����|�%��?DR�����B��i6�������e ��K �`�T�>�������^���H����^Z�Y�����S �%�������'���,//������ �� b�bN 6�?:d��o�0fe��/X��5��R^�a}sX�L���TQ��/���7����3ss���iH�����f��/�M���������{e/�yU��b����*��� � � r��� �| Xn�1<��� �?f�
�* O� 0tu��O�vA s���^ (�|�[vS����IR����������=~n�v��+1e���{BF���,�:\�� ��v�Qc�%�f_��W��� dN
��[���0��0���������zy��)v�.g�I^y������l�>��`��D������Y�#��?\�*"gp � +�QoV�j�� �E!� � ��2�uh�()����K9�)fGQa����� �������j�v� ������ �[�/R�@_�L ��Qf��g��o��� �En�������r@F#�����U�VUUU���D�������J�� qZ�I� ����z�E,�����R�IH�b����;g� 2�9��`��u�K?d������ �+��Yr�
c_`�f��_��������?��/���9���l�su�*�����~}o� � � ��������p�������O|/S��ol� o��~�����I7����uwT@�8�]�n �J���%� ����t~�U�<�pS����y�@����T��%������#�����������(x���<1�t������E�{�1�:<<�f 8�S�� �di�U��[h"�8Nl/� �M�a�d�� @��*�]k��R��O�������5����E w�����,�G[�h�� ��1c�l`Y��P ���=�����G��s�JM�k��W����� ���l��h �/!� � Ho������A��C�(g�q����6�C��%��N�I$����2]:����K[B��3_8{�`�"c��5�-�6�D�V��������f�,�v���������?F=��Ba���p� �4Y�V�B� IDATmD� �m !�)�\u����H��@H��t'
:{k��F� 6+��BG��U&j�Z�<�� �����#���E�" '�� ucv�X�����&���
~��f���/$|a��{�9�6�l'�
��D�ug�?�� � � �)�2�����Y���Os�G�t���?�m�������J���
;_� �xA��
9�+?�Z�I���C�7�sGjV��g����.������2���&�g�do HY����]G?�o������A>�
!5k�J�;�A�x�y1�:Y��}�~�����m�������'�\�$ Yh���2S:O�Hc��z F>r�*��]t�Z�v'���1�E#�
w}�{�����,]�G�v$���m7�9D������n���<�[x��&�_�� � � ���wA "5�t�X�RU�ky���6�S"878~��3d V.��[8��������������
9� U����������r�R� |�#C��|4b��f����G� W���I6O�B�V�L0�!eA�������<����;{uU�t�>�����@��q���&K��� @ �U����������cG� G��u�8��c�K���/����� �V���j��epDe� @ m�S<��)|�k��D�Z}7��������5YZW-�����` ,�me�5�CM[4"$����w_)�K �x@�M+ ���7NpkZ7Np��O}Q� � ����SAJp�=T�I J��������yG�k���� ��NI�r������'F�����fm�s��Z���'��r(��}���I�� _(<~?Uz������e���[�~>��F��Q�� ����m��CD�]�����*���'?�za0,��T��lf �K m��};VJU ����b)*�K0 `V�8��o-�x��� %�����G�����tI��l��xv�
. �xJt-q � >%�6p,/mt�vzTQ:jMN
@`Rt��v�7���S>E���UR�����y��OAAA�� (� �@��N���?{^������� ��,�:�OZ.�xtv��4�(8�3&���h��z�)S�KI��a[����\�n�6���V��l����6}�����<������|�����Td�>�� JR�<����]=�
!=�>����V�Al�����n)�� 9��J��Q���]%��O m�nk� ���m+
M��� �-�&�@��l�*r��&���hm�+���:���q"\�>�8�����8� � � �wed�X ������{U��kH�����6B|�+v{�yr�x7S^ �I� �nQ��p��$E���Wukkq
��*/q����5.�:t�� �s�
�TC���w�u�t��r�5l��^H%� ��4!`�w��.�,�xSs����K���u #Z ���m,����F�ElV ����� m��u�qy��7>@���C��i���v�\>{{�;�*��'_������m �a�� �W��D�M�7������'7������AA��� � �����#�;��Y�
��eB����C�h�$�W^�M�Im�-��.16�yEgO�2������j@<�~x�i �67(SE=�g�<��'��v���G�F��
��q~��&auq?>ht��+�
���b��3�� �f��}EE :d���� 9n-8�>����4��������:"X,9���f�<M�e�\���$pp����"%P5��,
�76��^� ��4��<�96}���Z� � � �w �E�~�� �^����������+-�a�,�a�� ���3]"������<q�,/��?���r{U���k7J��-}�. ;�+�]��sC"������<����Y���{��>x������� ���P��9� S�.1�����J�W�S��N��S�X���`������w���G���Z��sS.h�JT����BUU uC���+3�F�hxo�L���+?�� $���|n�v����l��KY~�i,�wf���v�`3Pq�(�sp���%k�/�Hi����AA9�A� � �6bh��ct�M��1c���[ �tF�7�bj?� ��9�S �%)e��yv� ���i ��O�( �\9N�OE��K� )s(v �������, L
�j4,����IHkP�h�8� &,��b��v�ZI,#I�b�L���%y�v.1 +��Z�^1/���xh���b�v��1��b�{�1�2Hh�@% �P�Xh4�^��[��C./M4�J�B9��cRt}�)D&����P4l�� U�K/qg�KZ(��Y������� #B�2"On �������
AAA����A�V/�����1�j_`k}� �c��1��������B���z�66?~s ��_�����������d����b6��d���.�5(s�J�
������Dg6���U�ZI�c�Soh+Z�#=wA�I��ca. ����G��\ �� ������l+
]~��u�fk����_�Z�<�Z� p X����_vn�tm�� P�Xl����c����l����h\ z� ���I�������>b��@t�W�"�$��i��#vk���{�6/����5{ ���c�� � � �we�/��cB�m��|D���R�Ma��O p�������V��w����TCM*��iD|�A�!W� f���k��~��?Ls��9t��<���L�r���3qa����#�}*������f��j�\s�,����)'R|�Je5�C��_��������7Cx�+U�z�O�����U�X��� ���AF�P��cT1����@����d�w��|G��8KK(���?h��?:�dE��j�����;m�x����QNQ���Z�G��cSr� AA9@QA�R^�0&\=�% � S%�#jj ���<�i���[�O��=~[[S�2�
��.
��7��s�=�t���7� 9 �����8����BH�I���w�9w��{�gi�����46����P�bT[��C/���-��P>u����=t%GM� P
B����C�qY�u8Y������(��^
������U�����&���9~./��$���F$�(UU$�@�pY������������t�0�������O�������>�_���-8]��g��� �6����O��LOfk�F �/AAAN)0SA�k��BIZ����X����|��_�t<B{W*���������v�� �*�Q�+u(}�
�I�d�����$��)
��Lx��W���n��y��1l{X
�HZo'��YXte�\�� ��p2U��a�i�����n� ��}����'%I��o?���&E�� �-��������T��Z$���l 'h;�8����]�3�[*I����"�w��u }�z������
<��wL��wCS3 h�����y�{� ������.�0� �����4��y�^�@���*�
����)��l�|�����Nn]b�'�~ ��'�]��w�:� � � �(� H?�KJe���/�Wd A��b��L+������������ {>���f�Cp��������i��� ����g8~`���E�x�/,x�M y����U���;��������f� P0��OD�[���<�f"���� ��E���,��?\ �M{- ;�� ��+QEQj�L��1��~ ��&���� ������s�+��o�v�����������
� � H����9�-\X:\
<���<�s��JV��};�,?Wt�(td $�~
�}�%
FAA��
(� H
�E���M{k����?���Ym����Q af�xF��)�;P^oHz&7��St��t�[��Ty������_P�;�Xp~ �}��wm��i����= ��q������
@�nv���b�$g�������" ��q�J �a���|���%Z3�y������ 7T������C�h�}�B�����b�
�(
P��6`��8�����!� r����-S�kmE)����;4��|��RN����� �)�:����} �< � >=��m�Z~�o�Q�M09���.��F1�p~!� � rJ��2 �bSV_��r���0�H}�L��W�;|ok�%�x�>�(^�o��g�z��v>}?E���M��"ON?���iOZ\�'X\���nXxw����kPd `��?���5YsSu���l�0�i��<���1�b�}���+hx4�zO��15c��d�����%�-9�l#��D�S�n ��Z�����<�bV�0vtV��>5 @( !�� vGI�>�oT�~�JC������h2� ��_d�T��V�&�>I��$�R��R�GLzr��a�A�s
zr�����L����#� � �:e���#9�sO������#� C��� YyYN��2��S�f�i�(��+���U XvIEE��w���;��7���Pq�Y��8M!�������&�.����;g
`;�@d�2l���pL���3lT��,�=�����+�E�������8�{m2+�����N�M�s���Z,���e��e�p8�<�e���bb�#G���Y_~#aX�T���Ce 2��~$�+$� �~p�$f)2���X���2��R �&-(���mOI�3w� � ��:�(� F ��=���J� [�p��7;P ��1��X�S(�+u ���ZQQq� w������~z� ��R*2 �y��YCH�iTQ�P�!v� �>�-
p
�`f�x�8�(�� 1�vF{F�R�W��K�������8�]��;r��,��\R-�HZL"���� �����Xrs�1�bbk@���J�\sL;��l?D,�9.k��D�����[�7?}lJl�)��9�/K�� ��vkT��_��� �qg�-~���~��AA9@QA�|�H��{��m�R�A� ;_6���f%;P�� ����� ��_�{��uw�q�;����"��'�Q�����> ��Y� f[/ ���CS�#�J�y6��>_l[���O�4�������'mh�E&8g�97 t��BgRU�C���Q-���<1k�N� v�:�x0 �����\�����3�@�p�����EF7�������U���&���`�i= � \�� �nDdl����<g��ZB��}1������8�Ej ����)���� W���������w�1EF;CIq���!D�L�}������AA�7 E1�GS_�Hs��C^������g
W���XSI�1����j_��
�9�����v�m�������c�>�����xS4��������B�� ��
���4H+�v���������I31��oB��A�;��o;�TS��b ��G��PX���RX���q��k��hDU�������m �c����"b3�k�����z�u���CA0|JE������!� + �q�V�����P���/V���3w������yv����� J������ ���{�-� � �|w��_1R�/�8����- �r�7Np�;L���\��Y ?4�q����<W �wa���2d5�1 J%�b����j��x���n���
6L��O��V�K)f�D�P{{-
�{I�F���`f���0Z�<v�+��q�:*�z���G��\<YZg8���9Q���FU�������l�aCI����sZ�� ����~r���^�����D+���'������9��C���.k�B
��5ko�d\s����8���������]Q�n;� � � �~�SAR�5"9���H��� %O}v��[����
�A"Od�(������a��[o�u��
�'O�wgiI��!\��N�����Py(
��\�4vy���:w�L��CT��3!�6�����2�&�j�*�j���?������bb4�@*[
�$ �h|N`���[�4
��x�e �-x���P�T�r����)�`~�y��W#@��}�*$�2��C�g������wMZ��~����� J9�8���� � � ��(� �� � J
5(o���\{�}��p ?�4g����Cm���7�p���|.���J_�I4��r�Q�]%��a�u��� �+u_�g?�>�����T�G�
��d/J��/��{uU ���QouT��]�M���"ZTM���:�,X�Xj�E���5� � �!=O@U��{>�d�����IT��5'��3 �4�5�/��I*��W�*�l]�������X�N��C'��x��s��/��g��`���9��{U� �����vAAA��@QA���V/i>�A ���5�* �,Nn�S���DB���}��l������d�{��Y��� �h\�B�� ��|%SdT�.�*=l/Z G� w�h���.�_�]��������G �����8 A��HN����M���3�xh���b ��^�d����X���BTKn��cVd;J)h9�L����B�� �m(c �`m=��u�k����g�����Y�46&D����U��]:F�� ��9���N
��y. _Xi
�P����Ar��z�mGAA�� Bi��[D��K��� ��5��u�Fe�M=)��#� ����6n�x�u�����/�G��q�:�z�8�rg�����z���yK,�3��{uB������Ti5� ��~�0������aL���
�m�����Y�~��v�~ ���7����S3�Z��V~�
���O�6�= `�1��f��P)�FY��|�h�9�����Ck_d }��0_��*�|�`�)P�z�����G@(p~.�m��j�
��*N����H��qb�;�N{K�' Y� �nU�PJT2]����5�t��%��I��� � � ����=:e�_�����K[B��3_8{���)Z�) �RP�!x"Q�3�P�w�y����{��7�L������}$��OJw.m��s���� Y�����=K�p=�0f�0��-��UA^�o��g�#=�� P-�r \�����1iE}��`�JY�m t�o�t$��S��� T�"�vt Bv����]1?�3m����ow��| ������:� �|~�� ���kB h���m����xh���; �8�YO�_����j|� wAA��E�F�������>���H[1��E� ��+�U��! ���E
@�7��Bm
���:�]���>x��w����7�xc����k���$=���DS� �xAF���,��m$���wQ��El �FJ�#����4'�*��I�dWD �c+�:�����q���V�$��� 6� ���K�`�DZ��~���D��%���@��L����!y��,��-j=�2�O/6HE2qnp�J�s���s�qI/[ �rI*�I��W)��e[�������� � � ��`%6�|}^���'[��:dU�
t�d��������43E����5��r��� �'�rV�������M�?{�_�i��k�����_�:u�+5�,�^�7�%������D�����>�I�k��kX��'c
��F�)3���UA �
L����}_X��tH��i{N������gM�����
��Q�X #�m�$p���_d����Q���r�#�Y�r 0�-��RB#����xcR�.��T:�H�.�.��E$ U:h��V�r�wmgM��U����G:�BU���-;����3������g_s��xA���H�+�;���T��R9�r ���S~8��z�AA�?�Sq��T�� <CNr^]� ��d�����=!~�$w��6r�4�<�Q��Ma�Kb�_j��;�� �{7m�4��u���s�9 0�����6 �I��C����~� w����}
{E��"(I� }� #�t���������=��~��O:c* L���g���~�S��fK����]�J���/��������O��1?��5����rd�NB)q����l~�i`��e�������N�����9>q ,S(F� 6���u��b�Y�z�Z,��������9��������|3��8�o��x���Kp��*= _�A��0�,�l����^�h�7_AAA�N�G{ _B��OK�" -Rt�F��Yb���-��j9���qO�z#Gn�=%G�;;f� ���� .\�����)2 p2� ������� ��$���������� D2cq�G��8 �������������� &R�T� �~ IDAT.
�^��B�P��wig23O<�.�{<�� HJ��Jw?q�GO{#�)\*J)Gz� f6�XD Oe0�+ZR�V�9E�]1746��{�?:ch� ��H��Q D;�U2qv�2v�J8��I�����1��W\t�j��k,�( �=�ed������|�i|%����-�����J?��8d x����~{�AA��%�v� 0ey�6��;�v��������eml�O�N��g{
��F���i�>��[�l)+++//?��s��9j ��lc�Lb199�T��Hf\��g��N���
���-�� U�h���3v��<O���\l� ����p%�j @�4�p����tB�'�>�V���O�L �(����� �R;)Pv���i8(n4�$N�hxO�5�������S�t|e��so������xS�aO������I��� �~�_%���4"u>��-R��W1����bx�f���-
��Um *}�W�miH��-�B��1�,������C�y:o �s� � � :e� RF�2��f)�,3��]�Q�^>~�[{ VY� @�J���x�3s���4H���%����j�8S���M��E ��F�U�>f�����@h�x�U�}"5�v<�=���W����;Bev�aA�y����)����e�A���$
&�_�=j����������7����o8�������p�D��GH�+��br� !���Z�6y�2���i������Z.e��=���Tp$?w���7l��O{��K����C�����2�G���n ��x��W1< 6�]���<1����R�/R�����K� � � (� H_� �� ��`���?�|4'�c�����V���D�����q���5�6�m�-�����=�A�,�l!i�%�2Cw���z���~�C@C���w���Mo�I�>y�� Y�1"��^�������SB���G3������x �f�D���_�j j�@���N� #����-��7�6gg��_�m���P�|���]�;6~�`!� �]��BX�/5�4 v���-L��V)-�����D����n�������Wl������(L� ��R9�}��u�=��6UPnI����u�� w�_��&��@)��1� �?qJ�����}=��� � � ��/!H_��g#Sd `�m�x���1 �-zjOWhu����x}.^��m��K�*:������!4+�^� 1�Z �g~b��c�~��[ sgW���k'el���m\��q�6�>�H b��C�Z�{�(�c��,za��C��n��� ���F��=�r�w�EY+gy�q��� ���b)�` ~�/m3�"�8�s�8<��5��v��xm��C�����.{�w� E��k��Ut D��%��bi����1�~O���� ��vD;W� ����T��o5���=�DLk���������60��5l�
1=Nk�:�c��_n ��%%�PU��f���2hzzO�cx
AAA������m��e��{���� ��� ]�8W���jocj�8����� ElK�H+,4�P6���]�q[�O���a/w���� �� �J�g��J�L�}�i��C_��0��}a�36N�Y[y8��}[�M7�#�w�NV�����=�N���V���#���������'v\�Z� ��O �}0x.�I���w�+,���;w��YB�������y�e��$� 8'<h���� ��I�dW��<���'>u� ��^2]>r4���u*�h��lzQ��������oS����� ��UJ�Zhq�8_u?�����w��QT�������;{$dWQAT\PD 5
(����3�� D�Q�u^�� �+"���� ��
0��,d�����T�TWw:�����WR���� ����>g 9}S5�d~M�G����@�'��[}���gK)����U-d�W�A ��:��:YR��?1�`� ��lK���+����H��E�< p��?~nnn{�"eb�s����fK���>��&&�~��B������J]�y0���Q��
� *�� H�UR��\_��Gk4F�%F�%
'�H�M\��#������ ���v��2��bD���� � ���f��gO���$�������w�>yE��������1.�
K��4Q�jt�_r� ;���#lB�[�����;x����,�Tce��,P���w��R� ���;M��I���� !��Qr����rI��� �me�����u�u�*"
���� ���&]���M��o�Gio��ry������5���G��'���Y
��M��Tvqt.������q]�v$�
M/��f(����l�
� � d���j_ ��qE�hr� ;}k}��[�u�)������ ��7�8�����k��<}���Uj����|����D�'d�6Q[tDW T:8(l�MxJn�%a���� <-W4{��������q��}���������L���"jH� ��v����GzG<����=���a��#�(�?y�G��OW� ��9H����j =%� ����>&%\F���������\���M�|M�G������
�29c&�*O��+'9g�/vK�t�)%�+��%���L;�����6qb?{xP�������� LD�'����>E<r��V��
�Hy9����� ���(����9#e�R��3�pA��Gxe����D�3��a������>6E1,��� �|� ��
_{�5�A�����e��������kn?#U[��������`�A`��m�Sr�-�_��a�����Me���[=��v�N�m���-Q���w�0�&���o���n��[�����u�] �=��,I��G^V��$����t������R!1��-������JAf6dI�G���S��i�DL�\�c���G�p-P[]��B��i�
�-�&�,�!�U��2��Yq�������bFF
��79i�QZ���s���rG������to��+�������aYH ��6�4�1�JPO��8�����97)S��� �?Z U�-E�V�_�e�q'�-���)dq�����m&�}V�R���c���&?��C�6�0�-��F�E�7�\�}�hE��#�(�**#��h�5j�� �"tC�Y���E���>�U��� D�'��/��z�
c�a5�F�Q����B?�w���^��;��J�9�}�o���7t���K�
j{���r��Ey6�;I�*��"��E%!;z��N"��:j��Tp������^���[H�����������+��|������`
J��� ��.�������7�=d�t�����������#^����D��ZDv���X6m.�.�Y�����.������_!���H��K ����G�x\S���w�Q����A3�� i�Mx4�%�[~T;e��"Af~(���V+����+����)�:��v���.5��
f��K����(t{�?��YV�����������R����I��D��Q ���l]I������A+�R~f_�X��e�Q �?�(���:��.+������
(J�GWX��d��k�^L�"N�"YG����&""Y�����p>�|�BE���c��RQ�"eV�3f��~�i_���G�O�d �jM�����e�(� �fxY�!�OF����ut�*�T�j��?��������^ji�W ��� ��} ���A3Dd7�� |y�K����Z�<DT���Vd>�y�s����o �y��s��P6���Kk�_r �t��*� FG����oJ�������$E,����(_�-X+���Z�!"w�s�eR�"#��5fy��3�^yA�4m&K�\BD�^=��.L�5�(E5E93 ��QjX���m���ostS�>�N�p2:�Cf?���'�,�+pZg�+��4L QR�p�s6
���"2�L�J��{�����?�AwH�TcZ��r� �R��955�+ m��f{ h�����7��#�(�= )���pc���_2,���^����O���6�G�<!��_o������{��
�z^�_"�������`���e{>������4b1�>�$��cP.36d sbo��I�l�,�����}�����'�7����c�+����%(7�u��,
�8�~�Q����r���0����r�����`i�R3R���z�����/��:������?v<�Y���hy�(��o��?2q�����e��{�&�ue�\�y�}G:�{=�]t'k��G��������Y�Ec����X4&
5 �x�� @KL�5x����������^��<����1��z�#�.��o}�k�
�X%t[�X���H������a1�2�d�Vu���z^���Y����[�<�of~�Z�hk�)�����1������ � O���{�..��a��B�[����rd�eT���� ����6���'/�U���KD��5D�����:���k8�>�����3;+������==�& >���.�WWp�K
%�L%�[�f4,����w
V3 �Z�g�?���+��^�g���_e�?$�X�l2d��,�^������/��+*qQ�&��JZ������=�c���S^����� @<��-U�&>7�A��`����t1 � f!�����VY-!-0u�3�(�gHm����?�Ba��jp�=�8��NQ��=�>�nbbF{��7�������T���7�����ws��!>�:�OB�����QAGX�g����d���������2f�m�w������� 2���_�%�J����xe"�]uS��j�!8(1"2��L%u�5U�=�I���3�J4���� �|�I��v�"T�n�g���������~��<�B�&M�~\�M{�=�_���I�/���W'�0t�-�D� �����}<�)���;'���Tw��=v��W^y����p
� �}��M>w�
u�O��w��o��S���Ks��~S��,����~����%��&'5��&k��)X����Lu�<a�\M��Da�_
_��m 01��w�`��m�}@�� �+� ��Hio��=7y�5Y �s�O[X1$��u����AOT
W�a���?�* >��H����0�� ����5�X�?mN��Ugc{���LD$s��|�A�l���4|Qs^��5[� ��EM\._�'X�O{�B����w1F� �����}<�)���;'��f+��_~��o�+�Y���W� H�t�����w��k�]j�����\�M�(�!�/�Z9fI��O��� )�$�1M���8<5Ex��w���r���2�[�Z)�x
�Wn�]�<�A7W%���a���d������`��y�'%��/;o�w�Mu�P��KJK�`�,���� ��3^���t���>��E�h.�$�����E,�I�;��z�S�4&��W~�����2�J+��}*S�{���2���
��� \|~���%�f���yf��X�dT+���� y%������W\q�K/�t��7�%i������+\����J"���9�?>��nI�'3$��T!]�������Gt � i��u�P�L�Nw���H4 �
v��yf��Q�(p��y]Q�F����V6%QP�� �A#�M�Z��wi�5�y ryx6�rD�T�Z�p��O�f����?��6b�s�(����g"r��9���(p��+xJ��� +�����H��e�M?��Z�lV�/ ���J�|�y$ K�Gf|���};b�0��PW6e~���?������
w��K.Y�`���������m��.�C{�����J�N�a����;��hc_E�������nF��:����@��:|(���]w���G��V1�����~�T�\�yCF���G\��OgV � L��������4
��=7b6�r�]�)��Ds}�G7l�Q��&����=y���;����P��y�s�[7������A���l�g�� s&��G � �2 �R�7����y��{r�&�e�/�������L�S��I��j$9 z����L s1�.1�.Q��7��y���8�77V�c���=W�W�n#� ���Y�{*;�����[���. {�s���60x���.����v��5z��M���5_��5�m��v�o�����N��������\H��koU-d��+�� ��p?��K�
gb�&#���\3q��RN3�d������t��/���7������?�H�,S RS^v���?~|����=�] � �W��*J�P�[2U���������`v ��%.;=�X�Y{ONe�������V��O�cCO�Y� '&e b� o�:�����yK��DDf�{U���T��}���#S�����{���-.U01��p��v��,Kg��?�>������oyJ4����V4������]3��N�nl4�7�=�k�g+7�;�/���N�?k��F�X)�4�������'������0���}���c�"�C�����d
jo�g���GL�b?0�z�Z��MDK��TG���tcr����c����(��������������s)����{���3�����`����� G���a[F����"N?r��[r� )`K����R�]z4H��*���@���(&�� fk����6qy������/)wKKw�1N��J�^S� Me� �u ���� @����uA�]mMW=�T�;�j��V�T<|�%/Q�]6D�����Wl�����D�+�,g^�7�"��������p��*��s��ND�Ri/�\�=!�Y�z���> ���fOy�Ex�W����������C��6�|�d� ;��a��#W���(�TCi����x�rh2/�L�������,I���:������l��D^aV��w/����-[�e�{����s�)�y���Dd4�{���/���U9�;��[��U�~�l��w6L� IV��G�/+��W������ }������!L� ���)������
�[�F���/o�*&f����sh��MUd(�����[���e�)�7��w�� kz���fe�����t���P�����=D��O?�1�� A����*��;Y��3�mVJ$*���������UJ�7�m
����9�$�J��������*
\�VLL�5������}�#���c:�a~i K����4��y �x� ����Z���`Q����
P&4 N�~ U2���i��.�]���� ^W]|���*�����.P� ��K*�� @��� @���lo�uv�WY��k��p{����o���X 1���K��#�M�Um*������W>3�rSqC>�.0��m�n[w~�g����zs�c�Ah(]���
1�Jf�����.�r1#:+�TX����� ��~�E�%Y����R����*1������;�1���OP��Tk��d^d����s��J?�D�O�: +�r�{|��JL
]��$�?@��=��O�KJ���VZ-d� �Z�T�PZ`N�p�Es/-}�T����`���E2U�5�9� W ���} �����1�� T��[�� ,?r�g��G��]]�% &��L��RQ��t�9�b�\�6�D\2,�s��"u��.<0i��M�"�����mb��G�-9�n��W3S���w�D��,�sU S�m�$�l�^��jo��6������?{�_>RZ{7� �a�*��].�}�S�����m�����f��V�L\��rL��:oH��u�G�%)�7� �S> 2�)�.�)�<P�uq6��uo }�����������XD��J :e �9���*�_ {�4!�b6�8��Rw�+�����l�Y-I�������4b���� F��G���j�#
]��\�|�or���Y� �1��� ���S��)�gL���f�������\f ������$����������J�.1���q�� ��5[gx�}�\�(��� ~�[u�3�M;zX�D�iG�i/g�h��S�V�Y�x����\a^YR��}�c��u�9�� �zvx���9oT���� 6Q$�V�}����GI�K ���w L�@ ��q��>���^%h���}��1��
p�C
������(n*��Z����S�UKOJ�����M�; �C�z\���ys�W��1}@�eO���[ K�KjX�� e������U��� L������<�>�6
qb� �,{�}�S��B����s�����Y�A�.t��3o�z�R��G�d���OD�YA��� ��33Q����z������G4g��m�t 8����=�/ [ 3g IDAT���]t�C=�`�L� < F�������GD��9 a��mV�fK]!��m\2*�����u�+�X�I��Gf5��a�5eq�����pFIR���fl�=�d����JJ�� S�<z�a���1�=$p1B=e[��n�}���/9X�� &"�=ZOa���m.pD��Q���.�� ��T���2�e �d�� �1TXX8f��?�����sO]��&b�"�0j����4�dt�%�r�n%:��x���������dcH�Dr�[~���{*2�&o�&��dcUu ������ ���E��C������-���� ]R�f�(
���[��g�}�k��: �%� NN��8V <x�E=�����{/��uLG#�Op�v%��"�a�l����fB� �Cs��^�$$0"�?���eX��S���f*��f*����sS��)}����}�Vd�H*)����]��"��?[��3E ��U��p�L�?U;� N6���*����3�s 3f�����}���
����T���&�/��e�n_Og��� #��X�l��,s�xB3_iJ��Dr����� �w=��9h����M���:�� �w����]�%��Z!sc����9�����goW��������sn�������q��D��� �0�i ����3_���� H�2}\v{�[�V�fF{��7���������������Q�I�����y
�U|���=�8��+a7�-9�}�o�{H ������K��'��u �&��G�x\S���w���C���s�}���5����U���^�����K�%^J$F6�5��������$��p����n��,��z�v&w�� ;�������. ��Oe���f-��� z�+� ����be�����O/*f:p�=�*���W:L'���wO�o���H�d�����lB`s���� ����h 9.�bI��IrI��S��(/x���e���8�*
�������oJ���9��N&��w�3�
��q������jt1 ���> �o������Td����G y���j�$�f
UvU����,���]|���8���k�i�������:r�Y���1U� �=(�s�7�2������E�C�j�3�?��\����=5_�F^�h
�t<_o�a]vn~`��?��u=A��2������Jd� c�d�����+��{&������` �V�����hK���c��������:���{"6PD�� ��� �"CD<�r���&E�6�� ��� ��\�!F�,���:�*|�������0����t?�79� }�*rM��k�]�8 ]�7���K�f���J���&�����;��Y������"���������Ey�=�k��Pz���5[g~�t]FD�F�9���gI;\�������O�8�T{��WF�,�G����,H ��� �f���.����n���S���Q\w�/x����= ��rm�!Y���V���� ^����[7���Q�1:"d�v�Q�������T��=�m����"A$������W~�{�xI�����%��X�g������ '"���?�|��9W��ED<��1��D�m����rm�4����U -�]0G~ ��L]t���?4��cC����2�pE�3 �XP�h��{C]����������?R*2S�N}��G��[�ni!��K�v^Y��"� ����Nkq�Cgb_�El"�����A���pU�(������3{�`?�4 1�j� U^��}���%���H
>�5���e[���:����g�����7�������Ew\at�+y�P��g�Y�{{���, |� ��f�z��]�� �����E��)��� s�[rV8V �%f���p,$� , �s}��{��1U �9\ ���} V���W�{��K����3Z/&���_t�E�'O~�����y(���yP$��}����<��I��3eZ\���m���a%#� Vb�=���tc�3� F�a�!��e���^e���@z'1�[� �lj�����#��2���� ��;G�W�<���{������?��V^+�`�'����@�*6��l�\LD��ko ��N_�-]���y�%��W�v��n�Q����^�uq����R�����[�� q( sn�M|���WWz�,k�(�JE��n��������B����s~�V�������CO"�V.�H�,{��.����W�r�t;�e���
���
�(�������!��)��]/6��R�s�G}�����������S�}j�<"���}�*�����1> N��I�
��3��D a� y�D���
c
�6fD�
J y��C0 @,���=:e b����#�� ����'M�4������/:S���M`�w��GD�0���1>��,��s�-4��6Z~;���+��t�������k�@p-`�St�2S��H��������d}�����2~�������������7t9�D$���Y��a�e�c�'�@��j������yUC�+K�W6ZD�vE����A����@jE� '4d� ����#2�\|��'N�;w��� Y��i��z��,��wc��]l��e���l���3l�A`6q�Y�F_Q����H�w��~=���i�r�Ir�tp�];z������R����l��O����*��P'�����& �"����u�p�����eH�g��g����10!�,P�� 56(<�8]s� '4t� ���4�}�e%S��8Y�~�����'�|�" �����g �U��4�X�
G
F���z�������4l�)�u �7�~*�O�%��� � 7
�H��� �����Y�����'��S��a�T�*���:(�f�t5X
��!. �Y�@ &0���%&��
�!���0W��6{7�x���?���
�6�PXs� 5 ���N�X��� ]3����������+?m��� Xc���,��2>�1(W�� ���2�Z�������rD�O&��`X�Y0
DD)�*��fK�xW
��z�r��]���;����_�uI# =�`48z$ �v�q�d���8Y��D� 2
� ��~C
����\2:�r��3:�nXC���jX��P�v�10a�fw~� Nh��so�3
���#G.����.�����Wn.��:��� ?<�B�%�<6�Y��I�~�=5���������mEV �Z�������/�xI
�\c�S�v����w~���DCu�C������ �GD��.��Ocb}1�'[�;t��U�Y���3FgY<�k���v���}�c�vK��[��o��7��kg t��3�=i������fH��q]����J�9�~� �W|~���5���|���TWW_z��^x��� u?:!�����hO�$����s��-?J��_����������_�X����'��>r�y��H� V�:r�����#�j��p������#�.��>�����[{�K�*������&��������Mb���� �82����N�i�Q)�����g���P�i�� ��eF�� �[ �����}<�)���;�B��\p�/��Bs�m����(_�GWwP�/ �[5V;x����_zx��K��&�������� ��~^�+9�?��%�u���F��`
M���{c��� 3���
ui3�p,�-ybnqW�|�� �Tx��DF��ug�V�w=��Kj�L��A���j�{��8���Z��.���\m �Q|~�G�/@������ {�����"��f� �{B��C���?����x��[��uoq������[��K(����_�k�����#����I�o��T�T����R��Rqc+�����?W������������Z����@�k��q���Xr��F�>���������s�8�oK}�Y������n�' �{ :0��������.5j��/�����;���Y��D��[]3l� �������t�i��W�`)�A9X��~%��HQ'7����R�Q���No?Y9��e�i_���o�.wKA����#L?7Z��V���qA�Y=�%)P�?H� 3k����I_NJ�=Y9T� `��F�g���G���#Uy�������S�B�R���*�q�us�^ � t`(� 4���f�����w�K/���;�aJ+9�2c�Le���V�* �����#�����[� ��|n��g��z�����������b���J�\w�DS�R�H�
��}h�w�r|�mv@�S#�(�5�N����r+2�����p�iy��� /���a��B���r�S�B�K�8'&�X%t_�X��| �Z�4��_KeMy�S��g�Wn*���>�Z��
�/4������.1b��/����l��yZ���J���������2h��*�.�f��������� .�Q�U�n�4���f�KDi�������P�>�*���\r�f����h�%���[rF���%w3&K�Z&7��,H"���7�o{{��m�7�!�1"�[�-�D��k���?��M�V��p���KtW����F���J�������(#��p$�{ :.e �Q*2g�uVk*2D���������� � ����61�Jx��/+�\]����01��nM��[��
�Q=n��g�cs�y��M��u�R���{��wH��dvXs w����5�GU�%��0�M���/sp�xs����xWMq���\b�$'�O�k+#����8S��}_�| ��q���(3�� c� NN���(��u�UW
0���_g�5}A���''�Xlp�`�����4bq��SZ`K�w��:5�e����E�������� >YSM`VK���o'������j�$�o�F�_x{�?C������O�����N���p?Mie�p�Si��)�����+��R":e������������$��)>mv�5�����/C��n��3��lk��J�������Y���|eOO����I�����-w L����e��;6�"��) �O�����r� ?��SO]�xq++2�fM�i[��k�����#+���Kj*3��n<��)�*�1��p��U�%�t�Ivq�������R�/�� �M���vq� �� �%W4���Q��4[C�E��S-d�K!�5���D��K�%9�� Ml\j�]�4�p�'���R�k]9g�^�|�q���"������=�
3�F'���V{������,�[�M)� R ��)������+����� �xK�FD��`x
�����:�h���>�y�O�� ��m����J�K�(P��K
*�e��I��Jn��P����;���.��%c���?��+Q��z;���"2"����3m��hw]�t�/]������j����tQ��^S ��� FA�K�r�$�l5�X���Q� ��9��Q&[ @�C� ��R���������#.��dc� c �s�\A
�I�x�{�]��U I�d ����>���$�����f��x"rGfY~M<e����n �\ �d��!j�Q hT��3Z��k�0!�,�w=k� �d)Xk�W�%���� ��$�Y��jkX��������q3�Q���N7�F'��P�TLD��4m��7 hw(� �P*2={�\�dI���P�(�X�5b��>���k8�f�l�'9g��+XZ,-�3{^�Kw��d���{v�,��W��GD�p� R��Ijl��]?3�fQ��>�����21���|�g�^P[o� V�,|<�V���twHe�Y�!]�T�\��,FV+f�C6RU ��� l�%�Q �5ZVq �x�A>s� ��}�UW�����7�8q+2��e��Yji�l�-[7�����}��m9+ ���3V �W%. p�����vn����GL�B����g�^!"&zL�(���P����B�&��C����G�p�|�r��]�� z�ke���;�N���+�i�5d#��sL[W�m\�c�C
�t� ��x<W_}u����.]z"Wd��l����Q�f�3k��-������n�e~�j��� L�i��&�w~�����89������[x���#�j}-��g��`�u ��������J�7]+?�zn���������l����}���?4A~X
� 2�LF�'fIO ��sW���CD��p�s�-��d����{�����~�T��;��\/�a�k�s��1m]Q�qE�� ��q -O��1�J �R�IOO���E���Q��O�v���!�OF���js�W��i����-E�%�
����{��/m�����l���}����� I�m��[?��&��g�5��k+&I#���������?��p,���3���/�O;[Z�-�����?��z� �����t����yO;w<!����;5�p �����uS�BjZ<w b� ��)@>�o���iii���� ]��H�w���-�U� ]��6�uR&�zaZ� H1,�s�������a� ����;��g�N��TLn ���v(�����3fq�7���9�Lk"���j�R��.� 1Y.�n �U$0 o]9�Q� 8A�(';�����c����}����a��&�����t,&��]h��<��sS�tZ�7�)��2�<;k�)����)~&Y��#VR�b�)��Y�D ����S�J�B:e�����O�k}w��Y��h����jK����;U 8>N�� �6�Tdl6��e�:@E�� �r�����Kxi+ia!)��M�R�]���Z��H�N�/�9_����SnU���2W��A_��3����,2��|��������T���Hy6}vo���������L!N�2�c�+��6�gE����� p�t�O� -���'L�`�X>����Q��F��4�&Mn�i���(b� ����n����j�'�����+�-��������� m9�������53"=�O *S��Pm�q�u�;��E� ��(@s����'������w���"<��Yb���b�,4��t������}�# ���}U�u_ �*O��!��+q��1���FCg�8�/�4`N����-��kb�� @ �P�Eb�Td�FcG��i+16��`�vmXY��*Y.��`��^�o8�����r�K{X���Q2��xW]�~.���|e��%m}$B�{��M��{\~���T����-�����R���a� 7����N ����
�����Fc�yS�wf~���r%��
�|<��v��}�����5����V Y���L"����Q�U����{%f�����op�N�
� :�l����ok�k��2���5�F�I��(��E���-�����+�8�e��;��� �x� 8�(A>������t�!816�x
��f��|���~(���<��I���Hl?���?BD�@�;e��t � *_�0(7������S��+���!�P�$m���$�xC���5m�� �
�2p�$���o����W�n�� �Q��%��O����3x�EDJ�qf�=�5�����Z�������� ��,"���E���������]�t'$�M��qC
+Q�_t=�@F����� 2 �2p��$i���555 ���ln�� �]?����]c�.�4�8w�U*2
p9w�keQ�5�E�p�-9?�'�����a���q�5��j\�����ha��%M�+W^��l��u�i�4�7F���%Q�)��^@�$�`Y�[��s� � �('I�n������O>�$��L �� ��c�1*z�K�M+�m#A������tE
Q�;&����!�GgY�������D���F;e��8���tw���<����j)�����Y�Yc���t��(�����; /��S�m�|� �#�B�'I��i�� 9c��R�:�����} 4yUF��D���{Flu��r������k+���E
#����y
����Y���|eOO�2��N�
E H� '9g���x���r^Uz������5b���
2MM�M0 F�_S��� ��as�K�M ��5V�;=S �^�0�B0�W�&I�-��R^^��'�X,1}�-_�'X�O{���'#go� ��S�B#i��?.BE�J���� 9$�3o�ub�g�5��k�F�F�a�uck���Y�kS����=����/�j�� �S���B����m�D��^��Q`�>�;$h��Z��o.���h��0DiK�wI�i�5 @c���=�/AG�Td >�����X��V:�� IDAT������� ����(CXBJ�Z]��'��;���"C�;��xW]�~N�#�+q�/��� p���,�;(qt�%��C��9����F���.�7:�MI &L (�@�%I����ZVV��� ��`Qkra�E��cP����1��f���kJ+�`�!����z�D�l��"�T^8�9��`r��DD���J
E��3���z� �����*u����#Z�4L �?d�@�$��m��VZZ����Z��f]����
U�,t6{������0Q�HZ��{r���G&
� I#�o�K���3�+�������T������)�����*_���aI�)��7zT���nv4#�G*�D�Wjq ��"���u�nI��\���G� � �2�q������]���j��9J��$��7����qH-W�.����D=-I.��(Y.&"���Hx���?�x}���� Zs�� ~&�����-�p�_�����+���%��I��
���( �R�������]k���#k,��l3�O�i1����U Y��~����Le$��]�z �x�sI�>nf���������o?3!����+�@�1z�K��b��` 6 @�� t(��{���� ���s8 �� ���5"Uj@����J���b��ohS����iy�������0���;0qd�e� �v9G��Qq�/z"e��+�V����N��7���%�j���� ����W ���{���� ��"C�r@�
�����%,�TN���]��N�x���]�� :��j�%g�ca��]b�#b������D�L�k�������e��&�9=by%<��Z �[ �& � �) �|����w���� ���hu�y�-?��H�D�&&O��� g�f-�h�0}@��lK�U����}_����4������^5S����^^����� �Z����I '!t�@G�9����v�������a+J;Rf'%�#�[����[0~(�%��}F�K��9�6k�5��2:�r�)V]� ����['�X� ��i����,���-��hLK�: p�AQNx��Y�f���c��u�V�Q���,�$=)�.�!���N���� )/]�uHw<Y.�d�T4n?3�� ����)|u��$N����
�� fT @k�('6������}������n>�q2����N���(1����t[7��5bfc���`Y������&�9k�w�������I@���'k�7� ��@QN`�� x����^�fMbbb{/� J}��'�(u
�,R����`Y�K�xW��:�0S=gt���K�g����t f�h`���nf~��S g��"H@�v��|�+<0�S��f���� �I��I���umGOc :Z�9I�S 8F������*��9s��_�~��
)))��h���l&�,��z'��3���4|���d�%=�W^_�'5&Fwm)�.��&3�s|��t=��K������6z.�{.������� v�\��O�v�v�b�gR�i�t��L��T�}mWm��LF�� ���� �#>?�����M|�s����~���6lHMM=n���-^1$������sf}YY���Pc�������G�h��5 ?������gQ�{�|�K���Pz����&�9�����{�����7�R����7W ��� ��})&�������Z����� ;��-E������RP����]x(o�}��s�����H�����I� {]�<�Vd���]�,P�O���DF�@�@��<�Y��� ���S p ��?_�8��k� E����P���?�x^^�����gE����u��"{�&��2�2�^�Q��S��5=Q94< FQ������ti��M�����J���?w<r�Y���uZ>�Sc#�#����f��o�v06 ������ ��{-��('�'�xb���6lHKKk��6u�}��T���_�z{<��r ����2��1������}���������S�B�xF/�Z�Z1+�x�y�f� "�,��*����UB�����m��q��r��E 8���* ���qKU����g'��s�~��g7n<�"������@�$�`X|F��>����R<��;���E�-�1�k�)Q��b�����n�y���t����#����n����J"����� �2p_�m
}��m���j�z#�m.������[�}������F� �����Y�� ��/>?�����M|�s'�'�|��O?��qczzz�,@���$'
�� o���3q���W/lF1��w�]xMD-��P�]_g��7y#�Q}�Y� ��.�W�dL";��m� ����+j�!� �h�Y��7��������j|!7<v�* ������� �y���oE��Fe[8#���_��O�J�L�eZ����m��R�]z�r�$(3N���X���.�����������&�RM���2uG�' ����jC�|�o���9�����s
�#.�"��$��| ;�������?����?�� +2��Y�Ec����X4&� �u?��O��
:9w�Q��Q��;&d�����&�hT�Q�t4�Fw���o`�� �m��L��F76.J��� �P��x�����X�"??�K�.�����f�2��-b����A���1j�%G)�=t�]� =�`48z$�Xl�ucc��E����p,�a��YK����x��I���yU �a���� ������ Q����x�t��l5�T��f���1J���e]���kJ�T�p����q%*�Z���u�vKN�3�������f���p��k�������
�/ sn�M|�� ^x���K����gf�A�Dc���,������Ty$��& ��I��5
d5
� ��V��Fo)����V��&2���_���(�"�� ����G�x\S���w��{���,Y��������k�cN�zX���<�Gb�x��F2'��a�@3�Jl����
��J�x� p�����������^BE���L/��S�1L�-d�b�9Q�M$�� R��L���@�6R���� � A�w^~���_��/�@E�dq Sso16�Y��[��Kw����������"os�.��s @<@Q��+����k����ggg��Z�xkl S�D����6,�&�10ZfM�kCI�hQ�j8 �l_�8������������� ����%E�������M�;^��j1��� � �2/�,Y��+������������Y�������"�����3�JT"lt�Z3O
:�x �7�������������{����k���������F���#����R����.������ m+>?�����M|�s *2�b��1N�n���zT�/� @��������x��\��t��g�y��/����W{��T���/+�B{R���W��Ii�Um�f 8����=�~�=����O?�4*2��"���]'��j�I�jiY(o�U�m=�:�\a 8i�(�����SO=��_��������8�z��%�F��q@�Z�i�����d�6y��� ��P�����[o���n��1�+2�;5��&5Y?jM/Lc��?����E�j����H[� 80���o���O�[��O�>����E��h��P\.�1[��Kw����������"����O'����`G�Q������' �6�N8��y��� {l�����~z{�%&-K-9����Z�U*��#E�{a�[�|���{ hC����j���=���
N�� 5���.���biu�"��Q�zaZ����-bC�Lx� �XCQ���>�������������ki�8��~<����R����|:��,���-��hLZ�}:��+m*�������f�WN�+��_�= ����m�e�]v�5���B ���x��?��������>�*2D4*��i���l�QG�sI��*���#j��ilk��q�H�I!����������N �`���c��}��M�9|��5k�L�0�8�j���mO v�����t�7%�8<� E8 V�\��|���g�qF{��%"����������V�Xpi���2�Iy��v�Jr��m�d� �3������{!�,���Q t�DTw��?���79���GP��cn����f�����}��g��Z��biu������WRFgY���u��{������5q� '��fee)_s������;w�v�i����� x`���)))6�m�����~{vvvaa��w������[�.9���� �����; :t��� ���HHH�����9s����vn�R�Q]����P��BQ��U�V��5���?GE��X�KkF2��&+)6�P���Gd�������}_��O�i� ��,�l h]�v]�v���k��6t�P�"CD��O�={����cuuu_|��W_�����{�5k���?���/ >|x�m+**n������������_�}� ���<� 9�?r�CQ����W�s�=yyy h��@����D�7t<E��l)���!��74�'�����[� �z/������m����>��c�O�nw��]cD����������+��>���/���R�!�K/�t��=���o��G��f��� r���X?�����X���������k� <���m��#��J��M+��r������Z0x �m�z��Dt���� ~��������%K����^z���&MZ�hQ� =c�����u������6\vc��v6���0�����qO ���cb���w�}��5k� ��k��' ,�S���e�q� '�`0x�������iii� �����w�!�����]u�U�������������j��1�orq 2}�F��@Q�^^^�m�������s�i���1? ,Q*)��L4 � ���( ���� �4i�2���vWUUu�����NgMMMvv6u��m������{��w_x��B��,����k��;w4H=�n��c�L�����d��������[���[o����P���������v���� ����{��x 89�Z���v�|��������e������ |� MI��|�����c�g������e8��7o��a�����~z������S���y��M���3����R@�1�J�Z�n�-�����s��a��8��a�R�[� �x 8�H����2|�p��� �����������v�Z�v-����������N����iXW���*AAq�d�q�>��F��',
��eH�`P��5D����J�6
��unE'jt����1�`pAE%A�f��������i������W]u���������T��O�<Y��N����NKK������������ �t�"�^�~}��%���nnn/�����K�v�:l����$}sO7u>��1&�Q��S�c��M�4�� @m��h��%4����I�& >|�� � )�fp������C������� ` �������������^^^J��`�7��� ���)Uj���g*���_�����o����t,� ��[���R�g�����|������1�L+ ,F���L_B��9s&44���dd`1����A*��g��V�
GR�t�����������W_U:���
m�5 Yc-� �q$e�g�� 7n����}||������]�5� Yc-� �q$e�`���?������������rq���NvbX/+m ����Kh�s�����������W�X`��vv�D���H��; ����'� �H�/ Vu��Y���;vlRR������ ��cI �S��=��`�������k��_��@m����3�*g�|0!�`��g�U�'@ ����#00��W^Y�hQ�+Y�lY� =����104��`����n���{��5j��� u5�cI��[���Cl
4���J>�e5R��� ���i��M�v���={�4���w���s'22�Y�2&�����R�jJ:�j���� �Z�i��H������G�NHH #uj���� ��� �U�Z�T��&\.={��3 x��^�:&&�b�}z#�O�>�)+��is���t��}�/Z�u��H�###c��Q;v�=z��� ���e��}��� Kzt���<�� �Z������]�-I����333���������wvv�����BZv�hyM�~�������� ��I���1r�����+ �D��XRg���'�eL� ����GJJ��;>>����0##��5k�l�b��r5�u���<�d �GRO���)gd���t,@=����
�:Cl�0q �6l������� <x��� ��G���;u�d����j{���5 ��,���N<^fffPP��m����)6����~�;���o!�ql���G4} ��{�!???**j���vv���N���y���s- ��#\Z:�]Z:� �p *�H<��K�F� ��k�) `^�Cl���l� ( @%jkk###7m���C��W�\�|�rhh�����}���O>��[�nf�'��@I ���F�fm����A]�/_��q� ���c,�qs� ���� 0`� �������C�~����yS������������M�>��`��%� �/_ 1b���_�u�c �*99Y��DDD��III���z����7\BOm�,eee����_���7�P: ��.\��r�JAJJJrss���+��!C���A�j�������rp��w��3f����������3�������o,0}IU��h����F�W�����k�N�0A�X hN�|�g�A�^�:|������ �AR���W�������k���J� �� )c�~��'�?���dd �$�26-;;�����5o����� `[H�����l__����'� �����Q��]�32��MS: l���6��nVS��u���'**����V: �N������9rF��w�!# ��H�����o���.Z�h���J� �M#)cCn������`��3f( ������32����9s��� 11���k����������w�5jT``��������1<4��s�6�|P���s���g����g�V: @�����8q"&&��J�2k�,???ARSS�����'�4w��q�^����E��vN-�z�x�;Y���R��=#e�~������s��!# 6(??�w�����!((H���s����� �K 4�Z�T��&\.={����Z{��y������L�:u��9J� h�o��f��
���������zdd�(�rQ�~�������$�>}:33�O�>� <��3K�,1�G������;�����_�Tj�����vVl��
��y�����1b��)������ �$###???!!������v������?�yxx��������===���c�i���o�[s\X�����#kl ���Z?�������o��x�b�c 4������_���"���}ll��_����t��
���������+W>������ �[��wp�����#kl I������8qb�!j +����_E���/33S^���� ���QQQ�w���{L�#''����� ;��kg��C{�8���E';1�����"L_z
���������GGG+ �y�_ ���E ��� A�������6m������_�vm�� ={�4o��khg'I ����r__R'�2O9#��o,]�T�X �����UUU��2�$ ?~|����
������ �F�),,���������N����d������I��`� ���*���~~~aaa��v= @�\]]g����hA�j���-�������\�����h"""�������$���+Wf��y��}F�����K�,~�K�$���l�(Z�Y*((���
]�|��� ����{������
�k����������P�'��Zm�v���� �]
��#�e��E���k�����7n|������� �"��h����F�W����??����+V��� iIDAT( ���G{5��6��r������������]�t, ��: �y���+**>|��� �I�����#F����[�N�X @���b > >|�������+ h�2�J #3l����8�c
FR�* :��>P: �$e�Oqq��#� ��� * h$�2VF #3x��
6( h<5~�[m��1�������~��m��ME�� �:�����cR�\�����#G���{���dd 0�J ��`��u(// 5j �$e��F�=zt�^��� �"7�
>>/�����+n$) ���q���(;�I��t��}�� -Z�D X�����O��\^[��y�p��H0�������F�+'gd����;w�� <������������������m�i����� ���_?wwwyKI�N�>�����O ��T�E�����k�[��4.���>�:�2�J�'�h4�����u## 0�������v����K�N�f��M�6���� ?^����#%%E� ���i��� ���u�h���Q����JUTT�3�k�� dd F���n���K�.� �h�"&&�{���l��A��������r�J De��K�5. h�����FrF���c��]dd ���s�o���EQ��� 7���� ���QQQ�w�����@T��W�-]~����k�X �kEx�W������� :�� ����������:kjkk###7m���C�D��#�m�v{�nb������ow���e���SF]���������w�^�d. ��G� ���^^^��N�;|�p���DGG���
0@�FSXX��a��D��'8w�`�V�1TD��IJJ��'_ 0����.\���#�V�}��w]]]
7HNN�h4�bFFFRR���D j�"��X��Y������wtt���O�� $;;;**������~��q�$����__�j�������rpp�7.,, 3f����
����Il5��6�r������dd M���/( �Z�3)��%����-[����� ��
� 92x�`�A=H(������������~��eK�� X�i����wOQ?���6m�( ����;jc�1NrFF��� �L�9}��2��j����_|� l
Ieh���' <x���Q�p ����Q�V�}��7 >|x��!22 �&��di�����"�� `�)cQZ�v��I <8t������� ���� �V��[o >|�� 6����h��������� x��e�t���'��� >|���Y�p ��)cvrF&'''%%�� �v�hNT������v �ZG�l��tD�BR��$I�1c��7RRR\\\� ���J�p� U� ��i nl%�-����KE��dFrF�� ~HIIquuU: ����?,,l���~~~����V��{��\�h��>}���qc�� �G����9q������� 9r���CG� y��5y}bb�������� ;6f���������;wJ�To�F�<xp� =V�X!������c���������rF�H�� ��E�#� F�bc��$I3g�����SSS�� ��~��w���EQ,--�����cG�����������qqqNNN����������:u�K���N�:�k�N�����������E�������������]\\�Z��U�DQ�S-�K���BBB���+Q�����������;�����n�$T��6�S����4����)c�$��5������ ���h<<<��G����m���{w}����� ~(����-$$��o������ rFF�n���t:���������7c����X���o�������]�v�����rv��5e����_�v� Y����Z�)��$I�={vff���G[�n�t8 ���c���+�����}{oo����6m��K��� �����UUU��$$$�����������eXs``����(\�p!88���y��M�:u���� ��_�z��-0���m��h��3�Dg'�Hs7j]H�43I�������AF lP� =>��cA������ ��_�zw��������C���h��F����F��ol�����K�..\�������>���� Y��t�k%I���K��h��0}�9I�4w�����.55�� ����2�k}�t�2m��5k�$&&��������_o���G� )++�/j��c��
8��RA�y�����k���m�V�sk����3Im���~&��a?��L��$i��y���;r����4 ���r������/����JLL 4h�)��o�>==]�]RR������y�p��={N�2���T����y��� ;�m����������7o��yM>P45�|XmL|E����� ?~�� �k� 6���K)))�$�����q���2<<|���� TWWO�0���S���������g���9s�����]�^z�� ��5����E�nnn���S�Li����5k^z�����'N 4h���Z����6""b�� r��K�N�:�m�����[����:��������+�d��c���8q�}����
`;��KLLL#J��,Yj8��v�3)���f MF �f���7n�����j�1Q�6��iK�.MII9q�D� , �F���7>>��_~���WHH�< ��RI�&M�TXX���_^^>l����x���
� )����������9r� �L�I�/5��e����K22 P�+V( X7:R x*�1Q�6�M�-_��������?:v��HT `E��� `E�H�����2R�1�{�=22 �)�����X��o����'�� �Fc�L��[����'O�|��g�������jk9X5�i���V���,��������e���U<T:R3�����9��e���UC�t�f�M
���qJ��������&''���366�������_�k���'O����5g�0�:'��� ���d �k�ykn���X��
<���� �������6:���M]k9X5�IGj�������F�H�4w����o���[���
�=..�� h.6�NQA���j�� |����;�� ��bCI����s��S��� ��BR�$W�^uwwW:
�%���!X�� ���d �k�ykn���X��
<���� �������6:���M]k9X5�IGj���pq�I����U@ �����~ ��� �H� (�� �l�E�:�n���%%%eee�n��v�� ���C� Q:4 `sD>- `yL_ P I �� P I �� P I �� P I �� P I ��1Uyy�����o,[�L�X �:��s&''��������=�������
5���_`���s��_%)cI����{�����z���T�p �
��s�={6)))--����:t��9s, $ �J{N�U 6��=��U�2&E1!!a���=���� �u�����a��-[Z�l)BXX��{����,# �K{N�U 6��=��U�2 eTUUu��Y������E� �3�s������sZ�_%) P@UU�������]���uK�x @������ �������WI� PSSS� 9''���*�� �3�s������sZ�_%) P@��-���VYYY��= �!�='�* <�)=�e�U�2 8::��������e� `����~ ����2�*I �2�������_LKK0`��� ���9�W�QM�9-���� (c��s���j�� |���>������A���9�W�QM�9-���7ouO+�N7q��������[�n]�vM����!C�( �T�=��a����F�agg��g�-[�(/ (��='�* d�����(IR�W
��� � �2
) � �2
) � �2
) � �2
) � �2
) � �2
) � �2
) � �2
) � �2
) � �2
) � �2 �KLL���Z�zu��������V���~�� ��������E���D��EC){&+**&O��\�5��;x��g�}�������{��������/
��)G������ ��?������������� �$e j�q������o_ �-���7m���G�n����{�]�6##� ����=���_{���\�������]�tiUUUS;v���Gg���� �wd|_����:r�������'55�q� `S�� 9r���#�WEEE�~�. ��������e��3gi�77�#F���/22���7�*�� @ $e V��o���aCqqqmm���� )������~�������/ ������������Y�F?� >>�����������3����oO�>����G� m����r���&�`��E�}���)))��-+...//_�t�����5�������9sF���t��c�����'J���-[��� ���y��� a��{��������6m���3{���7o ><66611q����G�����������%I?~���SEQ���p��)EEE�V����oTT��;w���>����\�vm���������m�����~����1����y������k�������)S:w�lJ��+���m�y�KNN 4hP�N���z�� ��_���*++������o����M�""",XPos���sww�K�t������>}��7�>}zXXX3&e���6��;2e_�W��}�����{#777**��_~i��u��}/^lz� �J @���? 2{����2I�jjj�.]�����70�~��� �S����p��'5�����{����M �I5{yy��9���B��� z{{����X��I����t:�$I�Ci�������*??���4y��� 5j��s��Eoo���8�N�w��g�}��o���t�G�����$����^^^�'O��smm�����{�=���l��v�Z� ~�A�������oK�������s��My��w��5�_���~���G�>}Z�������3##��R#g���6�<���5k����k���{������gF���Z���������L9"I�������n��t��:h�Zgg� 4�@d?�����s��4~o�sSJ�\�z��F�-HF�����W_}�����bZZZdd�k��fJ�&�
$I��9� �t����������`oo ���_?x�@�������?trr���-$$��o�5e����?��3����� ;;� ����m���>|������&11Q?��u�� .Eq��!�@Q�����_����(>>^>�vvv+V����o��� ���O>����}�
����.]���|����ww��M����=11q��5��4���� s����m���{wK�0~�B���+W������J#�*66v�����i��ELL���#��a��#===55u���u6h��E��=�^���1��{�|�\}��US���Fll� |�� /����7 �c<f L��% �uptt�/������������TH�����S���tttLJJruu�ww'''������/�9�|PP��Q��B>{����;
K��o��u���2WWW��Q��S�R���@��DQ �p�Bpp�~����g���+W���9����~�I�{�� +W��w�^�����������icb���Wh�y6t��-�:+���;w��I+}Q�Kf\���A������:x�������xxx��ys���
;�'���0 #W��}������q����� |��qSb �t$e �����-Z(�m����y��eTUU�~�$IM9W���5�`��>�?��s� zl�@��G��?�X�;w������������R#�x�&������w��c�\UTT�)jh&���622r��M :tx������#����7~_��o���������� �1} ` � =j�X+I�������]SS#I��������EQ��5Wm����3--��J�F#�9t�������E���EEEnnn�T~�����2��V�=v�X�!�5h� #� .++�{����K�.��M[�fMbb�)��5�xM��O �Q?2#��G� ����"�Nw���� &��V�������t��������{CD��_}�����<<<233
� =z��� 0 I �upuu�1c�<6A��.[�����c��ri�6m����h��� ������������x�$�={����Uy����������~�������Ax������s��I������7����1���={N�2���T����y��� ;���������y�����EI�� ;6g� y���+����'����
dJ�qM<^
:��7�9W��/_�paNN� Z���w�5eF�^rr�F�����322��� 7��� �w��t��1~���W��[����5k�������g�~�)w, �D��W @���� 4h��]�vUVV���������������9s����$I�&M*++��m��]�^z�� .��I-))������1d����hA��
����������:w��������p�B�������z�� �N����NLLtuu=s����3 �������266�����(VVVzzz.X���g��KKKKcbb����-[�����(]�j�_���1c�$$$��������W��8q����������{���'N 4h���Z����6""b�� r������}���7o�j�Jn���^2|�oyy�{��w��5�N���������5�k��� \�t)%%E�����7nTVV���O�<Y��xi���I�����W]]��o���t} 0==���!;;;**������~��q�$���� ?��#�j�������������p��1�s�f�����/���[� �����m�����"�po������;2^j�� F�+�� ����__�dIii���� /��t���]�6,))�� 4 ����+{��i��� 6�� � r�����< ��[������������_|1 ������ 2�����/�m�G�20�2 �z1} �~��O���n��G� i�/%���Fgdd� =z���[�%$���� <c� � � #e � ��4�_~��W�^!!!�������j��i���A��k�~$ @ ) ��/ (�� �H� (�� ��?�U�J��o IEND�B`�
Re: [HACKERS] Custom compression methods
On Thu, Nov 30, 2017 at 8:30 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On 11/28/2017 02:29 PM, Ildus Kurbangaliev wrote:
On Mon, 27 Nov 2017 18:20:12 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:I guess the trick might be -DRANDOMIZE_ALLOCATED_MEMORY (I first
tried without it, and it seemed working fine). If that's the case,
I bet there is a palloc that should have been palloc0, or something
like that.Thanks, that was it. I've been able to reproduce this bug. The
attached patch should fix this bug and I've also added recompression
when tuples moved to the relation with the compressed attribute.I've done many tests with fulltext search on the mail archive, using
different compression algorithm, and this time it worked fine. So I can
confirm v7 fixes the issue.
Moved to next CF.
--
Michael
Re: [HACKERS] Custom compression methods
On Thu, 30 Nov 2017 00:30:37 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
While the results may look differently for other datasets, my
conclusion is that it's unlikely we'll find another general-purpose
algorithm beating pglz (enough for people to switch to it, as they'll
need to worry about testing, deployment of extensions etc).That doesn't necessarily mean supporting custom compression algorithms
is pointless, of course, but I think people will be much more
interested in exploiting known features of the data (instead of
treating the values as opaque arrays of bytes).For example, I see the patch implements a special compression method
for tsvector values (used in the tests), exploiting from knowledge of
internal structure. I haven't tested if that is an improvement (either
in compression/decompression speed or compression ratio), though.I can imagine other interesting use cases - for example values in
JSONB columns often use the same "schema" (keys, nesting, ...), so
can I imagine building a "dictionary" of JSON keys for the whole
column ...Ildus, is this a use case you've been aiming for, or were you aiming
to use the new API in a different way?
Thank you for such good overview. I agree that pglz is pretty good as
general compression method, and there's no point to change it, at
least now.
I see few useful use cases for compression methods, it's special
compression methods for int[], timestamp[] for time series and yes,
dictionaries for jsonb, for which I have even already created an
extension (https://github.com/postgrespro/jsonbd). It's working and
giving promising results.
I wonder if the patch can be improved to handle this use case better.
For example, it requires knowledge the actual data type, instead of
treating it as opaque varlena / byte array. I see tsvector compression
does that by checking typeid in the handler.But that fails for example with this example
db=# create domain x as tsvector;
CREATE DOMAIN
db=# create table t (a x compressed ts1);
ERROR: unexpected type 28198672 for tsvector compression handlerwhich means it's a few brick shy to properly support domains. But I
wonder if this should be instead specified in CREATE COMPRESSION
METHOD instead. I mean, something likeCREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler
TYPE tsvector;When type is no specified, it applies to all varlena values. Otherwise
only to that type. Also, why not to allow setting the compression as
the default method for a data type, e.g.CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler
TYPE tsvector DEFAULT;would automatically add 'COMPRESSED ts1' to all tsvector columns in
new CREATE TABLE commands.
Initial version of the patch contains ALTER syntax that change
compression method for whole types, but I have decided to remove that
functionality for now because the patch is already quite complex and it
could be added later as separate patch.
Syntax was:
ALTER TYPE <type> SET COMPRESSION <cm>;
Specifying the supported type for the compression method is a good idea.
Maybe the following syntax would be better?
CREATE COMPRESSION METHOD ts1 FOR tsvector HANDLER
tsvector_compression_handler;
BTW do you expect the tsvector compression to be generally useful, or
is it meant to be used only by the tests? If generally useful,
perhaps it should be created in pg_compression by default. If only
for tests, maybe it should be implemented in an extension in contrib
(thus also serving as example how to implement new methods).I haven't thought about the JSONB use case very much, but I suppose
that could be done using the configure/drop methods. I mean,
allocating the dictionary somewhere (e.g. in a table created by an
extension?). The configure method gets the Form_pg_attribute record,
so that should be enough I guess.But the patch is not testing those two methods at all, which seems
like something that needs to be addresses before commit. I don't
expect a full-fledged JSONB compression extension, but something
simple that actually exercises those methods in a meaningful way.
I will move to tsvector_compression_handler to separate extension in
the next version. I added it more like as example, but also it could be
used to achieve a better compression for tsvectors. Tests on maillists
database ('archie' tables):
usual compression:
maillists=# select body_tsvector, subject_tsvector into t1 from
messages; SELECT 1114213
maillists=# select pg_size_pretty(pg_total_relation_size('t1'));
pg_size_pretty
----------------
1637 MB
(1 row)
tsvector_compression_handler:
maillists=# select pg_size_pretty(pg_total_relation_size('t2'));
pg_size_pretty
----------------
1521 MB
(1 row)
lz4:
maillists=# select pg_size_pretty(pg_total_relation_size('t3'));
pg_size_pretty
----------------
1487 MB
(1 row)
I don't stick to tsvector_compression_handler, I think if there
will some example that can use all the features then
tsvector_compression_handler could be replaced with it. My extension
for jsonb dictionaries is big enough and I'm not ready to try to include
it to the patch. I just see the use of 'drop' method, since there
should be way for extension to clean its resources, but I don't see
some simple enough usage for it in tests. Maybe just dummy methods for
'drop' and 'configure' will be enough for testing purposes.
Similarly for the compression options - we need to test that the WITH
part is handled correctly (tsvector does not provide configure
method).
I could add some options to tsvector_compression_handler, like options
that change pglz_compress parameters.
Which reminds me I'm confused by pg_compression_opt. Consider this:
CREATE COMPRESSION METHOD ts1 HANDLER
tsvector_compression_handler; CREATE TABLE t (a tsvector COMPRESSED
ts1);db=# select * from pg_compression_opt ;
cmoptoid | cmname | cmhandler | cmoptions
----------+--------+------------------------------+-----------
28198689 | ts1 | tsvector_compression_handler |
(1 row)DROP TABLE t;
db=# select * from pg_compression_opt ;
cmoptoid | cmname | cmhandler | cmoptions
----------+--------+------------------------------+-----------
28198689 | ts1 | tsvector_compression_handler |
(1 row)db=# DROP COMPRESSION METHOD ts1;
ERROR: cannot drop compression method ts1 because other objects
depend on it
DETAIL: compression options for ts1 depends on compression method
ts1
HINT: Use DROP ... CASCADE to drop the dependent objects too.I believe the pg_compression_opt is actually linked to pg_attribute,
in which case it should include (attrelid,attnum), and should be
dropped when the table is dropped.I suppose it was done this way to work around the lack of
recompression (i.e. the compressed value might have ended in other
table), but that is no longer true.
Good point, since there is recompression now, the options could be
safely removed in case of dropping table. It will complicate pg_upgrade
but I think this is solvable.
A few more comments:
1) The patch makes optionListToArray (in foreigncmds.c) non-static,
but it's not used anywhere. So this seems like change that is no
longer necessary.
I use this function in CreateCompressionOptions.
2) I see we add two un-reserved keywords in gram.y - COMPRESSION and
COMPRESSED. Perhaps COMPRESSION would be enough? I mean, we could doCREATE TABLE t (c TEXT COMPRESSION cm1);
ALTER ... SET COMPRESSION name ...
ALTER ... SET COMPRESSION none;Although I agree the "SET COMPRESSION none" is a bit strange.
I agree, I've already changed syntax for the next version of the patch.
It's COMPRESSION instead of COMPRESSED and DROP COMPRESSION instead of
SET NOT COMPRESSED. Minus one word from grammar and it looks nicer.
3) heap_prepare_insert uses this chunk of code
+ else if (HeapTupleHasExternal(tup) + || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED + || HeapTupleHasCustomCompressed(tup) + || tup->t_len > TOAST_TUPLE_THRESHOLD)Shouldn't that be rather
+ else if (HeapTupleHasExternal(tup) + || (RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED + && HeapTupleHasCustomCompressed(tup)) + || tup->t_len > TOAST_TUPLE_THRESHOLD)
These conditions used for opposite directions.
HeapTupleHasCustomCompressed(tup) is true if tuple has compressed
datums inside and we need to decompress them first, and
TD_ATTR_CUSTOM_COMPRESSED flag means that relation we're putting the
data have columns with custom compression and maybe we need to compress
datums in current tuple.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 11/30/2017 04:20 PM, Ildus Kurbangaliev wrote:
On Thu, 30 Nov 2017 00:30:37 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:...
I can imagine other interesting use cases - for example values in
JSONB columns often use the same "schema" (keys, nesting, ...), so
can I imagine building a "dictionary" of JSON keys for the whole
column ...Ildus, is this a use case you've been aiming for, or were you aiming
to use the new API in a different way?Thank you for such good overview. I agree that pglz is pretty good as
general compression method, and there's no point to change it, at
least now.I see few useful use cases for compression methods, it's special
compression methods for int[], timestamp[] for time series and yes,
dictionaries for jsonb, for which I have even already created an
extension (https://github.com/postgrespro/jsonbd). It's working and
giving promising results.
I understand the reluctance to put everything into core, particularly
for complex patches that evolve quickly. Also, not having to put
everything into core is kinda why we have extensions.
But perhaps some of the simpler cases would be good candidates for core,
making it possible to test the feature?
I wonder if the patch can be improved to handle this use case better.
For example, it requires knowledge the actual data type, instead of
treating it as opaque varlena / byte array. I see tsvector compression
does that by checking typeid in the handler.But that fails for example with this example
db=# create domain x as tsvector;
CREATE DOMAIN
db=# create table t (a x compressed ts1);
ERROR: unexpected type 28198672 for tsvector compression handlerwhich means it's a few brick shy to properly support domains. But I
wonder if this should be instead specified in CREATE COMPRESSION
METHOD instead. I mean, something likeCREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler
TYPE tsvector;When type is no specified, it applies to all varlena values. Otherwise
only to that type. Also, why not to allow setting the compression as
the default method for a data type, e.g.CREATE COMPRESSION METHOD ts1 HANDLER tsvector_compression_handler
TYPE tsvector DEFAULT;would automatically add 'COMPRESSED ts1' to all tsvector columns in
new CREATE TABLE commands.Initial version of the patch contains ALTER syntax that change
compression method for whole types, but I have decided to remove
that functionality for now because the patch is already quite complex
and it could be added later as separate patch.Syntax was:
ALTER TYPE <type> SET COMPRESSION <cm>;Specifying the supported type for the compression method is a good idea.
Maybe the following syntax would be better?CREATE COMPRESSION METHOD ts1 FOR tsvector HANDLER
tsvector_compression_handler;
Understood. Good to know you've considered it, and I agree it doesn't
need to be there from the start (which makes the patch simpler).
BTW do you expect the tsvector compression to be generally useful, or
is it meant to be used only by the tests? If generally useful,
perhaps it should be created in pg_compression by default. If only
for tests, maybe it should be implemented in an extension in contrib
(thus also serving as example how to implement new methods).I haven't thought about the JSONB use case very much, but I suppose
that could be done using the configure/drop methods. I mean,
allocating the dictionary somewhere (e.g. in a table created by an
extension?). The configure method gets the Form_pg_attribute record,
so that should be enough I guess.But the patch is not testing those two methods at all, which seems
like something that needs to be addresses before commit. I don't
expect a full-fledged JSONB compression extension, but something
simple that actually exercises those methods in a meaningful way.I will move to tsvector_compression_handler to separate extension in
the next version. I added it more like as example, but also it could be
used to achieve a better compression for tsvectors. Tests on maillists
database ('archie' tables):usual compression:
maillists=# select body_tsvector, subject_tsvector into t1 from
messages; SELECT 1114213
maillists=# select pg_size_pretty(pg_total_relation_size('t1'));
pg_size_pretty
----------------
1637 MB
(1 row)tsvector_compression_handler:
maillists=# select pg_size_pretty(pg_total_relation_size('t2'));
pg_size_pretty
----------------
1521 MB
(1 row)lz4:
maillists=# select pg_size_pretty(pg_total_relation_size('t3'));
pg_size_pretty
----------------
1487 MB
(1 row)I don't stick to tsvector_compression_handler, I think if there
will some example that can use all the features then
tsvector_compression_handler could be replaced with it.
OK. I think it's a nice use case (and nice gains on the compression
ratio), demonstrating the datatype-aware compression. The question is
why shouldn't this be built into the datatypes directly?
That would certainly be possible for tsvector, although it wouldn't be
as transparent (the datatype code would have to support it explicitly).
I'm a bit torn on this. The custom compression method patch makes the
compression mostly transparent for the datatype code (by adding an extra
"compression" header). But it's coupled to the datatype quite strongly
as it requires knowledge of the data type internals.
It's a bit less coupled for "generic" datatypes (e.g. arrays of other
types), where it may add important information (e.g. that the array
represents a chunk of timeseries data, which the array code can't
possibly know).
My extension for jsonb dictionaries is big enough and I'm not ready
to try to include it to the patch. I just see the use of 'drop'
method, since there should be way for extension to clean its
resources, but I don't see some simple enough usage for it in tests.
Maybe just dummy methods for 'drop' and 'configure' will be enough
for testing purposes.
OK.
Similarly for the compression options - we need to test that the WITH
part is handled correctly (tsvector does not provide configure
method).I could add some options to tsvector_compression_handler, like options
that change pglz_compress parameters.
+1 for doing that
Which reminds me I'm confused by pg_compression_opt. Consider this:
CREATE COMPRESSION METHOD ts1 HANDLER
tsvector_compression_handler; CREATE TABLE t (a tsvector COMPRESSED
ts1);db=# select * from pg_compression_opt ;
cmoptoid | cmname | cmhandler | cmoptions
----------+--------+------------------------------+-----------
28198689 | ts1 | tsvector_compression_handler |
(1 row)DROP TABLE t;
db=# select * from pg_compression_opt ;
cmoptoid | cmname | cmhandler | cmoptions
----------+--------+------------------------------+-----------
28198689 | ts1 | tsvector_compression_handler |
(1 row)db=# DROP COMPRESSION METHOD ts1;
ERROR: cannot drop compression method ts1 because other objects
depend on it
DETAIL: compression options for ts1 depends on compression method
ts1
HINT: Use DROP ... CASCADE to drop the dependent objects too.I believe the pg_compression_opt is actually linked to pg_attribute,
in which case it should include (attrelid,attnum), and should be
dropped when the table is dropped.I suppose it was done this way to work around the lack of
recompression (i.e. the compressed value might have ended in other
table), but that is no longer true.Good point, since there is recompression now, the options could be
safely removed in case of dropping table. It will complicate pg_upgrade
but I think this is solvable.
+1 to do that. I've never dealt with pg_upgrade, but I suppose this
shouldn't be more complicated than for custom data types, right?
A few more comments:
1) The patch makes optionListToArray (in foreigncmds.c) non-static,
but it's not used anywhere. So this seems like change that is no
longer necessary.I use this function in CreateCompressionOptions.
Ah, my mistake. I only did 'git grep' which however does not search in
new files (not added to git). But it seems a bit strange to have the
function in foreigncmds.c, though, now that we use it outside of FDWs.
2) I see we add two un-reserved keywords in gram.y - COMPRESSION and
COMPRESSED. Perhaps COMPRESSION would be enough? I mean, we could doCREATE TABLE t (c TEXT COMPRESSION cm1);
ALTER ... SET COMPRESSION name ...
ALTER ... SET COMPRESSION none;Although I agree the "SET COMPRESSION none" is a bit strange.
I agree, I've already changed syntax for the next version of the patch.
It's COMPRESSION instead of COMPRESSED and DROP COMPRESSION instead of
SET NOT COMPRESSED. Minus one word from grammar and it looks nicer.
I'm not sure DROP COMPRESSION is a good idea. It implies that the data
will be uncompressed, but I assume it merely switches to the built-in
compression (pglz), right? Although "SET COMPRESSION none" has the same
issue ...
BTW, when you do DROP COMPRESSION (or whatever syntax we end up using),
will that remove the dependencies on the compression method? I haven't
tried, so maybe it does.
3) heap_prepare_insert uses this chunk of code
+ else if (HeapTupleHasExternal(tup) + || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED + || HeapTupleHasCustomCompressed(tup) + || tup->t_len > TOAST_TUPLE_THRESHOLD)Shouldn't that be rather
+ else if (HeapTupleHasExternal(tup) + || (RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED + && HeapTupleHasCustomCompressed(tup)) + || tup->t_len > TOAST_TUPLE_THRESHOLD)These conditions used for opposite directions.
HeapTupleHasCustomCompressed(tup) is true if tuple has compressed
datums inside and we need to decompress them first, and
TD_ATTR_CUSTOM_COMPRESSED flag means that relation we're putting the
data have columns with custom compression and maybe we need to compress
datums in current tuple.
Ah, right, now it makes sense. Thanks for explaining.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
Tomas Vondra wrote:
On 11/30/2017 04:20 PM, Ildus Kurbangaliev wrote:
CREATE COMPRESSION METHOD ts1 FOR tsvector HANDLER
tsvector_compression_handler;Understood. Good to know you've considered it, and I agree it doesn't
need to be there from the start (which makes the patch simpler).
Just passing by, but wouldn't this fit in the ACCESS METHOD group of
commands? So this could be simplified down to
CREATE ACCESS METHOD ts1 TYPE COMPRESSION
we have that for indexes and there are patches flying for heap storage,
sequences, etc. I think that's simpler than trying to invent all new
commands here. Then (in a future patch) you can use ALTER TYPE to
define compression for that type, or even add a column-level option to
reference a specific compression method.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 11/30/2017 09:51 PM, Alvaro Herrera wrote:
Tomas Vondra wrote:
On 11/30/2017 04:20 PM, Ildus Kurbangaliev wrote:
CREATE COMPRESSION METHOD ts1 FOR tsvector HANDLER
tsvector_compression_handler;Understood. Good to know you've considered it, and I agree it doesn't
need to be there from the start (which makes the patch simpler).Just passing by, but wouldn't this fit in the ACCESS METHOD group of
commands? So this could be simplified down to
CREATE ACCESS METHOD ts1 TYPE COMPRESSION
we have that for indexes and there are patches flying for heap storage,
sequences, etc. I think that's simpler than trying to invent all new
commands here. Then (in a future patch) you can use ALTER TYPE to
define compression for that type, or even add a column-level option to
reference a specific compression method.
I think that would conflate two very different concepts. In my mind,
access methods define how rows are stored. Compression methods are an
orthogonal concept, e.g. you can compress a value (using a custom
compression algorithm) and store it in an index (using whatever access
method it's using). So not only access methods operate on rows (while
compression operates on varlena values), but you can combine those two
things together. I don't see how you could do that if both are defined
as "access methods" ...
Furthermore, the "TYPE" in CREATE COMPRESSION method was meant to
restrict the compression algorithm to a particular data type (so, if it
relies on tsvector, you can't apply it to text columns). Which is very
different from "TYPE COMPRESSION" in CREATE ACCESS METHOD.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Thu, Nov 30, 2017 at 2:47 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
OK. I think it's a nice use case (and nice gains on the compression
ratio), demonstrating the datatype-aware compression. The question is
why shouldn't this be built into the datatypes directly?
Tomas, thanks for running benchmarks of this. I was surprised to see
how little improvement there was from other modern compression
methods, although lz4 did appear to be a modest win on both size and
speed. But I share your intuition that a lot of the interesting work
is in datatype-specific compression algorithms. I have noticed in a
number of papers that I've read that teaching other parts of the
system to operate directly on the compressed data, especially for
column stores, is a critical performance optimization; of course, that
only makes sense if the compression is datatype-specific. I don't
know exactly what that means for the design of this patch, though.
As a general point, no matter which way you go, you have to somehow
deal with on-disk compatibility. If you want to build in compression
to the datatype itself, you need to find at least one bit someplace to
mark the fact that you applied built-in compression. If you want to
build it in as a separate facility, you need to denote the compression
used someplace else. I haven't looked at how this patch does it, but
the proposal in the past has been to add a value to vartag_external.
One nice thing about the latter method is that it can be used for any
data type generically, regardless of how much bit-space is available
in the data type representation itself. It's realistically hard to
think of a data-type that has no bit space available anywhere but is
still subject to data-type specific compression; bytea definitionally
has no bit space but is also can't benefit from special-purpose
compression, whereas even something like text could be handled by
starting the varlena with a NUL byte to indicate compressed data
following. However, you'd have to come up with a different trick for
each data type. Piggybacking on the TOAST machinery avoids that. It
also implies that we only try to compress values that are "big", which
is probably be desirable if we're talking about a kind of compression
that makes comprehending the value slower. Not all types of
compression do, cf. commit 145343534c153d1e6c3cff1fa1855787684d9a38,
and for those that don't it probably makes more sense to just build it
into the data type.
All of that is a somewhat separate question from whether we should
have CREATE / DROP COMPRESSION, though (or Alvaro's proposal of using
the ACCESS METHOD stuff instead). Even if we agree that piggybacking
on TOAST is a good way to implement pluggable compression methods, it
doesn't follow that the compression method is something that should be
attached to the datatype from the outside; it could be built into it
in a deep way. For example, "packed" varlenas (1-byte header) are a
form of compression, and the default functions for detoasting always
produced unpacked values, but the operators for the text data type
know how to operate on the packed representation. That's sort of a
trivial example, but it might well be that there are other cases where
we can do something similar. Maybe jsonb, for example, can compress
data in such a way that some of the jsonb functions can operate
directly on the compressed representation -- perhaps the number of
keys is easily visible, for example, or maybe more. In this view of
the world, each data type should get to define its own compression
method (or methods) but they are hard-wired into the datatype and you
can't add more later, or if you do, you lose the advantages of the
hard-wired stuff.
BTW, another related concept that comes up a lot in discussions of
this area is that we could do a lot better compression of columns if
we had some place to store a per-column dictionary. I don't really
know how to make that work. We could have a catalog someplace that
stores an opaque blob for each column configured to use a compression
method, and let the compression method store whatever it likes in
there. That's probably fine if you are compressing the whole table at
once and the blob is static thereafter. But if you want to update
that blob as you see new column values there seem to be almost
insurmountable problems.
To be clear, I'm not trying to load this patch down with a requirement
to solve every problem in the universe. On the other hand, I think it
would be easy to beat a patch like this into shape in a fairly
mechanical way and then commit-and-forget. That might be leaving a
lot of money on the table; I'm glad you are thinking about the bigger
picture and hope that my thoughts here somehow contribute.
Thanks,
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On 12/01/2017 03:23 PM, Robert Haas wrote:
On Thu, Nov 30, 2017 at 2:47 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:OK. I think it's a nice use case (and nice gains on the compression
ratio), demonstrating the datatype-aware compression. The question is
why shouldn't this be built into the datatypes directly?Tomas, thanks for running benchmarks of this. I was surprised to see
how little improvement there was from other modern compression
methods, although lz4 did appear to be a modest win on both size and
speed. But I share your intuition that a lot of the interesting work
is in datatype-specific compression algorithms. I have noticed in a
number of papers that I've read that teaching other parts of the
system to operate directly on the compressed data, especially for
column stores, is a critical performance optimization; of course, that
only makes sense if the compression is datatype-specific. I don't
know exactly what that means for the design of this patch, though.
It has very little impact on this patch, as it has nothing to do with
columnar storage. That is, each value is compressed independently.
Column stores exploit the fact that they get a vector of values,
compressed in some data-aware way. E.g. some form of RLE or dictionary
compression, which allows them to evaluate expressions on the compressed
vector. But that's irrelevant here, we only get row-by-row execution.
Note: The idea to build dictionary for the whole jsonb column (which
this patch should allow) does not make it "columnar compression" in the
"column store" way. The executor will still get the decompressed value.
As a general point, no matter which way you go, you have to somehow
deal with on-disk compatibility. If you want to build in compression
to the datatype itself, you need to find at least one bit someplace to
mark the fact that you applied built-in compression. If you want to
build it in as a separate facility, you need to denote the compression
used someplace else. I haven't looked at how this patch does it, but
the proposal in the past has been to add a value to vartag_external.
AFAICS the patch does that by setting a bit in the varlena header, and
then adding OID of the compression method after the varlena header. So
you get (verlena header + OID + data).
This has good and bad consequences.
Good: It's transparent for the datatype, so it does not have to worry
about the custom compression at all (and it may change arbitrarily).
Bad: It's transparent for the datatype, so it can't operate directly on
the compressed representation.
I don't think this is an argument against the patch, though. If the
datatype can support intelligent compression (and execution without
decompression), it has to be done in the datatype anyway.
One nice thing about the latter method is that it can be used for any
data type generically, regardless of how much bit-space is available
in the data type representation itself. It's realistically hard to
think of a data-type that has no bit space available anywhere but is
still subject to data-type specific compression; bytea definitionally
has no bit space but is also can't benefit from special-purpose
compression, whereas even something like text could be handled by
starting the varlena with a NUL byte to indicate compressed data
following. However, you'd have to come up with a different trick for
each data type. Piggybacking on the TOAST machinery avoids that. It
also implies that we only try to compress values that are "big", which
is probably be desirable if we're talking about a kind of compression
that makes comprehending the value slower. Not all types of
compression do, cf. commit 145343534c153d1e6c3cff1fa1855787684d9a38,
and for those that don't it probably makes more sense to just build it
into the data type.All of that is a somewhat separate question from whether we should
have CREATE / DROP COMPRESSION, though (or Alvaro's proposal of using
the ACCESS METHOD stuff instead). Even if we agree that piggybacking
on TOAST is a good way to implement pluggable compression methods, it
doesn't follow that the compression method is something that should be
attached to the datatype from the outside; it could be built into it
in a deep way. For example, "packed" varlenas (1-byte header) are a
form of compression, and the default functions for detoasting always
produced unpacked values, but the operators for the text data type
know how to operate on the packed representation. That's sort of a
trivial example, but it might well be that there are other cases where
we can do something similar. Maybe jsonb, for example, can compress
data in such a way that some of the jsonb functions can operate
directly on the compressed representation -- perhaps the number of
keys is easily visible, for example, or maybe more. In this view of
the world, each data type should get to define its own compression
method (or methods) but they are hard-wired into the datatype and you
can't add more later, or if you do, you lose the advantages of the
hard-wired stuff.
I agree with these thoughts in general, but I'm not quite sure what is
your conclusion regarding the patch.
The patch allows us to define custom compression methods that are
entirely transparent for the datatype machinery, i.e. allow compression
even for data types that did not consider compression at all. That seems
valuable to me.
Of course, if the same compression logic can be built into the datatype
itself, it may allow additional benefits (like execution on compressed
data directly).
I don't see these two approaches as conflicting.
BTW, another related concept that comes up a lot in discussions of
this area is that we could do a lot better compression of columns if
we had some place to store a per-column dictionary. I don't really
know how to make that work. We could have a catalog someplace that
stores an opaque blob for each column configured to use a compression
method, and let the compression method store whatever it likes in
there. That's probably fine if you are compressing the whole table at
once and the blob is static thereafter. But if you want to update
that blob as you see new column values there seem to be almost
insurmountable problems.
Well, that's kinda the idea behind the configure/drop methods in the
compression handler, and Ildus already did implement such dictionary
compression for the jsonb data type, see:
https://github.com/postgrespro/jsonbd
Essentially that stores the dictionary in a table, managed by a bunch of
background workers.
To be clear, I'm not trying to load this patch down with a requirement
to solve every problem in the universe. On the other hand, I think it
would be easy to beat a patch like this into shape in a fairly
mechanical way and then commit-and-forget. That might be leaving a
lot of money on the table; I'm glad you are thinking about the bigger
picture and hope that my thoughts here somehow contribute.
Thanks ;-)
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Fri, Dec 1, 2017 at 10:18 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
It has very little impact on this patch, as it has nothing to do with
columnar storage. That is, each value is compressed independently.
I understand that this patch is not about columnar storage, but I
think the idea that we may want to operate on the compressed data
directly is not only applicable to that case.
I agree with these thoughts in general, but I'm not quite sure what is
your conclusion regarding the patch.
I have not reached one. Sometimes I like to discuss problems before
deciding what I think. :-)
It does seem to me that the patch may be aiming at a relatively narrow
target in a fairly large problem space, but I don't know whether to
label that as short-sightedness or prudent incrementalism.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
Tomas Vondra wrote:
On 11/30/2017 09:51 PM, Alvaro Herrera wrote:
Just passing by, but wouldn't this fit in the ACCESS METHOD group of
commands? So this could be simplified down to
CREATE ACCESS METHOD ts1 TYPE COMPRESSION
we have that for indexes and there are patches flying for heap storage,
sequences, etc.I think that would conflate two very different concepts. In my mind,
access methods define how rows are stored.
In mine, they define how things are accessed (i.e. more general than
what you're thinking). We *currently* use them to store rows [in
indexes], but there is no reason why we couldn't expand that.
So we group access methods in "types"; the current type we have is for
indexes, and methods in that type define how are indexes accessed. This
new type would indicate how would values be compressed. I disagree that
there is no parallel there.
I'm trying to avoid pointless proliferation of narrowly defined DDL
commands.
Furthermore, the "TYPE" in CREATE COMPRESSION method was meant to
restrict the compression algorithm to a particular data type (so, if it
relies on tsvector, you can't apply it to text columns).
Yes, of course. I'm saying that the "datatype" property of a
compression access method would be declared somewhere else, not in the
TYPE clause of the CREATE ACCESS METHOD command. Perhaps it makes sense
to declare that a certain compression access method is good only for a
certain data type, and then you can put that in the options clause,
"CREATE ACCESS METHOD hyperz TYPE COMPRESSION WITH (type = tsvector)".
But many compression access methods would be general in nature and so
could be used for many datatypes (say, snappy).
To me it makes sense to say "let's create this method which is for data
compression" (CREATE ACCESS METHOD hyperz TYPE COMPRESSION) followed by
either "let's use this new compression method for the type tsvector"
(ALTER TYPE tsvector SET COMPRESSION hyperz) or "let's use this new
compression method for the column tc" (ALTER TABLE ALTER COLUMN tc SET
COMPRESSION hyperz).
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
Ildus Kurbangaliev wrote:
If the table is big, decompression could take an eternity. That's why i
decided to only to disable it and the data could be decompressed using
compression options.My idea was to keep compression options forever, since there will not
be much of them in one database. Still that requires that extension is
not removed.I will try to find a way how to recompress data first in case it moves
to another table.
I think what you should do is add a dependency between a column that
compresses using a method, and that method. So the method cannot be
dropped and leave compressed data behind. Since the method is part of
the extension, the extension cannot be dropped either. If you ALTER
the column so that it uses another compression method, then the table is
rewritten and the dependency is removed; once you do that for all the
columns that use the compression method, the compression method can be
dropped.
Maybe our dependency code needs to be extended in order to support this.
I think the current logic would drop the column if you were to do "DROP
COMPRESSION .. CASCADE", but I'm not sure we'd see that as a feature.
I'd rather have DROP COMPRESSION always fail instead until no columns
use it. Let's hear other's opinions on this bit though.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 12/01/2017 08:48 PM, Alvaro Herrera wrote:
Ildus Kurbangaliev wrote:
If the table is big, decompression could take an eternity. That's why i
decided to only to disable it and the data could be decompressed using
compression options.My idea was to keep compression options forever, since there will not
be much of them in one database. Still that requires that extension is
not removed.I will try to find a way how to recompress data first in case it moves
to another table.I think what you should do is add a dependency between a column that
compresses using a method, and that method. So the method cannot be
dropped and leave compressed data behind. Since the method is part of
the extension, the extension cannot be dropped either. If you ALTER
the column so that it uses another compression method, then the table is
rewritten and the dependency is removed; once you do that for all the
columns that use the compression method, the compression method can be
dropped.
+1 to do the rewrite, just like for other similar ALTER TABLE commands
Maybe our dependency code needs to be extended in order to support this.
I think the current logic would drop the column if you were to do "DROP
COMPRESSION .. CASCADE", but I'm not sure we'd see that as a feature.
I'd rather have DROP COMPRESSION always fail instead until no columns
use it. Let's hear other's opinions on this bit though.
Why should this behave differently compared to data types? Seems quite
against POLA, if you ask me ...
If you want to remove the compression, you can do the SET NOT COMPRESSED
(or whatever syntax we end up using), and then DROP COMPRESSION METHOD.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 12/01/2017 08:20 PM, Robert Haas wrote:
On Fri, Dec 1, 2017 at 10:18 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:It has very little impact on this patch, as it has nothing to do with
columnar storage. That is, each value is compressed independently.I understand that this patch is not about columnar storage, but I
think the idea that we may want to operate on the compressed data
directly is not only applicable to that case.
Yeah. To clarify, my point was that column stores benefit from
compressing many values at once, and then operating on this compressed
vector. That is not what this patch is doing (or can do), of course.
But I certainly do agree that if the compression can be integrated into
the data type, allowing processing on compressed representation, then
that will beat whatever this patch is doing, of course ...
I agree with these thoughts in general, but I'm not quite sure
what is your conclusion regarding the patch.I have not reached one. Sometimes I like to discuss problems before
deciding what I think. :-)
That's lame! Let's make decisions without discussion ;-)
It does seem to me that the patch may be aiming at a relatively narrow
target in a fairly large problem space, but I don't know whether to
label that as short-sightedness or prudent incrementalism.
I don't know either. I don't think people will start switching their
text columns to lz4 just because they can, or because they get 4% space
reduction compared to pglz.
But the ability to build per-column dictionaries seems quite powerful, I
guess. And I don't think that can be easily built directly into JSONB,
because we don't have a way to provide information about the column
(i.e. how would you fetch the correct dictionary?).
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Fri, Dec 1, 2017 at 2:38 PM, Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
In mine, they define how things are accessed (i.e. more general than
what you're thinking). We *currently* use them to store rows [in
indexes], but there is no reason why we couldn't expand that.So we group access methods in "types"; the current type we have is for
indexes, and methods in that type define how are indexes accessed. This
new type would indicate how would values be compressed. I disagree that
there is no parallel there.
+1.
I'm trying to avoid pointless proliferation of narrowly defined DDL
commands.
I also think that's an important goal.
Yes, of course. I'm saying that the "datatype" property of a
compression access method would be declared somewhere else, not in the
TYPE clause of the CREATE ACCESS METHOD command. Perhaps it makes sense
to declare that a certain compression access method is good only for a
certain data type, and then you can put that in the options clause,
"CREATE ACCESS METHOD hyperz TYPE COMPRESSION WITH (type = tsvector)".
But many compression access methods would be general in nature and so
could be used for many datatypes (say, snappy).To me it makes sense to say "let's create this method which is for data
compression" (CREATE ACCESS METHOD hyperz TYPE COMPRESSION) followed by
either "let's use this new compression method for the type tsvector"
(ALTER TYPE tsvector SET COMPRESSION hyperz) or "let's use this new
compression method for the column tc" (ALTER TABLE ALTER COLUMN tc SET
COMPRESSION hyperz).
+1 to this, too.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Fri, Dec 1, 2017 at 4:06 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
I agree with these thoughts in general, but I'm not quite sure
what is your conclusion regarding the patch.I have not reached one. Sometimes I like to discuss problems before
deciding what I think. :-)That's lame! Let's make decisions without discussion ;-)
Oh, right. What was I thinking?
It does seem to me that the patch may be aiming at a relatively narrow
target in a fairly large problem space, but I don't know whether to
label that as short-sightedness or prudent incrementalism.I don't know either. I don't think people will start switching their
text columns to lz4 just because they can, or because they get 4% space
reduction compared to pglz.
Honestly, if we can give everybody a 4% space reduction by switching
to lz4, I think that's totally worth doing -- but let's not make
people choose it, let's make it the default going forward, and keep
pglz support around so we don't break pg_upgrade compatibility (and so
people can continue to choose it if for some reason it works better in
their use case). That kind of improvement is nothing special in a
specific workload, but TOAST is a pretty general-purpose mechanism. I
have become, through a few bitter experiences, a strong believer in
the value of trying to reduce our on-disk footprint, and knocking 4%
off the size of every TOAST table in the world does not sound
worthless to me -- even though context-aware compression can doubtless
do a lot better.
But the ability to build per-column dictionaries seems quite powerful, I
guess. And I don't think that can be easily built directly into JSONB,
because we don't have a way to provide information about the column
(i.e. how would you fetch the correct dictionary?).
That's definitely a problem, but I think we should mull it over a bit
more before giving up. I have a few thoughts, but the part of my life
that doesn't happen on the PostgreSQL mailing list precludes
expounding on them right this minute.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On 12/01/2017 08:38 PM, Alvaro Herrera wrote:
Tomas Vondra wrote:
On 11/30/2017 09:51 PM, Alvaro Herrera wrote:
Just passing by, but wouldn't this fit in the ACCESS METHOD group of
commands? So this could be simplified down to
CREATE ACCESS METHOD ts1 TYPE COMPRESSION
we have that for indexes and there are patches flying for heap storage,
sequences, etc.I think that would conflate two very different concepts. In my mind,
access methods define how rows are stored.In mine, they define how things are accessed (i.e. more general than
what you're thinking). We *currently* use them to store rows [in
indexes], but there is no reason why we couldn't expand that.
Not sure I follow. My argument was not as much about whether the rows
are stored as rows or in some other (columnar) format, but that access
methods deal with "tuples" (i.e. row in the "logical" way). I assume
that even if we end up implementing other access method types, they will
still be tuple-based.
OTOH compression methods (at least as introduced by this patch) operate
on individual values, and have very little to do with access to the
value (in a sense it's a transparent thing).
So we group access methods in "types"; the current type we have is for
indexes, and methods in that type define how are indexes accessed. This
new type would indicate how would values be compressed. I disagree that
there is no parallel there.I'm trying to avoid pointless proliferation of narrowly defined DDL
commands.
Of course, the opposite case is using the same DDL for very different
concepts (although I understand you don't see it that way).
But in fairness, I don't really care if we call this COMPRESSION METHOD
or ACCESS METHOD or DARTH VADER ...
Furthermore, the "TYPE" in CREATE COMPRESSION method was meant to
restrict the compression algorithm to a particular data type (so, if it
relies on tsvector, you can't apply it to text columns).Yes, of course. I'm saying that the "datatype" property of a
compression access method would be declared somewhere else, not in the
TYPE clause of the CREATE ACCESS METHOD command. Perhaps it makes sense
to declare that a certain compression access method is good only for a
certain data type, and then you can put that in the options clause,
"CREATE ACCESS METHOD hyperz TYPE COMPRESSION WITH (type = tsvector)".
But many compression access methods would be general in nature and so
could be used for many datatypes (say, snappy).To me it makes sense to say "let's create this method which is for data
compression" (CREATE ACCESS METHOD hyperz TYPE COMPRESSION) followed by
either "let's use this new compression method for the type tsvector"
(ALTER TYPE tsvector SET COMPRESSION hyperz) or "let's use this new
compression method for the column tc" (ALTER TABLE ALTER COLUMN tc SET
COMPRESSION hyperz).
The WITH syntax does not seem particularly pretty to me, TBH. I'd be
much happier with "TYPE tsvector" and leaving WITH for the options
specific to each compression method.
FWIW I think syntax is the least critical part of this patch. It's ~1%
of the patch, and the gram.y additions are rather trivial.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 2017-12-01 16:14:58 -0500, Robert Haas wrote:
Honestly, if we can give everybody a 4% space reduction by switching
to lz4, I think that's totally worth doing -- but let's not make
people choose it, let's make it the default going forward, and keep
pglz support around so we don't break pg_upgrade compatibility (and so
people can continue to choose it if for some reason it works better in
their use case). That kind of improvement is nothing special in a
specific workload, but TOAST is a pretty general-purpose mechanism. I
have become, through a few bitter experiences, a strong believer in
the value of trying to reduce our on-disk footprint, and knocking 4%
off the size of every TOAST table in the world does not sound
worthless to me -- even though context-aware compression can doubtless
do a lot better.
+1. It's also a lot faster, and I've seen way way to many workloads with
50%+ time spent in pglz.
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
Tomas Vondra wrote:
On 12/01/2017 08:48 PM, Alvaro Herrera wrote:
Maybe our dependency code needs to be extended in order to support this.
I think the current logic would drop the column if you were to do "DROP
COMPRESSION .. CASCADE", but I'm not sure we'd see that as a feature.
I'd rather have DROP COMPRESSION always fail instead until no columns
use it. Let's hear other's opinions on this bit though.Why should this behave differently compared to data types? Seems quite
against POLA, if you ask me ...
OK, DROP TYPE sounds good enough precedent, so +1 on that.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Fri, 1 Dec 2017 16:38:42 -0300
Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
To me it makes sense to say "let's create this method which is for
data compression" (CREATE ACCESS METHOD hyperz TYPE COMPRESSION)
followed by either "let's use this new compression method for the
type tsvector" (ALTER TYPE tsvector SET COMPRESSION hyperz) or "let's
use this new compression method for the column tc" (ALTER TABLE ALTER
COLUMN tc SET COMPRESSION hyperz).
Hi, I think if CREATE ACCESS METHOD can be used for compression, then it
could be nicer than CREATE COMPRESSION METHOD. I just don't
know that compression could go as access method or not. Anyway
it's easy to change syntax and I don't mind to do it, if it will be
neccessary for the patch to be commited.
--
----
Regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
On 12/01/2017 10:52 PM, Andres Freund wrote:
On 2017-12-01 16:14:58 -0500, Robert Haas wrote:
Honestly, if we can give everybody a 4% space reduction by
switching to lz4, I think that's totally worth doing -- but let's
not make people choose it, let's make it the default going forward,
and keep pglz support around so we don't break pg_upgrade
compatibility (and so people can continue to choose it if for some
reason it works better in their use case). That kind of improvement
is nothing special in a specific workload, but TOAST is a pretty
general-purpose mechanism. I have become, through a few bitter
experiences, a strong believer in the value of trying to reduce our
on-disk footprint, and knocking 4% off the size of every TOAST
table in the world does not sound worthless to me -- even though
context-aware compression can doubtless do a lot better.+1. It's also a lot faster, and I've seen way way to many workloads
with 50%+ time spent in pglz.
TBH the 4% figure is something I mostly made up (I'm fake news!). On the
mailing list archive (which I believe is pretty compressible) I observed
something like 2.5% size reduction with lz4 compared to pglz, at least
with the compression levels I've used ...
Other algorithms (e.g. zstd) got significantly better compression (25%)
compared to pglz, but in exchange for longer compression. I'm sure we
could lower compression level to make it faster, but that will of course
hurt the compression ratio.
I don't think switching to a different compression algorithm is a way
forward - it was proposed and explored repeatedly in the past, and every
time it failed for a number of reasons, most of which are still valid.
Firstly, it's going to be quite hard (or perhaps impossible) to find an
algorithm that is "universally better" than pglz. Some algorithms do
work better for text documents, some for binary blobs, etc. I don't
think there's a win-win option.
Sure, there are workloads where pglz performs poorly (I've seen such
cases too), but IMHO that's more an argument for the custom compression
method approach. pglz gives you good default compression in most cases,
and you can change it for columns where it matters, and where a
different space/time trade-off makes sense.
Secondly, all the previous attempts ran into some legal issues, i.e.
licensing and/or patents. Maybe the situation changed since then (no
idea, haven't looked into that), but in the past the "pluggable"
approach was proposed as a way to address this.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Dec 2, 2017, at 6:04 PM, Tomas Vondra wrote:
On 12/01/2017 10:52 PM, Andres Freund wrote:
On 2017-12-01 16:14:58 -0500, Robert Haas wrote:
Honestly, if we can give everybody a 4% space reduction by
switching to lz4, I think that's totally worth doing -- but let's
not make people choose it, let's make it the default going forward,
and keep pglz support around so we don't break pg_upgrade
compatibility (and so people can continue to choose it if for some
reason it works better in their use case). That kind of improvement
is nothing special in a specific workload, but TOAST is a pretty
general-purpose mechanism. I have become, through a few bitter
experiences, a strong believer in the value of trying to reduce our
on-disk footprint, and knocking 4% off the size of every TOAST
table in the world does not sound worthless to me -- even though
context-aware compression can doubtless do a lot better.+1. It's also a lot faster, and I've seen way way to many workloads
with 50%+ time spent in pglz.TBH the 4% figure is something I mostly made up (I'm fake news!). On the
mailing list archive (which I believe is pretty compressible) I observed
something like 2.5% size reduction with lz4 compared to pglz, at least
with the compression levels I've used ...Other algorithms (e.g. zstd) got significantly better compression (25%)
compared to pglz, but in exchange for longer compression. I'm sure we
could lower compression level to make it faster, but that will of course
hurt the compression ratio.I don't think switching to a different compression algorithm is a way
forward - it was proposed and explored repeatedly in the past, and every
time it failed for a number of reasons, most of which are still valid.Firstly, it's going to be quite hard (or perhaps impossible) to find an
algorithm that is "universally better" than pglz. Some algorithms do
work better for text documents, some for binary blobs, etc. I don't
think there's a win-win option.Sure, there are workloads where pglz performs poorly (I've seen such
cases too), but IMHO that's more an argument for the custom compression
method approach. pglz gives you good default compression in most cases,
and you can change it for columns where it matters, and where a
different space/time trade-off makes sense.Secondly, all the previous attempts ran into some legal issues, i.e.
licensing and/or patents. Maybe the situation changed since then (no
idea, haven't looked into that), but in the past the "pluggable"
approach was proposed as a way to address this.
May be it will be interesting for you to see the following results of applying page-level compression (CFS in PgPro-EE) to pgbench data:
Configuration
Size (Gb)
Time (sec)
vanilla postgres
15.31
92
zlib (default level)
2.37
284
zlib (best speed)
2.43
191
postgres internal lz
3.89
214
lz4
4.12
95
snappy (google)
5.18
99
lzfse (apple)
2.80
1099
zstd (facebook)
1.69
125
All algorithms (except zlib) were used with best-speed option: using better compression level usually has not so large impact on compression ratio (<30%), but can significantly increase time (several times).
Certainly pgbench isnot the best candidate for testing compression algorithms: it generates a lot of artificial and redundant data.
But we measured it also on real customers data and still zstd seems to be the best compression methods: provides good compression with smallest CPU overhead.
Re: [HACKERS] Custom compression methods
Hi,
On 2017-12-02 16:04:52 +0100, Tomas Vondra wrote:
Firstly, it's going to be quite hard (or perhaps impossible) to find an
algorithm that is "universally better" than pglz. Some algorithms do
work better for text documents, some for binary blobs, etc. I don't
think there's a win-win option.
lz4 is pretty much there.
Secondly, all the previous attempts ran into some legal issues, i.e.
licensing and/or patents. Maybe the situation changed since then (no
idea, haven't looked into that), but in the past the "pluggable"
approach was proposed as a way to address this.
Those were pretty bogus. I think we're not doing our users a favor if
they've to download some external projects, then fiddle with things,
just to not choose a compression algorithm that's been known bad for at
least 5+ years. If we've a decent algorithm in-core *and* then allow
extensibility, that's one thing, but keeping the bad and tell forks
"please take our users with this code we give you" is ...
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On 12/02/2017 09:24 PM, konstantin knizhnik wrote:
On Dec 2, 2017, at 6:04 PM, Tomas Vondra wrote:
On 12/01/2017 10:52 PM, Andres Freund wrote:
...Other algorithms (e.g. zstd) got significantly better compression (25%)
compared to pglz, but in exchange for longer compression. I'm sure we
could lower compression level to make it faster, but that will of course
hurt the compression ratio.I don't think switching to a different compression algorithm is a way
forward - it was proposed and explored repeatedly in the past, and every
time it failed for a number of reasons, most of which are still valid.Firstly, it's going to be quite hard (or perhaps impossible) to
find an algorithm that is "universally better" than pglz. Some
algorithms do work better for text documents, some for binary
blobs, etc. I don't think there's a win-win option.Sure, there are workloads where pglz performs poorly (I've seen
such cases too), but IMHO that's more an argument for the custom
compression method approach. pglz gives you good default
compression in most cases, and you can change it for columns where
it matters, and where a different space/time trade-off makes
sense.Secondly, all the previous attempts ran into some legal issues, i.e.
licensing and/or patents. Maybe the situation changed since then (no
idea, haven't looked into that), but in the past the "pluggable"
approach was proposed as a way to address this.May be it will be interesting for you to see the following results
of applying page-level compression (CFS in PgPro-EE) to pgbench
data:
I don't follow. If I understand what CFS does correctly (and I'm mostly
guessing here, because I haven't seen the code published anywhere, and I
assume it's proprietary), it essentially compresses whole 8kB blocks.
I don't know it reorganizes the data into columnar format first, in some
way (to make it more "columnar" which is more compressible), which would
make somewhat similar to page-level compression in Oracle.
But it's clearly a very different approach from what the patch aims to
improve (compressing individual varlena values).
All algorithms (except zlib) were used with best-speed option: using
better compression level usually has not so large impact on
compression ratio (<30%), but can significantly increase time
(several times). Certainly pgbench isnot the best candidate for
testing compression algorithms: it generates a lot of artificial and
redundant data. But we measured it also on real customers data and
still zstd seems to be the best compression methods: provides good
compression with smallest CPU overhead.
I think this really depends on the dataset, and drawing conclusions
based on a single test is somewhat crazy. Especially when it's synthetic
pgbench data with lots of inherent redundancy - sequential IDs, ...
My takeaway from the results is rather that page-level compression may
be very beneficial in some cases, although I wonder how much of that can
be gained by simply using compressed filesystem (thus making it
transparent to PostgreSQL).
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 12/02/2017 09:38 PM, Andres Freund wrote:
Hi,
On 2017-12-02 16:04:52 +0100, Tomas Vondra wrote:
Firstly, it's going to be quite hard (or perhaps impossible) to find an
algorithm that is "universally better" than pglz. Some algorithms do
work better for text documents, some for binary blobs, etc. I don't
think there's a win-win option.lz4 is pretty much there.
That's a matter of opinion, I guess. It's a solid compression algorithm,
that's for sure ...
Secondly, all the previous attempts ran into some legal issues, i.e.
licensing and/or patents. Maybe the situation changed since then (no
idea, haven't looked into that), but in the past the "pluggable"
approach was proposed as a way to address this.Those were pretty bogus.
IANAL so I don't dare to judge on bogusness of such claims. I assume if
we made it optional (e.g. configure/initdb option, it'd be much less of
an issue). Of course, that has disadvantages too (because when you
compile/init with one algorithm, and then find something else would work
better for your data, you have to start from scratch).
I think we're not doing our users a favor if they've to download
some external projects, then fiddle with things, just to not choose
a compression algorithm that's been known bad for at least 5+ years.
If we've a decent algorithm in-core *and* then allow extensibility,
that's one thing, but keeping the bad and tell forks "please take
our users with this code we give you" is ...
I don't understand what exactly is your issue with external projects,
TBH. I think extensibility is one of the great strengths of Postgres.
It's not all rainbows and unicorns, of course, and it has costs too.
FWIW I don't think pglz is a "known bad" algorithm. Perhaps there are
cases where other algorithms (e.g. lz4) are running circles around it,
particularly when it comes to decompression speed, but I wouldn't say
it's "known bad".
Not sure which forks you're talking about ...
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Sat, Dec 2, 2017 at 6:04 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On 12/01/2017 10:52 PM, Andres Freund wrote:
On 2017-12-01 16:14:58 -0500, Robert Haas wrote:
Honestly, if we can give everybody a 4% space reduction by
switching to lz4, I think that's totally worth doing -- but let's
not make people choose it, let's make it the default going forward,
and keep pglz support around so we don't break pg_upgrade
compatibility (and so people can continue to choose it if for some
reason it works better in their use case). That kind of improvement
is nothing special in a specific workload, but TOAST is a pretty
general-purpose mechanism. I have become, through a few bitter
experiences, a strong believer in the value of trying to reduce our
on-disk footprint, and knocking 4% off the size of every TOAST
table in the world does not sound worthless to me -- even though
context-aware compression can doubtless do a lot better.+1. It's also a lot faster, and I've seen way way to many workloads
with 50%+ time spent in pglz.TBH the 4% figure is something I mostly made up (I'm fake news!). On the
mailing list archive (which I believe is pretty compressible) I observed
something like 2.5% size reduction with lz4 compared to pglz, at least
with the compression levels I've used ...
Nikita Glukhove tested compression on real json data:
Delicious bookmarks (js):
json 1322MB
jsonb 1369MB
jsonbc 931MB 1.5x
jsonb+lz4d 404MB 3.4x
Citus customer reviews (jr):
json 1391MB
jsonb 1574MB
jsonbc 622MB 2.5x
jsonb+lz4d 601MB 2.5x
I also attached a plot with wired tiger size (zstd compression) in Mongodb.
Nikita has more numbers about compression.
Other algorithms (e.g. zstd) got significantly better compression (25%)
compared to pglz, but in exchange for longer compression. I'm sure we
could lower compression level to make it faster, but that will of course
hurt the compression ratio.I don't think switching to a different compression algorithm is a way
forward - it was proposed and explored repeatedly in the past, and every
time it failed for a number of reasons, most of which are still valid.Firstly, it's going to be quite hard (or perhaps impossible) to find an
algorithm that is "universally better" than pglz. Some algorithms do
work better for text documents, some for binary blobs, etc. I don't
think there's a win-win option.Sure, there are workloads where pglz performs poorly (I've seen such
cases too), but IMHO that's more an argument for the custom compression
method approach. pglz gives you good default compression in most cases,
and you can change it for columns where it matters, and where a
different space/time trade-off makes sense.Secondly, all the previous attempts ran into some legal issues, i.e.
licensing and/or patents. Maybe the situation changed since then (no
idea, haven't looked into that), but in the past the "pluggable"
approach was proposed as a way to address this.
I don't think so. Pluggable means that now we have more data types,
which don't fit to
the old compression scheme of TOAST and we need better flexibility. I
see in future we
could avoid decompression of the whole toast just to get on key from
document, so we
first slice data and compress each slice separately.
Show quoted text
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
Screen Shot 2017-12-03 at 11.46.14.pngimage/png; name="Screen Shot 2017-12-03 at 11.46.14.png"Download
�PNG
IHDR � � �;�� &iCCPICC Profile H���TSI���$$$�@���)�����FH %���bGX *"X�U���"ve�}AEEY 6T�I��_9��3o~���;�;o��7 �F�E�,T
�la�8:������$ue@ `�� ��/** �2�����Y{�V������sy H�T���
�0 � G$��� �&3sE��P%�C��Me��`7�*8\� 9 %*�-N@E����I�qTJ � �!�&�� >� �3�1��3 �ZB�L�.N��b���d��GX���(
$�,���s9�w����a+�/���,[��a2�B>/L�������+����44n��G � �g�R���0�z����q~C����B49� ����
�3����������8%| k��x���a�4A0 2|�h� �;�|� >�
�{���������#sI�e��;�@�d8�4M ���\�V��=<� ��M����!g�$���z��� � ��'�����r����w������&^V��n �M�3<�/n6E.8�`O�R��k�r�b�p& 0��T0d A[o}/� l ��l�,�#�=B����OH< �/���<h�2bU<mA��7O>"<�������' ���:�n���8����� b 1�L��.(�� 80�,X� �<��L�pX��8�'��C�uB'�6����� ~�&�M�0j�Pv��g��C���?��C�8���8�� �ss��o����K�U���(y��l��������Yn��T�J �$`������� �� =�e�!� v��5a������V���G��c���-Z�'� �������an���b����Y��'`�h�X���e�����d 9vc���� ��~����!?���o��f ���1���
���O ���f3y
?�U k�H�y
.{ ��/E���f�\�'�A`�� L����P�L0,����@����` 8�A8 ��K� \w�^�/@ x!!4��� ��b�8"n�7��#�H���#BD��E#�H)R�lC��_���I����F�� �5� �P*������X�
�C��Xt*������tZ�V�{�:�$z ��v�/�~ `�3�l17, ����4L�����2�
��5�7}��z��8��L���P<��9�|���w�u�i�*����_ 4� ���A` ����BBa'�� ��� ��D"�hAt��j1�8�XB�D�%6;����$I�dC�"E���\R!ii/��
���AIY�P�Q)X)YI�T�T��G�����Jd5��� I��g�W�w�������:���E��dPQ�)�)g(�(o�������') �*�+ P>������A��P�P���]�f�m��fN��%�ri+h��S��*t;�
We�J�J�����dU3U?�i���e��T/���������j��*����T�W��;�G�g����Q���L��a����X��]���#:F7��9��� �3�nM���&K3C�Xs�f�f����8�x�YZ�Z��:���bd1V22n0>�� �7�7j������z�=Z�W��]�]�}]��S'H'Sg�N��}]\�Zw��L���gt{Gk�� �]4���;z���^�� ��z�z���!�"�
���{
�k
���
�
�k
O>gj1��Y�r�if���Q���h�Q�����q�q�q��}���I��Z��>SC���sMkL�������f�����7�0O0_j^o��B��e�oQcq��f�c�cYey��h�f�i�����v��[WZ_�Am\l6�l:������s��j�g�g[c�e�� �+���{9�tl���c���j�l�e���������F����� �J�kN4�`�N
N�������<��3�y��R��/.�.b��.=���)�]o�i�E����w'���/por������q��/O[�L�=���[����1�����k�W�7�;�{�w����������/�w��S?+� ��~/�����G��x� h�C���4���*� ����8�� i%�������gqX��� ��M8 F
� �{n .o��N�0q��{f���H��\y?�"*'��I�IQ�*'=�v��}.� 3=fO��X����w�,��q-���S����'&�&t&�M��x)I7I���LJ�O���?9h����S��N�1�b�����N��vl��t��C)����=)����*v*+ucj '�������]���y�JyO���J���{��I�������� A��UFh��������2��j���S��
5����3 f���!��:s<r������;%�d��!W�d�J-�?I����*�>���yh��,��������~� ��� |gN�\����v����m>2?u~��Kt/ Y�{eQ��� � J �.NX��D��%�~
���P�P\xs���-��e�em���oX���[t������s ����?��<�"mE�J���WW W�X��zw�zi~��5���e�-Z�v��u���mYOY/]�Y ^���t��
�+��+�+k7�m\���&��+�}7�����x��������l��2�*�N������� �~q��z�����_v wu���}����z����5h���g������5������Q[| � x�k��7� l9�vh�a������!u�������
I
G' mi�l<���o����*�i [y�r|����'��E��'�O>j��r�T��k�'�n;v���������;q��|�� G/�]���r��������� isi���z������c|��+>WN^
�z������;n���us���[�[�ng�~u'�����������/{�����?j;]:�uv�>�yx����������<�=){j��������������w���-�S���/-_ ���������W�W��K�����v���������
�/���a�G���>%|z:0�3�s��/�_���� ��l�� +����] ����C; ������ �����b��M^\ �� @�B ��?�fX� Sa+� �����H *�4'GE,*��> ������������/;��� 4�(���"��n������?.w�aGo��T pHYs % %IR$� �iTXtXML:com.adobe.xmp <x:xmpmeta xmlns:x="adobe:ns:meta/" x:xmptk="XMP Core 5.4.0">
<rdf:RDF xmlns:rdf="http://www.w3.org/1999/02/22-rdf-syntax-ns#">
<rdf:Description rdf:about=""
xmlns:exif="http://ns.adobe.com/exif/1.0/">
<exif:PixelXDimension>1926</exif:PixelXDimension>
<exif:PixelYDimension>898</exif:PixelYDimension>
</rdf:Description>
</rdf:RDF>
</x:xmpmeta>
�$�2 iDOT � ( � � O*F��{ @ IDATx���\U���zK ����*��w:AA��&�6TTl`��O��� ET�� ��� Z(�����3������������ ���{�9�s����s�q n
(��
(��
(��
(��
(��
(��
(�� +0�`�c���)��
(��
(��
(��
(��
(��
(��I�`����
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=P@P@P@P@P@P@P�`�9��
(��
(��
(��
(��
(��
(��
t���p���S@P@P@P@P@P@P@ ��
(��
(��
(��
(��
(��
(��
(�@� w� �=�O����
7�|s�����y����2��q�����G��[��� �w��������w����s�9gXp��A V]u�A���n����a��i��+���rK:���.�[o�����7�$ EP@P@P@P@P`P �a��
(����^�8��0i����[o���Z+ r�!a�� a����n�A��7�|3\{��a�����Os�=w ��� ��� ?�mz�y����z*���? g�uV:�r�->����c�=���wP@P@P@P@P@�A$`0<���*�@u���6���`�;����`��v���
(��
(��
(��
(��
(0����� �I���p��`��v��9P��� <L�2%��uQa���a��7 ,�@�c�9��f���
�;������C���6�����0�LP@P@P@P@:^�`����*�@=�����g� ���������c�3r��v����n� �����7%���k������1�����VX!|��_
�/�xz�to ���M7�N9������{/,����+_�JXv�e�\s���Sy,P@P@P@P@P�� ��j8l�
�� ���������3������s���f����Z���5 �k0��a(��i��?����o��}�C�� .�F�J��n�P�G��� |pj+����Jj����������=P@P@P@P@P@����m��
"�W^y%L�6-L�>=P�8�����#G���g�y�D�]5n���= �����}��[�NP@P@P@P@�,���O{��
����p��
��ok0�|S���
(��
(��
(��
(��C�`x`���T@�V�`�uCc0�|[����zDP@P@P@P@�c�l�
(�������� 7��#*��
(��
(��
(��
(���0 �d+P�L�g �,��^{���9sf�s�9�3�y.0��^8��l����7�x#�������^J�E��w��������/���-�s������ x�����0c����������;,��Ba��a�%�H 2dH����'x��w��������/�/c��b��%�\2�7�|i�6+�� s�1`,h�\s�X`�4���?�1�
�v�i���O����� ����\F�������V�Gy[��s[q�m��F�:���r|p�s��2t����RK����5c{����|y�����2g���u�EI/�ot ��6���
(��
(��
(��
(��� 7n�
(�O� ��M j�^x�����_~9� -����5r����r��|�)t$��8�~p|�-B��/�K�3_}�����s����+m�mc��MaW�����V������'�|2������?�����/�xj���.����� �����j��m�Y
��y������z*����i�y�I���K/ V\q�l��V
������c�����
i� ���8��<�H�c,h����S��v�n�h��x<��c��K;���� s������� ��G����J�c^�c � {l�s��pN���x��y �����_�:����2����3��J�
n�?��v�W�SO=5�O���W^9|�[�J!w ��V[-�E��u������Wn+����{��������u���4f��v�^�����>���H��� ����vX~���gm[e�UR[y?s����o�������lJo,�&_��5|�����������<�����`W�o������\y���������"�����P��V
\��#����6���c�p����S�� �c����������qS@P@P@P@P@��0n���e
(P&@�B@3i��p����P�P���W�Ax��3&����a� vHAU��{�����
��sN����SEP��O:l��v)���%�������>�������+a6���g���[�8��B�_~y����������4l��l@`���]v�%������������V0�����G }4�r�)��oLSn/������6�lv�m����[W�\%<$ ����oOM�_�����?���]�����%|��Ha ���w��]�h����S�M��83�������|������l�I�s�=���f����'
�����O�S:'�@�� �<Tn���:�# �HB�Ja;nT3������Z$<�z�������e���]7���� �\rI���������t f�<W�/���z�5�Lme����m����+� �� ����gn+�����2A��;���������S�y/�����-��k���L��u{�QG�~1�����a��q��|���'�tR��������P��k�M��y�g ���������;���c�`W���u������
����� w��nL�����c@��Zk��m��V��zM�����
(��
(��
(��
(��}'`0�w��Iz @ AB`D�HJ�G�,AT�-�[���^
IX��*�J�����R'a�!� &L�����m���[RhLE]o7B��� ��mT;�s�=)����;S@L#!W����� BB�
7��bxV����R0L�����������?Ul~U��������M7�4���>������ ���K_�R (����HcB^ov�5�$��O?=�l�
���������R ���
�n�!�T�S�
9�,��5��~��)� ?~|
qd��b8W�2� Y ��?\39.?g���� � �' f�`A����������n'�|r�}��S���a<������R�1m���b����V*~7�U���� �������j^/�M|���~>����L�kTD����<����Mx��Q���1��_>��k��rmP���_�* ����p.nZ�����Jxq�J}�������k�����f��+��\k�������e ���?��|�0&n
(��
(��
(��
(��
(�~��7&�H������2R)���%�a�]* ����TI�����U�������z�a+���|�+���5��o �Zr�M����WT)W���/���\g����O��%\"0�����'�k���n�m�k���2��9�����E���������R�����f�[��JZ��P����joL>��O�6��Z��2� w��)i�-��2y���_���5���`�c�"� ��Q�K� F{��?�M�(cG�-V� ^^�� �9��1?��#Fl�{�'O���?R�� ��/T�S�����=�v�� � 3�m�����6�3�97KVS9?*V��_7����H��M
,;��A$�$�;�,-Me/7+��-< >9&�`,��9�|a>����KL�n~�F[Y �Jr��m�0���x?�$|�<�\��|��/�������O�n��s�m�-�H.��z�N� <���3� ��zkj��T�2�X1 ���������y>�Kn���\?�pn��y ?>W��Q�NX���{�
x>#�m�N��ESYM��M(�8�� �cN� ��#_�pS���o��E�;���)��
(��
(��
(��
(�@{ ��x��"��*���SpE�D�EAE%K��Z�QMLpK��� -�4�$���]�z�a���%T$|�w#L!hy����sB �����#aQ�����&8c�Y6 �A
�PY�sH ��x�)!����l5�����*���^��k0L;���h&�%|���T���YBo���e��3�I{Y��`�<�#�$d%,���KS� �>��� ��)P��%�O���Tpr^L��=��#�
�[o7� ��,;L��X����FR-J�MXJ;X���KHG��F������a�� ��q �^6�K���� ���%A2s�k���|������,��y��f�\����3n� G��FH�\��l��g��VBo�e6��5��M[q"$�����C[��:���JU�V�_��y�����m�:�� 7"�/����3�� �5�w�5�?�}6�����>��r�|������|�;�I��1720����{y�0�a�m���s�p��6V �f� s�ok�)�1�2��"��um#�� w�q������M���F;iA9������K��}����QN�5�P~CG�9�^P@P@P@P@�G�`��=�
�!@B%��G �BN��&x^�G?��
�:@�6� ^*�~����� :���/�P�0��Vo0�9�x^l���z�J&�"�����Z)l���� �<��w ����� �0��eh ��n]c�5R�E�E �>�,���i+�����@���@�[�`8�J�|��FSM�����p���0��"�#�b x�4A�I8L��Ka�P�1��p����Q�H �������1~��+K"3`�����REs����9�)�"���|9�#��z� � �s���F����x�+�c���`���
\Fn `�����,�M��-��y��y�y��f�A.�.�����~^����3WD3/YJ� �y�Fu?}��H���V�
�o6BW^�'m%����y~�V>0z��'�~�x�)m�F���P���_��o~���������u�UV)���7W�3v\��X��\�3���X��kx�?���1f������i���`���<�����|f0��=� &�D@x|���*qn^ac� ��9�k������82GO8��t��E ������{��8��a��Y�� ���9M;1�3�6�)��
(��
(��
(��
(�@���sS@�N %E � ��GE ��0�W 9��N:��nE L*�;�E�B-bi� )cK��18��O �]v����Gzo ��SN9��:uj��7��1@)�:��"f����#�(b�]����7�������N���_ �R�b�kj���a�y-����"��E��+b����ASq�Aq��������=VU1�+Yg�D p@: ����p8���������8�C�1�������1+�?��"��E �R��^{��U�<��1@+��!i��� s-w������+&1�,h�vM�a �*�".C^�@4�y�S�9b@Z�e��|���}��k #��~����������1� S�0>��|��E|nm���}� 1 .b�[�P�t����t]���t�J���4���6�����q�j[��-�2��k�~�*���x�@�m�71�/�5�E �����j�L?�Af�
m��p+�S���� �'18N��s-V�q����Tk�v�]^� ?���X!���|��Y����3.�/�9�}\�+.�>+8?�U���?���q������"�`Q�}���_��.=��"��E ��|�z����_�lR�f�1� ��s���X]����m�Yq�y����J��3P@P@P@P@� *?�P@�� �!8�����P&.[<��)���hB����')�! ��d�XYq�V���p9.�:���X����������\���<��P� ��c��� � �r�Dh����kC��-��a�+SX���yL�1]*���V���w�9L]C��<�b��7N��'���E���z.�bg���|&�?�'V_�j��~��"V����/J� �# ���]w]]������c�e�8��V0�>�q �4_b�o���Q�����<���`9>�6����`��7b%k)pe�����}�g����'�X������]w�Us�F��| Bhn$���c@p�M ��|�3 �s��!���Z0��!0���X=^�K�� z����b���N�0m���X��nt�&����N>��Y�k���.� �����}O�����s��rN~B{>��P@P@P@P@P�} ��g,l�
� �����jY��\n��(�l�a��vZ���}/�g�����"�/�fT�\��p���q��J�I�#@��-U�R�����E ����9����[���K�T~�S��Zq��q���R0L? �h/��V�m �*[*���S}H0I%d|^�,��Kr��JQ��WBp*��m��j��,o�|�����J�j���s�3<��������0�k?*�����!>���K �7�[+���� rH
�� 3 ��R��N3�� �f� ��z�J�Z[O������E\�=UQ�V��k��q����3�����q�jO�a>K���7X�s���6�+�e�K�2�q����j�0��>��\�n��g��p�a4�"�0U��.v�^>'�,�KS� � �� ��n�x\s�� B ���yC�_�1�;��W@P@P@P@P@�� 7���(�@�r�A��C*�m �X^�%Zkm�2��MK��S�L)x��t��_[ S%K%-�sI4=�l�I����6UW M��<�" ������O����^�;�3��O���p���P+>#��ZAX�o��W
� w�a�JW�c��3fO?�t�9[�g���j��*i�B���|�����y ��T
��J}f�[�"i#s��� � ����.� �s7>w5U���s���j��Kl���6�c�
�����T�rK�3�������d7!o>s���[ SUM��me �_~��j$�� �W�VBT�Qk+�����R��8�����g�]�2�:f�vV �6��Y��`�|���y�B���g�}�IcX�u���,�� �������)������b���T=�.K��_��z7���L����\W���MP@P@P@P@�C�`�=��V(�@��c�=6����8l,/Me)!!�L��I��� 5;���@����d
�h;!!'a7}�M�6�7����M��<�3W�U����p�<T)����3O ��
� �����a�#�<��<yr�[�����q�����L�� ��_�bzl�f���*y�/��o[?����>������� ^��7���z��Ku��~^<���]��md#���T��c�
�9v�{qg�`n"�j7����p���>���� +~������V�T��������F��s��J���OV
���R�q�`�yC�J����u}��TP���n�����o�����O����� �;�������
����� ����4��F��{P@P@P@P@P�1����|�
�� aC���!!A)K l�P=����= ��f7;f YB#B<6���+�AW�� � <�T ���Z�4�v,~~�q����6��-KYS����R0�xQ��D ����fI��<W����Y:�|������n�%~��g?�����n*k�{�r�B����R-���|����������KA �����[���vW������p��: ��07�1m��j�`�k�BL ;����`��F�O�?7��sR�_���T�N�0��d9��<+��5\-f�'M���
]�
7�����z��������g�S)���C�6�e�?\�<�����z����5���k������
>g�.>���
(��
(��
(��
(�@� �,� �P@����)�!,��3� q�����X��s1C iB\�4�g������K ^�9V��nu�7���ebpbj��j
1� ������b�������/Nm~�����b�b�����C�X��T�z�����lt���rO�Af���
1� ����>�~���r�!V>���*������Tk��~���{��C ���c�s��'?���->�4����\���!V�����w���7����z�y���{�]\B7� /�g����a�,1� 1�1�*2��<�����[���p.�s6��Hq9�o
1x ��
���� ������1 ���B����8�5VX���qz�����1/��b������z��\��vV]u�t
������c8���C�I ]������(��ty��*���2O��5��i�bus��fj7���q��o�1�t�����x������+� ����m�at�k���!V��c����X t�A����q��~�����1�N��~����| �o|n��x�K���w�l�M8����Y�Q"������9���>Cb��������x�E�7B���� ~8\z���3���M#��%>8���J�����3����@�s�C\�:�g�y����YP@P@P@P@�^�`���=�
4 �O��eSx�N��Q�9T\�4��B�����!V���� ��P����goF0L��.���@���>��A��5l����-Q�C���BM/��fo�'.� �b�e��5&�#�����|��F�F�F�LG� �e��0l���� � :V*����+������c�@G�E{�b�g����SO=5��m�z8�t�I!.��n"���=�cE����|&��{��g8�����l���t� w�p�����`�P�������+�����9/�p�|��`�����������V��sBl������ � n�k������� ��O~���9��b��x��i�0�������p��M���T �?�����z�����-.�>�� !� �v������)��f>�� ��� ���u��`��C
��?p#��
(��
(��
(��
(��
��@�GU7P��b������_ZB�%JY�e�cCj:���\�q V�Xq����5c S���XJ:G�<�����������k ) ��[����Dry���Y/���3V���������4��3|y�j5����`������E��N�X�Dm���m7���E��-��>�E FKK�c����N���3�y~o �f;nO�\�\���_�<����hY���
�{��������u��4���g���7K���������K�S�[���{� N���f�ce{ZB������P��on��)S���%��_�������v��V-%M��k������Y�s�g��{6�jf�v�8�mg��X=[����7���9��t��}��s:n���a���'��� ����Sl�+�<�7���Sn��{����#�8b�%�����
(��
(��
(��
(��
(��V �uS@�����i`�*}���R�0K��L3/����0�'���0#US����T����&�l���.�����b����n�-�~��!�!>�8U���� ��Yf�Z���������Y�O� KIS�H�i�sQ��2�T0SQ���Z���^�*���snC ��,�K�(�,c���]+��a��_���jw��[lXz�%�Y��jS�C�$�������!gi^0_��� *FYZ�����������2��n, �I�3�C|>r��V�0}��yCu+��SaKe5Ki���QMM%+�*/��2�,%����>�
�V-%M[i���o ���T ��L�ky[��������G�Ff*��hk++��\���j`�������������������2�w�}wj[ ��g�^{�Uq�j,*-%� eIh��F�#����3�r���1o�@fyk>O�g����SSu=��2'z�Q��g ��� ;���sP@P@P@P@h���p=�
�� �!1�e T��Kh����{#��Xu�������4M�a`o�aB-� B*��e�b��XM��G��F���KI�� � v���A.�E`���+����g�V���0�!�d[�j������f��l8����<��3��qY���p�
)( 3fL�w�}K8�q� ������S����j���O|"����������9�aK(�|ec��o~��)�&��wc��r�-i���$p�`�k���g��T0�:K�H��W���;�����,��3��ake0�5L�x�6!%�-�"K{�vm+?��0K#�����_��a������������<�h�������%��k�����3��k�]�j�0�;�*fl�!�v��v�����.�,���e)i�d'�3���%�� e�p
�Bg���n����X���������P@P@P@P@P
�
7P`@ �lj ��I�&��hY.u�
6HKI����Lj��-b@3����=]J�%{Y�5V�n�i�\,� ��"�t�|�<�~e�O}�S�%kY�5��E����}���KI�?-���<-��hC�����_��2y�j���n u� � ��GZ�6^i���Z��e�]��|���o��-b��~�r��Ki��Vl���iie,x������+�t7t:� +��X)����b0\|��_��8,G �����1D,���%����gE��!-��R���u��~�\1lm�R��7>:�Q^v=����jZ�����.ch���k�����1mm�R���qe���#��e~����!w��������^���<��/�>'b �3�W<�.%M�bH[�gn����C7�3)V5���������:�4��,��F.bExj+��y��n���P@P@P@P@P`�
����
(P l�K��P&V��=�Xz�e��KqFxni��+ �t=
�cupz�(���#`!`����R��|C��3I �8 �v\V6zUv�� W � �zd_|���9�<�����\W��Zm�9�g�qF)���)@���+��sq�����p<B�����`�R0Z� ��<V7�F�J����m�Yj ���-V��Pq��6* �R0L?��q���lfBV�K��s���<��XY���j����0�+��e~�J��\c�+�S� �q���%��X��U�Z)T��`�� �J��3�� ����
�*��L�X����L��t��V-��T��6:ob�y� <�[^����tz2s0. ]�*�"V1��s��<t��MP@P@P@P@:S�`�3��^)0�� ~�����o+xQ�I�U-�*�4a%��_��Rh���
7�0 �N�[O�a��� �".W\�eoS���.c��d�'�x"��������k
qr��r��
�P���
���l��-�?����"���JA[��-_�`�>>[� ��w�=3f�(~��_�
W�A��%���G }�������^[PI�#n|=��S�x��K����1V_����������z��_���)���\|�t�V����>��O��E|&t��x��a������<0����H_p�E|np]�
����-���q��RS���7��z�������{��h+�k6���a�+p�s�}`��gb������;���`�<�"��U ���������">S�����.>s��K����:�p� ����#.s]�e����_?������x���k��y��9&s��u>g�JP �o�8�t�?(��
(��
(��
(��
(�@� ��N�P�%��,}��~��a+� �E���'��U�& Yk��R�A�Q�5 �\������os8D�2�,m���ee��? a����s�S�"U��n6��?���S3���y�� �F��j�� �^��%��
���@�� L*!A �aY����*�L��������sc�g?���\.A)�cIiEn2h���iK�� %������*`�9/!9>�'���� �^�f��G?Zz ��,G�?c]�F���[:F+�aBl���k��J��Zx��������r ����W�0����.�l��� rH��0�%��k,�LP��3�������������<������*����.��s����� ?n�v�m��\�t�q���K�����z�I�E]�*���!�fI*�����9��Q@P@P@P@P@�� ���#(�@ Jx�v�m��QR� y&f�e�k�����m
�<r0�r�O�j����`����xY.��� ql��v�i�
��f����pg�5�,-�Lu�� �~^����Y��l�M
��XR�g�6k� �A(��&��nc���N:)aY ����V���.�<� w\���� U�TS��1 Xyn+ayw�����~O������RhH[�yu��k�>��ss�&��a����|����
�sT� {���cp^�N�Za2�w"q����1�� ���4������&7r�a$�% ����3�q����j[��M'�t��s���?�� �=����{|�o=��_���K�5.���c�����y�
T��Ln��{��=#.������q�k��i��3�?��N��P@P@P@P@P�} ��g,l�
� � �W@�ReH��]�Hx�R�,����9�e)�s�9g�j�z�a�*MO<��b��qi�^B1�����?u[�Z����Rmw�G�y���h����}��G�5 f���v�--�4*Y�hm�V-�|g������ g����e�]fY����R��9� 8yn-��~��K!�"�,2������y�<�������j � ���by���*x�.U��<hc������-� _+���,���;���\������RuL�n>_o�a�j���^zi���<�� ~x��0me�r�+���M0L���gJ�#��T�r�B>?sm��a�?���}�s�����V-��|v�������*_�[mc%���c��@�� j��g>u�#� ������%���z�<��� ���sv��V+}����:)8��wS@P@P@P@P@��0n���%
(P&@��s<�B��Uo<o�7��Mz�0U�]��6BB)�2*xY&�c��� �[o�5���ti�jBIB�K� �I�%�9�_������+a���o��U���������y>'�W���]��@�%�y�r6��#4���ky.3�`���$d�����|]�a<r����.��&�<��� \Y�GpO(E�+A?�#%�����@��z +�1%�$`��'?Y
����_��<���
��Vn�_��������p�;��� T�2nx��s�������F,�) W�V
�_��b��I�,_��b�0'���,0�f ��\c��q Lg��Z�d��a*�������f�O��f��| n����U����
�o|�7�c�V�:�w����4n^E�/����
�|� �T��6�%�?���g���4�ho���9 �b, �����t�p� �\\�1~��R���c�Q
L��K���o���t�D��f?���������/O�Ms��k0��@���Y�:{r�����rS@P@P@P@P@��0n���%
(P&@����OG }t
8r�BHF&� -��%X����:~����e� ;��T�}�3�)qe��;&��� �%x!L��<u��{�=aa'�����
�~pN�4 T1��l�E�DTR)�� ��2�T�]p�+9{�F�� � ><����?�}��_LA7���N�%�gy� f�O �_���)���������R��q��0#@�T�Z��y�r�e������ApF5/��,�K�� ����bf�k�O�O �C;�].�w��fY�|.�:n*����jt��������,��b�� ��Y�`�yF���
]7BE��>��Cgi+�.�v�m5����� ��f�l<r��uA�9���u
��_~��|�z�j.?.��f,�����u|��j� �~�������Py� �0���,>7O���1gr���K�dC�\ic<XE�0?����?T�s� �"�:g��\��\sM����,��o8/f|�7�uS@P@P@P@P@��BS�?��)��m'��p�W��4j��T��F��E��a�t!V��@���X]�~����B �B �B �C 'B K� {�b�f��l�� +o�B3�J��l�����C ��8��(�-�
��-bEr�AY:��!U���!�y����M �B F�&�lb�4�n|D��0����&�Z2����_��k��a��1X
1KN1d 1�
|��j��M:v���j6� )�}����x]{��!V����
�+9�y�� {,�p3��.�7Vk�:���/m����~������;���o�nq �p�y����va�;����Ms�qk��X0�s�C ���w��N�y�'���������L�����k�
qI��4�cUu���Y���3���N8!]/X�3���8.����������=�Hb ���8� 0�
�� �+N��a~�V<�� �f�NO��������^�g��>�K��8N����L�4��+������;�}��h+��?1h1�1Mme~�<�&hk OS[�/�$� ]��QO�����������#G��:������q]�{������� ��y��O��5����gB��@:������1b.��X�0�{�S~�3~ ��s�� B\:>�����������Wc��?}2���g �`L����_ �q����`9�@<���C|&|�������
(��
(��
(��
(���0n��GW@�^
��"V��X}�B���$�",&� h$d��m�| @ IDAT �F\7����a���)t���_�
� m~�� �v��' �'_ �Spr�n�:/��}���??L�<9�J� ������J���"z��^S��� �t�.�t
� �8����C�4 �*6�X�6��->�5�J���'��R�l�X�g�v#�"�#����?^�y��!D#��K��X��B7_��Zc�8�g4�8 �v���:����P s,�#a8!-�#P�� @$ &(%(��04�����[
3~O����K����X�����������7B^�[n�`.r]� ��0Vr������s7c�V�8�%����X@���f~�V�l�qn��g�*9���t=�V~F�1���h7}f^���b�oj+a+�.������Lu�K�����_��q�����;���63���!.�=K�^���R0�}��g?{n�a��S����x���S?V\q���j�a�����_}�U�!.�?�<�_9Xf^����� ��8}�2������
(��
(��
(��
(���%`0�^�akP�� � ��g���5B?*+ %� ! _�x��J�F�A���F�jV*��+m[�"q���LQ]�����
����'��2�!> v�`$��'_ > A�����6� nN=���<���)&���
p��T<R)���o�}�����!Q|fl�"�$ �|��S0F�FHK�G5#�e�����VB�Q�F������S`�h[�:*��Fpu������q~o������s2������|� x�0���g�}R�K��i����� :��=���gk*U�q)�T�KE-a|�>f{��~�<���<��3 � �sw�����C*I�F |�E�� �s/��s�k�y��A�\�A�q\9��<l��.�J[ � �> ����8��|�K`��r��y�*}������< s[�����R�������O}�S�c.��F�l=c��b�������\rI���1��=�����|�ss>���08��qL��sq#�p��y�r�|���J���[� �� ��s� P@P@P@P@����q�,
(� �BE*�n$������B Bc^FT(p��lm|�fX}��S�#a�[��JD�q |�����/���I|~o�p�.p��t��v}�� [m�U����~�G D��<�T�2eJ
�X����
J*2Y�*���^;�k���;��p ���v�r.b~G�7* ycGUi~/����jC�<�f�b��'m������M?^ ����OKv���t�u�� 2��b��Q
O%,� ���;����d��BPK�8/�Y���?,�u���T}^z��)0���6*o��}�G>��4���3i�����[�`���:uj�le_�(�e���1� �nN��m�
��D� s��uE�������K(L%?��[�`��� @������������V��9�'�6��3����F�������z6�� ��D�������<�H�������Jk�K@�g 7��3��z6��g-���>c������K�?�"<�3��^|�p.���e^2?�}�I=��=
(��
(��
(��
(��
(����p�F�C�D������@�����@�`�@���p��B���<������% �X-��+ 7o^{��)��ZC�������K[ ��i3K�l�0nJQuK�9Z�aFh�7�B8D���i3���CBd�R��]�k��� ��t#�'����s0'�y�.F��1_�7A99 �U��j�p� ��<����W��!t��q<�q � ���Vxq~���2G��@@�J���C0L�!� A!�&`�����5��/L���*S���;�\\?\O<G�y^ TD�Vn��������:��v�V�Gsn ����8�V���@[� �0�-�c��9 ���lB_9s5��*c���������g��F �Y�:�_�c�;�� S����q qss�����g�s3�#|������
����3O9W^��v��d�q�������
(��
(��
(��
(���-`0���c�P@�y���w^
���p�jl�@�� ,! 7
P]�
������x���T~�����/�)!;a-,�q�S����7L�������@����JEj����R�L{i+7�r# �e3�Z�W��Y <������73�����w�p���}
(��
(��
(��
(��
(�@5��j2�\P`� P
���?�s�9)d�<��*��v�)���S�h0���Tm��U�T�R�IE+�W�������\ ����>=Y��VW�k+�.mn�����">3>���a6���J�����������p���+��
(��
(��
(��
(�@� �[%�qP@�T��K*G�� �g�� 1a��{��*8Y��R�i�6��u� �:3���P�s�� �h��w����f��z7��z�|�
(��
(��
(��
(��
�F�`�7z���
(�/�9�T�R��W�k{��W�j�;��3=3�g6 x����T�V�6��x�#P>��X�������6y�����~����mMUu#s�`x�L��
(��
(��
(��
(���0 ��g�P@��)�����MK�E]4���+��[o
� w\�:uj�1cF
���z������)( �Z��7����^z) ><���k���o� |x��GS�0�g����>���ly��l �f��WP@P@P@P@�0�W��)��
�� �o��V���KSu0��<G��_ <�@`�_�6*����o�]w�5� ;�-�n#���o�.����,�y��'U�3�X���� ����
{��GXk�����p�d���
(��
(��
(��
(��=0���(��
�� �0���~����o}�bCF�F� ���o�M6�$ :�������\;��S��G Y��T�������_�z�b�-��a�*���
�k��;P@P@P@P@h���p�$=�
(�@�����NH���y���<v�
7 � zh�t�M��K.��W�&���}�k��\[w�u�a� ����K/�����`�Gl���
(��
(��
(��
(��
7��P@�W�p������w^z��3�<����*x���K���o 6�x���"���vS��,[��?�1 {�������Zh������jt���J���=�k�= �S@P@P@P@P@�F ����
(��m!@87e��p��7�^x!U\.��a�%�k��fXy�����#���6b` 0�����p�
7���3����4�xv�*��_|�^u2��p�����p0��y^� ��c��
(��
(��
(��
(��
(� ���_P@P@P@P@P@P@P@�0����[
(��
(��
(��
(��
(��
(��
(�@0�~U@P@P@P@P@P@P@:T�`�C�n)��
(��
(��
(��
(��
(��
(��Y�`8K�UP@P@P@P@P@P@�P�� X���
(��
(��
(��
(��
(��
(��
d��,�WP@P@P@P@P@P@P�C �;t`��
(��
(��
(��
(��
(��
(��
(� ���_P@P@P@P@P@P@P@�0����[
(��
(��
(��
(��
(��
(��
(�@0�~U@P@P@P@P@P@P@:T�`�C�n)��
(��
(��
(��
(��
(��
(��Y�`8K�UP@P@P@P@P@P@�P�� X���
(��
(��
(��
(��
(��
(��
d��,�WP@P@P@P@P@P@P�C �;t`��
(��
(��
(��
(��
(��
(��
(� ���_P�m�����{���)��
(��
4&0��s�9��#��s�wE^}��0c�����o����
(��
(PU`���
��?9rd��hn
(��
����p���W@��f������������P@P����!C�� .��r������z���&O� ����0u��zw�}
(��
(��Q`��aa�W &LHGEP@��0����
(0��SO=n����� ��tS@P@�^���_/��R=ztz-�������;�}��p��� |0����a�E ���9��V@P@� ���w'�n��� �/}�Ka����O6^P�3 �;c �� %�?<��� �F�
C�����P@P�UT������[/l��&a��1�>���~�����O��$l�o��z}\��
(��
t� ��L�>=L�2%�"�1� F���]�o
(��D�`x� ��T`0 ��_�
�\rI��%�\r0u��*��
(��=��?�n�����k�
6����0�.��G52KU�)��
(��
Tx�����L�4)<��������
(�@ �1��S@�� ��7�
(��
(�@W���"�YP@� ���q��
(��� wo�;P�� ����)��
(�@G w�0� P@� ��D���
t���p���R` ����
(��
(�_��%�yP@P@�Y �g��O
(���#`0�>caKP��NP@P�q�����CP@h���p+T=�
(�@3 ����1P���M��`
(��
(0H ��@�MP@h{��� "��
Z��A;�v\��0n���e
(��
(����;6�LP@�����o{��
$���4Z�U�A"`0<H�n*��
(�@S �����P@P@� ��� P@Z,`0�b`��� 7n�
(��
(����s@P@h���[��
(0�����&�D�Y�`����+��
(���0 ��f�P@P� �;pP��
(�!� 2�vC�N0����/
(��
(�W�}%�yP@P@����}��
(�@� ���gV@�*�U`��
(��
(PC�`���R@P@�P�`��=�
(�@C�
q�f� ���P�
(��
(�i��6��GP@����@ 9���
t���p���=T`� �!��
(��
(��m06AP@��NP@�v0n���]
b��A<�v]P@z,`0�c:wT@P@�*`0�TN��
(�D��&bz(h���ps =�
(��
.���5��VP@�W�`�}���)���]�`x�� ��@
����$P@h{��� "��
(���D�`x� ��T@���� 4��@� w��?P@h���p+T=�
(��
(����p�f���
(�7�}��YP����|�
(��
(�?�a��
(��
(� ��1�BP`v���M��
����p?��W@�0s��0}��������"�,�^z�0������
����p��
S@� ��1#L�6-<��sa���a���a������E�{���7��
(��� ����P� ���S)�@E��^z)������� o��fXw�u����K,Q���Ph��v������^w�u���.
c�� �o�}�a� �ux���
�T�`��� \P����sWh���pk\=�
�/���s�=w�`x�%��� �S�c��>�t
(�R���I�f �'L���szpP�Y����8
(���0n���S@�^ ���(�@/^y����#���/�<���[a�����\l��zydwW@Z'`0�:[���}/0u��0y��p�5���VZ)l���a��6���xFP��=@sP@�>0�fO��� 7��{P�o��Fx���r����NXj���� -�P+N�1P@�� 7���(�@��� ��C=��/L8���+�I�l�
(P[�`����U@�?������
(PE�`�
�?V@�>x������3���� ����3OXp���s��gm�D
(�@�����~hgn�c��7�|3�=��a��� ��?;7��)��%����(��
����p�
��Q@� ��
(��
(�@������
(��
(�
��V�zLP@�f 7C�c(�@S �����P���x*Tx�0�T�,���a���
T���^{-�������%��g���wC� IU-�,��U�=�w����p���K�S����N���w�y��7 :4t}�/������~�n�����+�� ��5� s�gGm�
t���pG��R@:B�`�#��N(�Y��5��F��(����7�^p����h ;vl�y����#R��������0m�������N�x���`��W_}������@y �fx�o�l�
Tx����-�����������-��"�7���0a�����^�>�h�����
s� �$ 3fL��r ,�@
�K�P���-���
(���0��;+�@+ �[��1P���^z)�������P�]w�0q�����K�J�{��7\r�%��_ [m�UX|���s�=n��������������)��}!`0���C�J��_8i��p�E�pw���&LH��2�?��O�����|�<�Lx���S�0�1U�������Yf���Zk����5����yP`� � `�P@�60n���i
V���:��[���7~��g��n������^HK .���a���� /Y��M� ���P�
(�W��a��5e��p��gVza�hn�6lX
��y��T=���Xj�@x��v �n�iZV����yP`�
����
(�@� ���>���� t��@� � ���K,�*T�����[/-!Me1*�X��M� ���P�
(�W����n�-\}��i���^8� l��w�.�h
�YN��� N�PSUL� {��n�[n��\R����(0� ����uP�� ��|�l��Q�`x0��}V���
��T�Be�5�L����:�����7_�g�yR���!C��s�F:V�`�c���)0(j�<w���sx��G�&�l��r��� ���<��~����R��\sMz��( ?~|Z�z�9���vZ�N�`���=�
(�@c��y�n���>@�
(PS��`����N' p@� �_~�����
(�@+ �[���P��j��^zi���K��n���6�l�^{��*� ������<b����?VXa�����}�|
(����?��P��u�l� ,`0���k� ����sO� ���� 6�h�`�� `��@�
w���-�@�`���k �:uj
��������k�T1��-�5H���
����p?��W@�*`0\��_(�@ ����U@�,Po0���#F��;,-%����
(� ����9P�U���;��3�D���_�n�[b�%����j���RK��[,=���z�j�<�
�0����P@�S�`�?�=�
T0���P��
� }���|�A �� ��-�T
(�������������>�����k�
�?�xz������]v�@H��{��O����/ >����C��y���rH�i�l�
(0� ��x�[�����&�@G � �|���� ��g����Q��
����p��
T@�j�EQ�W^y%�}����[n ?�px����;���^���nZRz���NG[����+� F� i8���V����p���SP�������
(��� ���
4U��`��'� �G�'N ���ZS���P@�F ���
(����a�M���o�����^x!<������|���� �M����o���B �m��6l���a�VH� n���6�� ��
(����:��K�,`0<���+�!��c�� ���
�;d���Y�`x ��mW@��������o��f� ���[�����'-3��O�n��|��� ;��S�&���Uh���p+T=�
(�@3 ����1P���M��`
(��� ���
����p��
P@�&
T
�w�a������W_}5U��<a� <d�����{�L�n���p����%�7�h�p�G������^�Q@Z!`0�
U���
(� ��f(z h���pS9=�
�@�`�h����.`0��C`P��������>��1#\s�5)��c�9���� x���s�9�� �y����N<����k��
6� |����9��H�h���p+T=�
(�@3 ����1P���M��`
(��� ���
����p��
P@�&
T
���9sf��� ����������]w�5l���a�e�����>�l�<yr�����7L�����e+�g��
(�l��f�z<P@�f 7K��(�@� ��F��P���=�s7�W��~���
(�d�J��� R�{�%���/�8<���a�u�I��c��
,�@�k���{x���? ����p�-��e�Y&l��a����U7�� N�NP@�v0n���]
b��A<�v]�60n���
(����pC\�Y�\�Z0L�������M7��~��0r����Zk����
E�z������ <�y�m�
�m�]���>X��Mh���p+u=�
(�@o �{���
(�����zPh@�`�,���m#`0�6CaCP� ����_~9<��C���/�o����@x���K����aB�y��7��T �3& 6, 2� -�
(�@u���6�FP� �����+�@��
(�H�T�V0��[o�g�y&�s�=��\z������ �Xb�>m�'S@�
w��
(0�j����~x��W���X.���� ���z
���s�9S ��"����Z*���a��V
#F��$�]���� ,���
2��A6�vW�� `0<F�6*���������S@��,`0<�G��+�@W�Z�p�{_}�����x��k�����I��� /�B��_<�3�<V ����
�\�`����@P���=�s7h���p�l=�
�'���/����/�w�y�
�u�]7L�81,���� �w)��� `0���RZ&�R��^{m���K���o�}�0a�l�#|y��w������T
�=�������h���p��=�
(�@O �{*�~
(�2����z`�S��g�
��rK��_�X�p�
6{����E����P� �����*�@k����0i��p�UW�5�\3 �rS@����@%���
N���9��Z��0n���q
t� �����3f�� {,�t�M��\h���f�mv�i�0r����� S@��/`0<��� (0��~�����o�������n���p����
7�0l��va��q����+�� 0 e3P@�A(`0<�.+����>B�O��� ��� �������O�qr���a� wL� 9l������R@��0��a�
j�'�|2=�� ��G�:ujx�����-;��s?~|ZRzP�y0�f�l�
(0� ����a�_�`����*�����]w�������#��..���i�-��"U ��:7P�] ��udl�
�+����Ja���9sf���W\1l���a��V ����P�O�W��~���
(��5 �k��+� ���q��
v��^{-<��Ci��^x!-!������W_=����a��!����+�@� �� �<�V�*����/�~��a�y�
K.�d��cc��1�V�7(�@; ��h�P@�r��r
�W@��0n�a�
J��(�r��������WP�] ��udl�
�+�������My|�c�9�=��S@�B�`�-��F(��
T0����P� �����+��
(���0 ��f�P@P�� �;oL��
(�)��2��C�0����+
(��
(�g�}F��P@P@���5y��
(�@?
�#��V@����]��
(��
(PK�`����S@P@�N�`���=�
(�@c��y�n���>@�
(��
(�q� 7�vHP@���� 8���
��A0�vQ��&`0<�F��*��
(�@; ��(�P@P �ag�
(�@�
����.���� |���
(��=0�1�;*��
(��M0n*�S@h���p1=�
4G�`�9� EP@�����o{��
(���+`0��cc�P@��.`0<�g��W�
��pPl�
(��
����p��
T@P@�A"`0<H�n*��P�`x �MV�� �;}���
(��
�B�`�� SP@h\�`�q3�P@����q�,
(����pX�UP@�����SAP@h���[��
(0�����&�D�Y�`����+��
(���0 ��f�P@P� �;pP��
(�!� 2�vC�Nx������ F�� �I]�/
(��
(�@��N� z����:��M7�4�����>�;��N>�����O�#F���_>�7�|�>�P@P@:Y������O��sO�9sf��~��.��}�o
(��C�`x`���T`P ���^������$Y�d
�l!��$�)������AG�\ g�Qpg\���FD@@�M�a�d���}D��_�TMw����[��S�>�����T�����V�������7��N<�D��O�����R��; @ � ����/��$rg�����c7m��e
�N��;�8w��W�?��On���n������� @ X��+����jXrx������VYe�EoH @���=Nw��� �|�����Ov�G� ���t�&`�M��#F�p��
K���P �Ogz���F ��#zs���q8r�H ��PZ� �������1���~{ ����Q����Z�s���?� y���N;�3f����z�E�s� '��t�j=�M�4��5j�qk ��]���\�?���������C�:rN����������]�9]� =������O�c2}nU
U�����>�w������}� _`��P �g@ -$�naQH C��.�8w�\��f���'I z�����iM�.��rCz���A��Zj29l�x��P ����������x��Q��>���5���& �x�O�� +=���lu+c;��skx<�19�;���_�p�-��r�������o� x����n�
7t�/��P��/�9��������z���M��L�� H�N2�������,KVK:�#���h��F�u�c���5|�F��C�yU\�~�iw��W�U������� ��@ �� � [�����y�����o?�=v�����I+}�.9�nzS.�8�\�<�~,���K/u���Yu� fk?Mp<��3����L� �d2+�_G = �����i�QBQ�I�:<��s��Q����:T�����s�9����zn���n�-�pA5�;g� ���@V����8n�8��X�Es�D�^cklwn:����
|��J���^I�s�����B�!5= U��M�t�� ���?��G@��z,�<��������[�����3���>�~����c�A ���o@ 0%�6�O��@� b���
�pnt���$r}5�e��Q��
1\_��1b8O1��R��sz)��q���K��.E�>b��� 1�'��s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C��C&��._L 1 QT����|]�#��p��pD����F��<(b����1�V���a�E;E �q"��\h� {�a�� �@ #��!Q�!��U�/&��(��A W]����]8�b8�hd�sz#X{ 1�s��t�n�IP�� ���"��8�y.�B ��=��}
� H �������I����G GU� ��._W���. � 1 Q4��9��=��9�F:D 7��$(b�{�N�y���<Z! @�� b��d $���dHT}�$r����#�#��w�U��+y�p�x��(������E � y# "��j1l��h���<N�p� �� `O 1l_2� �a�p2$�>d������E�;������<b� G<@ G��pNok��"�{�����`5 �6�^�S�p 'b8��V@ �'���@ �0b8U 2�\u�b������ �p���J 1��# �#�Fv8�7���A�=G�H���F��E �`/�)b8�1��B+ �@ ��� �� b1� ���D��|1y�pDQ�b���u%����E#;���������#o�C�p#XM�"�M��1����s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C��C&��._L 1 QT����|]�#��p��pD����F��<(b����1�V���a�E;E �q"��\h� {�a�� �@ #��!Q�!��U�/&��(��A W]����]8�b8�hd�sz#X{ 1�s��t�n�IP�� ���"��8�y.�B ��=��}
� H �������I����G GU� ��._W���. � 1 Q4��9��=��9�F:D 7��$(b�{�N�y���<Z! @�� b��d $���dHT}�$r����#�#��w�U��+y�p�x��(������E � y# "��j1l��h���<N�p� �� `O 1l_2� �a�p2$�>d������E�;������<b� G<@ G��pNok��"�{�����`5 �6�^�S�p 'b8��V@ �'���@ �0b8U 2�\u�b������ �p���J 1��# �#�Fv8�7���A�=G�H���F��E �`/�)b8�1��B+ �@ ��� �� b1� ���D��|1y�pDQ�b���u%����E#;���������#o�C�p#XM�"�M��1����s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C��C&��._L 1 QT����|]�#��p��pD����F��<(b����1�V���a�E;E �q"��\h� {�a�� �@ #��!Q�!��U�/&��(��A W]����]8�b8�hd�sz#X{ 1�s��t�n�IP�� ���"��8�y.�B ��=��}
� H �������I����G GU� ��._W���. � 1 Q4��9��=��9�F:D 7��$(b�{�N�y���<Z! @�� b��d $���dHT}�$r����1�����g�y�-\��id��n��1�������k�Yy��������_~y?j����"��K���G �k��s)��9�I�8�a[��zG �"i 1l_��f��D ���
@ ���5 @ !�F 'C��C&��._L>��� z��q�����.w��������s�F�t�_
���f����fo�v�q��:�x�]M���(bx���O��� ��\J�rN/E�6b������H��A ��`� ���y.�B ��=��}
� H �������I����b����s�_t������WZ�� ��a�*��g������p;o��;d����{��&L���`�A ��"���D ���j��^��m ��-�R�#�K�������@3@ � "��\h� {�a�� �@ #��!Q�!��U�/& ������Y\�N��:���3��pj�Mu;��s����6a�{��wt�����<yruw�? #��C���E ����s)��9�I�8�a[��zG �"i 1l_��f��D ���
@ ���5 @ !�F 'C��C&��._L�K _r�;�����{ ���Y�6����G�� �vs��C��� p����c�+�PM1��<b8��T+��R$m� �m��� 1\��} ��}
�b8O1��B+ �@ ��� �� b1� ���D��|1y�pDQ�+��._W���. � 1 Q4��9��=��9�F:D 7��$(b�{�N�y���<Z! @�� b��d $���dHT}�$r����#�#��w�U��+y�p�x��(������E � y# "��j1l��h���<N�p� �� `O 1l_2� �a�p2$�>d������E�;������<b� G<@ G��pNok��"�{�����`5 �6�^�S�p 'b8��V@ �'���@ �0b8U 2�\u�b������ �p���J 1��# �#�Fv8�7���A�=G�H���F��E �`/�)b8�1��B+ �@ ��� �� b1� ���D��|1y�pDQ�b���u%����E#;���������#o�C�p#XM�"�M��1����s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C��C&��._L 1 QT����|]�#��p��pD����F��<(b����1�V���a�E;E �q"��\h� {�a�� �@ #��!Q�!��U�/&��(��A W]����]8�b8�hd�sz#X{ 1�s��t�n�IP�� ���"��8�y.�B ��=��}
��r�O?�������#�<�6�l3�����q��-���M�<����?�� ~x������p�
��k��VZi�����z!����G u/���S�r�-�&L���Xc
���k���G����/�����W^qz�z�}��� �� �iL3v�X7i�$ g���>V����������:���(�?b� G�(�a�
�� 1�g��s)��9�I�8�a[��zG �"i 1l_��f��D ���
@ ���5 �� �
� w���=�\w��w�����n���^/������W^�.��Bw�-�,�f]������g�}�v�m��l�O�\MtK0+��7����d�^�K���Zn�M7u��Ms����[q���#B��[�C�=�����k�m������^����{1<~�x�����b� 9r��9��b1������2����)A�p�0��F������ "����y.�Z9��"i1l��T���R$�� ��k0� �y���<Z! @�� b��dP9I�_���n��y~�p_��SO=�.���7 ,����a������s�z1��c�yQ ���_��/��[e�U�� ���$o����K���ov��~�{�����������~�/����[�XZA<f��4������
���1��-+�2��^Fp-�3�p�
2�t�yx��<�R���K������_�w�p)��q��5h��<A�p� �� `O 1l_2���V��}����3�p���r_��O<��:�,w�UWy����[{i+y��M�7e�I�N�{����k����f]BZ�V+�u9�Q�F�g�y��^�N�W+�w�m7�����a��uu')|�W��/��i%�.=-<q�D�7�Z�|�M7�'�|�_�Z�f���Ww�b1<�!��?g�U�X�d����U�nU9� b8�1��R��sz)��q���KM�YO @ IDAT��.E�>b��� 1�'��s�� � ��k@�d�V����/��/���H_~��^�J��E K��v�i��n�2���r���~��� I\���K7�E�.�|��g:�T}��D��_�z�]��~a�a]�ZZ�W�O�� {�������;8\�Zq.��Rw�=���6�����n��w�bX����b�c���3V�C=�Il/Ij������a�p_�J
�c��*-=G����p �p
U�[���<'�p�K�V���H��A ��/�;b�I�8�a� 4�p� b8��V@ �'���TD�����+��]����.^���X��W1������w�w����� v�� �q�������.c}�)���<K����~��$r��F���.s'�|������o����n�/��r��3g��������n��6����~�� �����>�;��/��]w�p��F #�<�Z�I�c � � �E�nQ1�
b81��R��sz)��q���K��.E�>b��� 1�'��s�� � ��k@�J�����K7k��.��7�a������p� ����r�G y��<y�=zt�����z����������W�����^���������Q�N�������?��������1c��e�%{o��6I�;����{�g���W w�$�X���/���k�]v������Rb1\j,�!��m���s@ �a" ��P�29 �� �y.�Z9��"i1l��T���R$�� ��k0� �y���<Z! @�� b��dP����i�4�|���z����]���%W���+_��_u�����O|�����R�������Z�{�u��
6����
op3g������wk��V���o���e�uik���SO=�6�xc������tij�V�t[�))]jC #�K��6�a�
Ux���3lC�p�P&�p�#b8��T+��R$m� �m��� 1\��} ��}
�b8O1��B+ �@ ��� *"����V �Mob�8� �� 3f�R��^�}������s�?���:u���'?��k8���o����:��Y��i���
` ����������~�3� ��l����?�V]uU/������c��5k�{�k^� �/�=o�< K�KX r�!��.K#b1�,����$r[+��������[#��Z�����3C ���j��^��m ��-�R�#�K�������@3@ � "��\h� {�a��AE�bWr@����0��{o��f��r�-U ��� ^"�R���l����G?��r�����?��O����X��%t7�h���+O]��{����nd}��.a��j�������+����g�}���n�/o� �����
w��'���{���{������2�!���4pZ�GL"��0�L 1�O`-�9b���Y���yh��<�R���K������_�w�p)��q��5h��<A�p� �� `O 1l_2����O?�e��7��/���KI?���w�����w���x��7�� �K8?���NN���V ����^��;���EJbXBZ�V9��-oq�m���������$�������_���6�%��*X�X�$d%�<�@��f���W^9����I�������Z}�����?�c�� �9����:�����\7S}�&B�c�nZ��! ><�/�[H@"J���@� +�� ��'[� �1��������u������v��y�s�f�?��e�� ��{���/�wO����t��� ��1�y?��Q�I�:���Zvnzn��Q_�1T���\s�;������L�>�m�����i_��9s��+�l��V�C�z�:����9�����^z������;F� ����z_������~:#sN��Q�� ��e���� �z\�x�$��}=���c�s�j��������J�t�>]eP�\� s�b`4��! @ �@ /����@���� ���_�N=�T�Y�S�|�"������U�7@�����V�A��>K�j.LtJR|���v�� �wkpq��'��]�_������5�\� }��n���r��a��w ���:��� r�V=��?� ����u)��?|�o��o�����lS>���s�=}~��7���L?��7W�9N����:��6��L� �.�M ����� �����O�8��X��}�1��jue��G����M�1/���+���u�+"�%C%���W ���� X\4����{��=�9��zw���=u $���{L��I�w=�f�f���h����K��{�skw�4��c����>�/����>��F� @ ����C��b��W�X�����2�Z���-�zC�7?���ae�&�$}���c����t��������=�y��6m���9�zA�\�����z�[c�5�QG ��[o=��K�d��~������&�4A����/}��<c� �bX�����M�I�� x���w�=��C�����{o���kw� � , F1��������4 1l9~�M���� ���&��k_�Z7u��!+��� �! @ <���/��������@ �@ �^A�7'�_1|�����.��O�I kU�d�V�N�0�_BZ�+�����{��
��X��w�}�����/�,1���~�����������
���'g������/~���R�����;��#�����^���|�_�Z~�}�{}>+��B6�>*1�X�������������wN�2�������1;%����������� `I`p��^� �-}C������d3�W *a @ ���a��!��@���� *!���P��^��w�q����]w�_���;������*����GG����J�|��^"k�����;��f���6�xc/d%�%h��xibX27�a Z���~���_�a��� �E ����=��c} ������N��������r�-���K��VK��MoB /
��� b���� �'�p?�qs@ � ��pC` @ &� 0B u��Z ���}����wK �{�u��� :Z�� ��
��/�����������>�v�m7��?��?��a�V��f�m����D�����`�ZV� ����%����N8����l�!� �����8�I\K K&�o���;�_�zq����$����6}��������c�ryi���! �� b�W��X��e���@ � ������ �@ ��H�!L��b���^r���~ku��1c���/��=��&�������/�E]���������w
��3�]y����� |�� ���Z+
����d����uw��w:��S����}�]w9���a���mo���WZi�l,][�J&+������� ���/w?�A���`���`�k����w /I�K���%X/� �W!z3� }0�s�A:?������ �����W_����rw��w�g�<��i���l_3z����=������]�n����7�I�&������ ��1�y�|�[;��o/�FR-;�PG�r��#���Z��_��U�����[l��h��u%�N��go��i������?
� �s���~��O |XU���-�6���P}�������zq���f������������Q��� �����u>������PL�8��3�tZ q�1�,�����L4@ ��� �������R �W /5`�
��Zo�N=�T/����w���X}��~���������]�~����)S:"���.�,���o~� `�R��'>�^���������r�)^������}5�7q��� �����������e����� �`� ��>��s��P�Cq ��w-��. �K�3i��i��d� ��X�j��c��I�@���5����j �M �����=��������^ �q���~:'��x�����d�����i ������� ��>����������%��>8u�T�����E��B�;������ReH�j���s���zI��^��C�z��� [� ��������]��e����g���� �Aw� ��#>p�������� ;vH>�����G u��~��oC ���o@ �&��� �WO��b��M��|_& ����~���;M��{wg���/������a]z���v;����d�M�L��>������ �%��8���j��K[��'?����>��*}�qn[�p����+�I'��t{��C}u�\�Mb����s��������VF�Do��}��E��d��b�j4�l����b�s���
�&���O@ Z-7��A'�ws��C���(H :�� �z<���|� + E=���A@���:����yu�����W�����[������[o��U1�9�����e)!���� ���H����
���/������,�W�y@�9�7����z<������������gk7=�����d��nT;�P��C�yU�������
1��qLv� ���P�6���� z� V����$��^ �M�i���?a�7KZ1,1�����?�]z�����_X�t�n������ ��@�d��h�d��H,��W���C/\��v[�Zw��������[�e�]�/���(�������.K#b���
�������
������d�^j���p�j��!����y.�Z9��"i1l��T���R$�� ��k0� �y���<Z! @�� b��dP9���a�X���[��7��/���f���6���7.+�%"4������]m��1|�M7��/��������v�=������;�H�%u�� w�����GX��V���>�_nZ�L$��[NZ�����}*�D�����v�u�b�D #��
�b�E(�b� ��@ ���R@ �A"��\J�rN/E�6b������H��A ��`� ���y.�B ��=��}
��r��zQ(���Z5,���
6�`�K j��V�������t����������w�^p��;�uy ��7����2��&����Z/� x����[���n��_��7���KCK�j%��� ��GN+��&1�Xg�y����b��e�]�w���
�7b1<�1���g�M�X�\����M�nS5� b8�1��R��sz)��q���K��.E�>b��� 1�'��s�� � ��k@������%��:�,w�]w��W_�m����-oy��������>�����]w����M�6���Vk���y��� ������ ��W�u�e�;�'��{������/~��n���r |���Sy�� ����v�O����w��������w�}�S �f��G>��jX2���F �Km��$r�0� �g����6T�L��<G�p�K�V���H��A ��/�;b�I�8�a� 4�p� b8��V@ �'���TN�?bXwUBx������Z��k������W^ye7j�(/` {�1w�=������=��3n��Vs x��d�M��I�"5]"��W/'-A<r�H�g�fIg����;�u��;����� {��f����*eIk]&Z�����^4KF+/M�?������o��41�������)�cr���F /��i��1�����;)�p������,�2%��cC ���j��^��m ��-�R�#�K�������@3@ � "��\h� {�a��A��+���V+xO?�t'�����G�)S��UW]�� ��]�p���_]2ZRx������^q��$�� Xq��W�������hV\ f�c�����������W����.%���;a����zy��0��m����W����[���'��t�F h ����DnYA�1 ��2�k��!�[V�����C ���j��^��m ��-�R�#�K�������@3@ � "��\h� {�a��A��+�u�eM*��;_u�U��G ��U�n%�u����_]�Y�t����r7]�+Y!���M_r�%����X���(�J��w���^�.N7���C�k�����7�q>�����#bN����S�N;��c��������=F #��;f�|{&��\�����;�6�1����/7�p�b8��T+��R$m����D��#unz���
+��*�=[� ��]��d���v�V��zL�<���i�HsFzn �����0�
@ m#�n[E��:z��K4?��^����zn���]����� s��`��'�|�=����E�^4� ?���]s�5�~t����Ul�S���w������;���\/�WYe C���%�;�8�M/��}�Z}|�����9V�^�+/�R���^��dn��=b1������D��z��;b��X�����z��#��y�#�p ��o���p�uT�p��7� b��Q�������D ���
@ ���5 �!N@�3��6K�j�S/ u���VZ��^~���.��!��*���~KdH �R��,���pSo����W�S K,�7��M���1b1\j,�!��m���s@ �a" ��P�29 �� �y.�Z9��"i1l��T���R$�� ��k0� �y���<Z! @�� b��d $���dHT}�$r����#�#��w�U��+y�p�x��(������E � y# "��j1l��h���<N�p� �� `O 1l_2� �a�p2$�>d������E�;������<b� G<@ G��pNok��"�{�����`5 �6�^�S�p 'b8��V@ �'���@ �0b8U 2�\u�b������ �p���J 1��# �#�Fv8�7���A�=G�H���F��E �`/�)b8�1��B+ �@ ��� �� b1� ���D��|1y�pDQ�b���u%����E#;���������#o�C�p#XM�"�M��1����s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C��C&��._L 1 QT����|]�#��p��pD����F��<(b����1�V���a�E;E �q"��\h� {�a�� �@ #��!Q�!��U�/&��(��A W]����]8�b8�hd�sz#X{ 1�s��t�n�IP�� ���"��8�y.�B ��=��}
� H �������I����G GU� ��._W���. � 1 Q4��9��=��9�F:D 7��$(b�{�N�y���<Z! @�� b��d $���dHT}�$r����#�#��w�U��+y�p�x��(������E � y# "��j1l��h���<N�p� �� `O 1l_2� �a�p2$�>d������E�;������<b� G<@ G��pNok��"�{�����`5 �6�^�S�p 'b8��V@ �'���@ �0b8U 2�\u�b������ �p���J 1��# �#�Fv8�7���A�=G�H���F��E �`/�)b8�1��B+ �@ ��� �� b1� ���D��|1y�pDQ�b���u%����E#;���������#o�C�p#XM�"�M��1����s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C��C&��._L 1 QT����|]�#��p��pD����F��<(b����1�V���a�E;E �q"��\h� {�a�� �@ #��!Q�!��U�/&��(��A W]����]8�b8�hd�sz#X{ 1�s��t�n�IP�� ���"��8�y.�B ��=��}
� H �������I����G GU� ��._W���. � 1 Q4��9��=��9�F:D 7��$(b�{�N�y���<Z! @�� b��d $���dHT}�$r����#�#��w�U��+y�p�x��(������E � y# "��j1l��h���<N�p� �� `O 1l_2� �a�p2$�>d������E�;������<b� G<@ G��pNok��"�{�����`5 �6�^�S�p 'b8��V@ �'���@ �0b8U 2�\u�b������ �p���J 1��# �#�Fv8�7���A�=G�H���F��E �`/�)b8�1��B+ �@ ��� �� b1� ���D��|1y�pDQ�b���u%����E#;���������#o�C�p#XM�"�M��1����s�� � ��k@�@B 1�N�D��L"W]��<b8��z1\u���G w������� ���`�yP�p��7�!b��&A�&��v���D ���
@ ���5 @ !�F 'C���0����O�3�<��s�=Nm��w#G���k��c��q�&Mr x��2eJ�w�_9#�����7F ��4�N 1�G��s)���z. ��a���|���n��� ��n1�p�����������"�[^�>���CB ���
@ ���5 @ !�F 'C���0��p�B��O�]��c��Wun�rn�����Tx�^~�-��sn��<����/� v���;��������[#��Z�����3C ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&����~7{�lw��M� _�.��Mpn��:��S��1�\�V��$w�I'��3g�y?��5b� �Z|S�p������y`��<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$r�km�����:7~�s#*��/tc�>�����"����w|��G�� �vs��C��� p�n��W�� �������|-�y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �I�(���������[y�zW ?��3�,��i��F W��D WU�%&���A ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U��q)����G ��F �k��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^��N1|�
7�������� n���n��Qn���+�eO�)���_�m��Vn��7�I��� ��+0���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^;���9s���\��r��1�
1�
6�t���{��n�_��&�������� x������ �-(� S@ �"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{�����g�iW����� �VX��Qu�v/��F<v�[��_�<���G>����Z1����+)�p b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^��N1|��G� �t�{��O97aU�F�-�]o�=��q������u�����G ��~�{A �@���<D�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����; u� �e�&����Jw��x�=�F,��������>�^����������.��� b8_ �p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D���b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8A ����������*�� >��5����l��n��&n��vr����3f���8b����gj��~k����� ��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8A �p�
���^����{n�x7l� n���n�������L����s��v�jS�����m���n������?B �R����^:���1��b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����y������v����3��`�MB%z������Gn���n�f�}���M�8�D���@ ��4�N 1�od���p�$��<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&N���Rw����gw~�s��d�P�^���ss�w3�<�f�� 1<f�T{ �������q���������<����`�#b� z�.�y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&N��x�;m����tn�n6 ���O��� �f�|���1 1�.1�z1�3��u���E ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K����K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $$��=�\��7�����:����W^y�i���^X�/��rn���n���n��a��?
�"�I+�<��n�����)��?��(�V^���#��l_�y�A7f�Yn����N<�D7c����e����_����y'�r�%������=����M�U�}r/������]_y��w�������M�<������59����_�<�#F������n:?�u�9q��1n���N5 ���W_�_CM�2�M�>�m�������9s�d���S���n��� ?��5x�������i�i�����Q�B3�[L@C�!�_~�;��y��O�gg������8����������n��'�v�����^n���K����[���s��G� ���{��/;7i
���P��{�)7|��n��|����w�#�8�������x����M��c����t��ek7=����:�}S�T;�.S-����^/<������Oww�q�;��c���P{~C �@� ��] ����$�`�w�9����������C���E��m
@o�����L;��fP�|��� =�L7����������w���� f�����UW]5���.~��k��n�}�u�&M*H�}��X �����|*�������}M<����_�A
�s�5�\��?�|���z����-���m��6n�Wl� i(;Mzk�uNz�+�7= �$4������~�W :1������v����s�A/�;������� ����g��w���7�8hS`��;?p��tN��CQ(��>}�E����L_����C��b"1|�g8-�@ �e4q@ ��p/(� �/a��$������� ,7�+M>j�b�iE�VDir(��Jy��Xo�5����x)i������f����2�Mo��Znp�~� ��k����?�?� V2�1�9iV�����:?�u�9Q+R�Dr ��l����>\���a1��a�D �+��L���wz����<��n��^g��>(W �|��������^z,�G����b��{�;��#���*�� ��e����a�0 ��$��}�WR
�A��O�N��� j�^/<��#�g?�+��Z���� ZN 1��� �"�c�� L�>| �`��������NbX �<���?��[1����� p�CB k�J?�bX�L��MZ��A@��i �$���P�D�;�� ���.6�a=���1 �����V���7o���)1<c��g�}��� ��{0���^Jz��������;d�0�1�����d������^�����
+ ��1<w�\w����b8 ~C ��9��y H H ������8L"#�k����|G �]Ce/��������| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
�i� � @ IDAT��"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V@ �'���@ �0b8U �Id�p�eD �]��=b8��~1�/!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
@ ���5 @ !�F 'C���0����������G G�� ��%D ���j
�t�p)�6q�6�K��.M�. b��}���y���<Z! @�� b��d $���dHT}&��U�1\w�b������ �p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+ �@ ��� �� b1� ���$2b��2"��._� 1 QT����1��R�5�������A �p/�+b�4Q�x�a;��zF �I"��\h� {�a�� �@ #��!Q�a�DF W]F�p����#�#��w��"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ���������|1{�pDQ�b8_B�p�K��pNG �"j1l��t����D��!�����1�'��s�� � ��k@�@B 1�N�D��a1\u�u�/f��(��A �K��s)�����RDm� �m���1\��]<�� �R=#��$�y.�B ��=��}
� H ������0L"#��.#b������E�;��| �y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V 5z.��������#F��C���]����@ �@ #��!L"#����_����3�������.���|����=un��z���tn��n7��;t��� �&O�\���C���>@��&��|��y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V %�K:���O~��w�qG7c� �3�8p_�G 1�����<�0bx0=�$2b���"���_� 1 H��1��!b8��Tk8�#�K�����^�W�pi�v��v�K����D ���
��B��W^�W���g>��;�8����C9��� �*��-%�niaH C� b1<���DF �]U�p�� �#���#��5D ���j
�t�p)�6q�6�K��.M�. b��}���y��I ������^p�?�������s&Lp��
��yZ! �f�� b���!7 Q�a��`�a1\wU�u�/d�$�����1��R�5�������A �p/�+b�4Q�x�a;��zF �I&1|�e����;��}������mosG u�=z4���/?�@ 3ZM 1���� �&�0bx0��0����������G ��F �k��s)�����RDm� �m���1\��]<�� �R=#��$�>��s�����N9�g?����~��n�����|�i��'���s�=�]x���x��7w��M�? ��%��%}�� ��������1L"#�+-������~!{�p Q�o�p����<�R����.E�&b��{�^�����C ��/�3b8O1��B+�]NZ��pv���*��K�aK��
d ����Qic�DF WZ��M1\w�B���@�����|
�y.�Z�9 1\��M ��
���"�K����c_�g�p�$b8��V 5�O� ����?C���]����@ �@ #��!L"#���*b�������D����"��\J��s:b�Q�8�a��{E �&j1l��T���<I�p� �� `O 1l_2� �a�p2$�> ��������u�/d�$�����1��R�5�������A �p/�+b�4Q�x�a;��zF �IZ�a��c��G uO?��{���K/���f���n���s��������I���+��W1v��� z�i ��G qz������]t�E�&�����={����]m�������cw���P�?��{��'}N:V�VM�5��7�M�0�����+�����V^^{��>�����)S���W_�-���>��� ���}V����5n�u�I��_|��y������X�� 1b��C�VZi%7q�D������l��r�'�x��!����;~�x����z�������� x��Iy���Z��;�V�z_�� ����U�U�c����,^e�U�}�&7���3��|t��Po�:�������/m����w~�j�����N�G�Qc[�T}�~b�~4�[�B����mVL�L��� ��
��C����PL�������W�\s�5�xL������,X���r���O�����S�k��� �O5}�����H�U�z���[q�[��/�j� � �W�C��v�_�G�Tz �5v_��6���5 @ !�F 'C���0������������ �1�����Mo��&��o����� b8� 1��R�5�����jrO�!z��
[� h�O�����sg\<��������tn�n�O~q��e�~t��9�I7{�t� {Mb�
1<x�����z^�cR�I��MM��VXa�n3�[�a���������kw�UW����� J�C�A� ��Y�f��w��m��v^x�uM�~���� ������e����;�v���vo~����>�1/�:o#�$qw�Yg��/���$I�v�W������n���w�����[n�E^g�t_\9�w�9�x�x��G�7��Mn�
7�����������v�����o�;�#{�u������%p�:O��$�% �M��v�uW�J�Q2�/���$�����.��I��~����S���F7c� /�����~�� ��'����}o6����B��� .�w�m�� |��M�<�x�u�u{����e�]�[l�������x���������d�^�H����z�
ox��q� �&�l��|~�� �������r�=�p�{���� � �����g���s�=^�� I�M7�������uK���+O�����w_|������p��k��f{����s�=}����'���Nr�\r��f�m�;���7�7!\������c���J p���n�n��o�����/��z�A�w�����J�_y������v7�p�� ��o��d���cPcU ���&�������_ 5F�<�1* ��V[�q���b��}_��6�&�nw}�C� b1<�~�DF �]UV �]��=b8���7b8_C�p�K��pNG �"j1l��t����D��!�����1�'i!�%�$�$�$b%;%��.Y��0�C��`���V�Jj��=�����$�O9��(����y%1+!&6 K 'Ia����D�vT 3L���\��;����n���#'��UR-�a i����t��Z��������{�;��> , �������X�����4]1,F�Ob��z��|L��\�&9*�(�'�V%����>0����y�?���D�~K z��1��/�WIl�I�����;������
]���������-��}�i��c�~��=����`���V�������V ke�xJ
��G 4.s�>��)�>�`�*WBS+�5�V�����V+�%�%�%s%���!N��T����._3�H������bn����f��r���j�1�,���g>��.1��-N � _�bx�����u�����+��cI9���K�K�S�\��%m����z�z��>��Sn��bk�H ��$���������K����%���O��@� b1��6�Id�p�%��#���_�^o��&Z?�T�����
1��R�5�����jr����&b�7���1�4���G ��uS=!��d{-����V��z��N�Jm�p�<z���O�4 K������������6]�YrY�S+���R�u�$���X�V+R%7�*U+m����]I6�6�v� V ���S�FB6\
X����S�h% �zX���M�)�a���;��N"Z�X}) H�@���L�&Y����o~3�6�D�I��L��R I�p l�'�"1�X���M���]�b��/~�%��w��(�^SJ������j��Z�$1��T �����g��z �JY���W�O}���u��! ���W�2�����ZR�����y�����`U! ���`�������Z\+�5�����S Kxj��X�>h��bi,hL����j'��m�� SG q�,(��M�I���9���}�T� �U3��0�u t, jq�5��m������@2V�U\cE�4<���>p���������� �>���1�<;7=�T/}`C�M� � =�U/=>5�u�1�X(������������V���� ��&�nw}�C� b1<�~�DF �]U�p�� �������1 ���1��b8��Tk8���$l��@�����zj��L�K�S��!���W.[�p���RVbE��$-�9�/^�� x�$�����>���{a������9�Z1*����w�q��M�x?H���._,��M��L�Q\�.�Jdm��_��W��
+{�
B�H�&�%q����o/I+����- ��S���� �|���o�������z���[���iU��[tYh k���VPJ�Ib�G�������m���n������b����Z�)Q+1�~%V�������J1q���`�����.��R/�� ��V
$0�\��������Wl�6��%�a�p����'?�e��I<K"�G�^|u; [�@��m�����~'Q*� ����?�����>�9/1�E�k������x���b��z�W����~��~���I�v�a}p@ \��G ����n��6�rX��U/�T����}���;��$�ao����|��~�+'�W�% ��'z��fC�;�%��a �Y���+�%����� ����l�� +o���*s����M��_��_=�}�&>_��L�Y���;T/=v�VW��X}����]�� �����������/��O 1���!� �0bx0
�0�����z3�7�zc{�%������=����^7kV�w��?:7�x��{� ��fN�u�O��
1<x����1��R�5�������A �p/�+b�4Q�x�a;��zF �IJ��R k��V �R�z������d�V�i������ki�&�{������C ��G�e{�=�8�&q�U��t�>��:} �b�(�����������N�T+/�����<$�$�$�[+K%-�����@Kf*���?��?�������M���:�=��$���T�'�<��R������K/��_�Y�V�M�)�`T �p?$�$%�%�$�%���Y���}�]�u{�����������_V��%l/�G"O��+�Z�-v�t_�$�u�� ���~E�n+Q�U���s��>��*q��.����"Y9�C ������2��1!����G���q�|T;�Gq <%��O�]�W>���P�V��s�����*�j��K���0�4�%�u s����XI��?�Kd���i�(�O��^$����� h��b���j���5�4^t�����V �A�W}C���q�|4�4~��V����<$��bX����Z+�U�[B^�5��r^���|�}�D��q��xH:K�klkLKR�Cz H$��K 1\o�� �� b1<�w�DF �]U�p�� �����,�'Lr���&Ro��&����|��y.�Z�9]�%a����C����*��@ ��F}�1�Ju�1\G���%b8OG���bXrU������OHb������ N������� '��%�.M��2��Z���6z�� �����>����~��QR��i5�$�V����~%.%�%;�"7���^���|���%��S� t�A^�u���1,��<�
y�������������z��_��_�����_��\t ����R����7Z--9�������O��';WLjU���V�j�V�j����;���Z_�e�������Wh���_N ��/��������:^v��=��l���;7�Y�}u�` n� ��.U�Z�j�cq�����g?�Y/`%|���F}_�l*yu �P�Y2Y9�|��V[y��~O�����Z},���zkE�dfZk��0����� ����V��M���j�L��5^U3�n��� �8L7�O��j� h���bX���>T��� �H��8�cN�U�G��h��H��M�c��W \�K�K�W��2eJ�i������^���z ������n�z ��� �C`�@ #������������~!{��A u�F ����s)���z. �&���F �5���jM��q�<w��'��3^����[ w ���^ �?:�� ���=c������`��������k��U=&u��~�$������ ���{-��ZX+I�[�V/ {����U
�% ��Q� ��U�{���a�*��ZX������I+
���mZ���M��� >��O�|��J�y�<�H�2���R1���Z=�U�Z-�����ZZ-)�ZQ-!&NK�p���VXj��$�}���BU�W�&�Z��Kn�������*X]�wq���.s�� Kb.N K�J>k5�.O�������K�$p���3���[�p�_+;g� /;k��8����n�s�QG��3ZE��w����y��}��|$|5>��>� !�K|k�qg>�bX�/Z!����&��7�t;�^}�@�}��W������K����~��>���F�f����:wq������/�U��^om���8)��-�����f��X��8�%�%{%�uyn�x�>���|�c ���V�����7�<�l�����,�,��U��Z����~,�b~� 4�� b��e!) m�a��`z�Id�p�UE �]��=b8���7b8_C�p�K��pNG �"jG���a�%{E ��i 1l��D�z^E /JRB��+��jPb8|�D�����.���t)���F�/��U�������h;��3}w����[Y+{q�������V�Jj���W�(��bX �������.������.�+ -�&����;���u��-g�m�;�ERDE �[*EJ$]�$6����]S�8%J�r��V���& I�\�C8.�v��s�}����|��.�Zc�9�Z���Z��k�1������������$;��� �G }T��H ������}X����-Z�0�F��d��k(���WN�r�' <B��~�;�P��� B s,!������(k'k�D�b�Z�`�h<��a�Z<���������LyX�����ry#yy�0�� @�����e��i�� � ��>����P��
��d�3z�h3�|��5F��i3�e��_�ZfBue]`�� � ��\�Lp�:����Q#i/&H$[R��y����e&c5 OkB�'*�_yu[xP0 ��������P0�`��~ �!K�p�[U�p����^��U"��
���P���.���>]��+E��G�p0��>��a��������wuf��J� &�p�xar �c�W<��� &j~�I �U�z���l�=
x����d�q�a����" � p@) ��a>S/B�k2������x��Y��UmHb�,�
���Od���=��x����\x���s��I� � �����;��$d8p;U��q�x
Z �����e��qN����Ly����D*:�� ��/��k���Ld Tr���\���^0��&M��� ~8�6�CS�Q�Zd� ���w �M����5F��U�>������ko���N�p
��\C�5����7���R&����pA}i/�o�c4M�E�� |<�����ms��y����_hP0 ���������a��t�[#���h����h��-��a�D�� ����a]\m�}��aW�����`tw}V��
.?��i���
����7� ���uA�m�}��#^��uf�W �/J�����<m��x��i^���t��[���$�/ ����e/�����b�p��2p @�g
t�c<��gB��f.�I���
d_��Y�a��L�^>F�5mYk9�c���� ���0��s��I�i� �������[O���[�����%���#�on`�����c�>3�k��9��7k��m:��F��.�Ao����� mm��aBG��.^��B+�� ��I�S� �; �7�p��������y������f�Q��p
���� MRy���+uN��]=���f��R_<��}q %���������o�����z��|�=8 ���YP(�`X�p�K#������H6_i� �J�
�#�|e
�`�� �(.�"' l��`8'��-S�y�:�'R0�Sy����������)�����P��)����Y�J�W�x�x�/?<4 � �dm�d�(]0��%^�����{wY��_��[)+���j�+���� S���c�jCI�X��W�^�� .(�a����k�u���Nx���\����f � ���H�`�}��@����O������/`�
�/y ���|�������+W����QM���l����\�9�w)�_�Dhj��eRmAy����+�<\/\?%%%�N�-� ���&lx��� �
�< ������
� �g�&R%��c��GS�`�u� ���~�2���'&�k���M��������+ of�UsI�C<>?�;x �ZU@�S@�����K"�Z#���H7�<�� ��KW�g�l1���aL�����/?3}�ib��
k��>��Ut+�%W0 ��]~����~R0�U��g��+v�m>s�����a���f�� �������$��p{�_1f��l���w���& ! 9).��U0 �����o�~��6� �N�#� W(� �|�atP7����h���m��H�}�[�a���<�H ���M 9�Gq|J �m���x�����[�R��g����.\h�~�iY������:��-`EW ���� �3o�T��a7m�$!����N�6@�O?�$/ ����X�3��y�u��Y��� �e V��L�RA=��^}���)� ~����-��b����&�v��7�����h�����75��m"$8�����F� �p� ��x���]$��M����$<�;u�dz���Vhn� ���� !��t�"^��Z� �� ���5 ������'8`�������x �����,_�t�f-i&m0 "�P������O���Z��
�i*�`X�p��J$v�Fd��h���d �`8�<��B�pd�*eA �K�`�_W[m��5�`��X�:z��u��L�O�P0\~��x���0�JfeR0��na:J��k�mIxv�� ��
�X`���~+� ���'e~0�7& ��k�����a�%`���xx �-���j���F���S��+��R�0�9n����<��(��7�r�={�y��D�`���p��4g�[�)���.��<���a@ `��iIhdl�r'+��Ao��a���@@����R%$t&�������nN7/�Ay��_/���=�] 8����^�u��)��q<�KS�'�� & 4���OK���1��#G���� �x�.�~��'���|~��9���� �� ���<�)z
(�^�i�U��W@����B���Y�p�[U�p����^��U"��
���P���.���>]��+E��G�p0��>��a��������wuf��J ��K�d]Y�dy�]�V # ��
ee�[� =���j��f���*�&]0|�]wIk@t�J�$�A����������� &�������x����1���N �<��6 �L<<�c@U�(��W2�8�-0|�����x0 ��������/�����N�)uHRw��� }�����-3_}�����c���t2]s� ���;z�� |���e��|��a�iC@g���e��Tk ��x8��Kr J ]�a<���#����^���^�R ���P0\T���U����a���R�+�5"+NO����`8�-S�r).�^a�[���(����V��+v�h0�(Fw�gU0�Z���S0 �����`�_�0�a��8�� P (2.]�T�W H6n��t��U<'�K� � z�! ����',��?,�3���k�tN�6M���7�|�xL��K�H��a�>�|���5k��</�y��w�yz���2��������o`�����{�������(C�� � � ��GB0������K/�d>��C)+ O�T�RF��6�<��f}f ��I���,O��0m���U�&Kq�f���y ��q� ��6�� sm����E��z}�����+����~ ������@�)�`X�p!]����`8���`8��gK�`�*�w��m�`�_W[m��`������`8�]�U��kE��O�pp��:��a%�
�YOx��-�}J(d@��� �[8��
��Kw��@Y�fMYsOWo��t��3�<#��_~�eS�Bs��
,���(��m<����+a����z�����6f���x�RWB S6�����#�w��z�j3s�L�xX�{ ��� 0��� KH�s�9� 6LBT'���c��L(���'���)S���'��'�o� :� �l���D ����u��u���M'lr|^����Hy�/�Q�FIx�T��r< � |�����Y�����om��)W`���xa�f\�xj�fG uTR�O��s��������l{��6�1�����R%��� @���{^��w��x�>|
(_�h�T��W@����B�X#���h��}P�!r����5[�w-{S�yt+����L n��m�G�Z��z�y��n��Q0 ��+[v�e��)�J�����
�s�o�rU0�/�s{ ���7��+����9��a]�
���%�1���-<Y����_R�s��@R~� ��-[��o�]BA����0k����� �Q�#�0�vY���+����+&�H�[��u�]'�� {�1���n����y {S&`����G(m�d�7����{ >���%K�Nx�R�D`x��Ro<�o���q�Z�����{���ON��Q �P�� ���� �f-f4���k�%�\"�7Y=���@���,��\x�r�d� ����r�JYg��& r����������^@Z ��<��L` ��f�nX/�6���y�����/��6� � ���a�{&]p�s-:��j�������{�����������3�4�_z����w���y��p*�`8����R�Z�
� �`��
���� �� �`8���`8���-��a��?+��E.>�>]�p.��_�
���u.��`8���7o���;gS0��j��0���K�0���? ���^n"�X & � R�7'�t��@���_/�t�0k �]KXd�� �z�x�X��/D�O=����y�x�����Y�2�d�yv dS�5k�H9 , :�6��� @������h� ������������^�z%��D�j������ �a��b�=�L����kK�����>i;c�z���e
d�I�k�f �1�I�������v9����L�������u�w9 ������ ����M����9s�H�}��'r��fLF`�i?����l&-p-YIa���+V�0W]u�xk�3�E8l<��k�{�g2���4hg��_v�e2)���~��
���^Z�*@���"1����J���H�8�N�,�� +3��� ���>Z�Jv��0':{��q�?��6 =�����u����|b�A�SB�B �L@���A���D���+�w�
�3�v�x�5"+c��_&{�V0��fa����~���M{����i����}�=
(�o�����j�t�%6a0� ��#�������P@�p8�!�R(�V�� �`8<m�iI �+�o0 T���+b l��G�`�x^;%�C ��^{M`&!uy�=������.��6��a���A��6� � �^}�U�x�� ROd 1�����n[�V-an�@m���[o(�-��2���73���8{��-�n����� �~���E���;�*4b f�d ����)���;��J��U�T1_|�xe�r�)�U �f, <��-�;.u#5m�T�8��&�_��uq���k�}��V�t�{�� Mb�����' ���U�VR w������z�u� �
l�z�R �oA/�AW�S^��/��p}���C�����K0 ����� ���2�o�^�9�����O��J����z��m�1h��w�.]d����|&$:����6���s���d��}��'|�ss� ���$K����Y���a���x������$�.^����'\t�Er=��
���~� ��u���5����� ;M��3�����S@�p��LK 2x e�7Hf�qC%�>!R�%:�����M����7�=0X�& �e � �F�� A�l%n� (��3c��N����>���+���r�'��_�B����eb@�,.��������G�B^�g�s� +vy=��5"+�%�;?�G��N�0 �`8 ���
��uT0������OW0�J�`���g ��3�,7S6�0��t5��&���Y�MV�8�4�s��]r�3��tq�0��`8 ���
���d.�W�M�O��&lE�� ��,�O!������-�5�ws�t��e��\� ��;�8�\x�}��Y� 6�Y�fIT,l����cG KH]��Y�"��L���W�.�a/�)��mPH���d���0�/U���?�N���o����� �L��� �97�� �\c� @m��-_�&� X�)�+�
�v���
�N�6��6k��h���_�w.���n%�����+��; ,���yt���P�4i"��P ���x���ss/��c��[�n�
_!t�� �8�%�Y@ �� �# ���=�����/`�:`���7�|���R����L��\w������<�v��-� ��h���T����K0�M ����j����X?�� �k4����
( `��D{�����-��nu|
�L�m~�����^\/�|���9����r�b� g_��L������7�6����������<$
0��N�Y^�$��t| �9�13p��It��3� �+ ���rs�&�����7:0>��|x��c(0 �����c� Xf�D���1 8 #�&_:6�E��gsy���g�
�3�n�z�5"+k �W.����c8=��������L��e �y��������7(�}
�]�pn[���
�s�s�s��F�p�U�}�,y~� ��I���� 5��#�����/@.���7���5f��o4�v���8M��T0 �)y���[T0\V;�q���� �����<�O�z�I P�g[�,m����3k���kb���|�����%�.���P ���/ {!�f�����+��8��?p��:����������E<* e]�^=��[��~���y����[�K�JY�}( ���
��
���I�(k�N�J�fp�a����� Ggo���R ��y�w�NR ���[�!�6������o
8�98�-�@;��3�d���1��3�������<?���T��mBhS �i��u�����\�<�7� YW�0� w��O��� =�C.����G+tA+������f����q[�j���f�vr�->� �;���<�����u����C����
��H��~8�L������XM�U@�pt���J�
�N�St��|��y�q��YX ������e�3� ������e���H��b@�M���)���x �A�w� ���aW�i����� �3�`?n� �IB���������L=���2fF�!��c��=e��"g���a�Y\>�;���}�`F����2?w�[� �Pc����7t�+�W�����m�2H��k�� �m�AR�p�[����#//P���>6��-��+�o_�����j�t�%61�� �����+�`8�m�N 1�����`8 ��������M�%S0��T��0���T-Z$�x�����xxh���'�/A���-��� �� /�` p�<�f�z�&���9 u�� ��_~Y���P��r�"ZI �m�^ (���Ov����a�8��Jx�M�6 ����q ��9{*0�)k�R� |P���=P�Nbt�q<�c7 �����1#���~$l��U �<t�P� �]�v�*�oh���w�����2^����Q>���E� /����m��1x���,�6,i��m�D�����( ��>���5�
a�)vo��_�5�<���m3�
l�$���fhGn�: �x�s<���a?� 0L}h/�� EN{��H�P�����:���k�k���K`������.������h�_(K���� �3�FBG@G�M �����*�^��t�D�m�@�<���7 �)�
��9���#_B0`�N�V>�����r�L���9n������IS���l:i><�����N�0�����I���z�-� 4��Tp��T�������</�/m� �+ �?7{ 7�g�A���F��t�L���a
��K`�`X�p�~��8�5"��p>���9�Or�W0�;���3�&�^�?lb�B��`�*�w��m�`�_W[m����&�G��(������1��4�t��*Y y��Ff���
�w
���F�J�}��$��^ c!��|��R�9 �cg���0u� ���B�]�
�'1m�w���g ^x�ag���,������b��� ���6E�c���;�M|��;#cT�2e#/��y( ���0�{��@���k��� ��/7��G/�j�J�o�9� 0�'�]����1 �N�Yc��~
��������Fi[�k��x��b��z����;`�6� �6�C3���w���&*��W_-kS' ;����D���������8�b���n�q����� ���K�( ��`�&D3�ll��A3����a �3�I������v�8l�\7����� G.t@c@;�n�D�[b���|������ @ IDAT�.�H�PF �X��������.����J�b �<>M�8Q�1��'L5���� �rS�;�!�3�L���6j�(�����3�';�D9���% �d~ \�$���{���s�
4�
� � CS n� C:W�],�N 7zn��H����sc��� ���t ~��7 fMqS$?:�|&1�`f>1�����) 0�,"��G�!u�s�[ �ZlF ���q�1c�H����0 ���$:(f���� aL,��������3���<���D���+�5�"��]��2� �&�����h+�b�BgF�]%��U ��m� �A'�!M�V�� ���R�N�p*���=� ��x�I��U":�
���J���.���>�{�M�� [5���A�1���h�W�Rbh�?�]�p"����� oiy� 6�����~��W�-�O��&��6T0��5������2�� �b�������Mg�t���-v@ $��8����' /l����.@� �(� � � a������<\�M����d�;P �'P
PNhi�cmn�����C�- ��U{�S�9�cya?��� ��a�dc�E{�_u�U��������*v�T�� �j���� �;5����#��O�<����0��� Wq��������!3� �IU���������M��M�p]s=�f�;�5��r�-�9���| ��s
����N`@8���}s����a���+�
�k���C�Ar�� 8�8���N��?!2Jb ��T�3g�����_�����3������N� uc���a( 0�M��Y��A��R7�@ � x3S�53��3�-a� !��t�h�e������-<c� ���Q1����a� �t|�{�t��'��2� t�����/�%�p0�kg�R�UR0�`����|�������C�dV����� ��E2/��a� ���j��� �R4�|xvU0 ��.���@��KE��K�pp��:3�U��U��n��`�0��}�;��a{���e'`xm�����m�&�� a���X�
@�l �q cx���� 7 !��)� ]� m� �P t����� Pw� u�{���J]��[b ��`��F���}m��7�q,v\� �,6���}MX��u�y�zu��b���KY���X���G�<��{;�������xm��(��9pN�����a�mFQv�@]`�����L@�=��C��K���v��al�QJ��������l[��'�{D���e5F��^N`f^�61���7V
�U`a��-[��,�13����D�!!�pHq�����M��,��G���(# ��^6��Q6t �BIs<a0x� �B]� �7�%1 ��x���Z����m��CX�� �c�D�����0�/yp^���6/<�x�r~ 8n��L f����`X���k) �X#���0�F�eP0��va:�>�>^��6Y� M�P�1R|;��N2�b�Y�I�pn[����Kl�H�=�^{v���W��
���>��L�p�J�?��o�T%���`��*X���-�n����%
#��yV!$���_n�V�*�n�ss�`�%2#!��F� N�I� ���~��������vi3��`��n �_�~��-m���&�������/O[��t�*6 ��E"X #x�W OX=H��*;����"�������������C����� �L��!�.`?g� QmC7+��k������]� �������<q 1MG�
�uL���" �7�E��N6_�1>�Y���3�p�e�I�1h�� !� ��h"L7��/�.����b6p�5+���l�EgF9� �K���03h�x��p����7O`3��1���^L�'fma|,V�w� a��B�B��L (k q�(�d�0�e����5[�w-{S�yt��K,B�������� ��� [���i�0�W/�~J �KS4��oG �Sk ��^s���p|�:s1c�1<��0f�I�xFcbl�u����M�,F��^g �Y�o���f�� �������$�EO\�_c�&1���iz��*�� \����b(�{��f����Bn�����^�b ���)���YS��`�g_>�D����<I������4��B��{�Q�W�
��:��� <X���q������ x���g2N��3�
^8ac���� p�y��G�6�^���`����u�}�ITL���f-Z���w�1m�s6u��C� N�� ���6�m�i��P0�N�����M��A� ��� �` GvQrBj���;f����./` ��M��l����o�y�~�|�X�(�r����?!������OB�);:��N���w���e�^::�T`�������/ �w?&ug��������K`2�L�6M�0M: �9��_��B�q���/��B�����FY�73����^�z �@f�1~�x��F��3�&���G6Q:�bL�/��+�5:��W'|�;����
%��9d� ���O����)��D���t)40|��n5�N;^&���@!'����Mz�5Zqo�
� iK�K��?2��F�]��3�����L��S���� ���x ���������DO����G����Sh`�i� �W��������O'|������}�s������\p�)����=6�1����������k�)�����>=~l�}z����� c{�e�#}#�3>[BB��L��K�pa_��q����D K�a3�^{�I'I�h �q�!B$/���� L�c�0����)�
0���
7� mG_D��Y��g ;���O�1~\������W�Al�,�I�iR�������T������&M2�'O�N������7M�4����K&qL8���c� b���%����^*7T?c �)�HYs�1!��Q�� dgI����`�
SBgc�d���U��/F�g�� |��t�UW LFS�4 P�
�T@��#GJ; ����/P���>�����)�]�%�L��Q�FI� �����>f��� ��Q��, ��'��������P�c^L�)$0������f���2Y% j��<�]
����k�ce2�}�������}��&�������7������� c$�������0���9`��<.<�1��1�x�1{��u����P�������@]��p�� �~a�}:�3�����+���:��i�������x��B��y�����K�B�����m�r����_�}d�2+�����e� ��-W`��0�� ��
=qMa��o��$��+�7�}qp�� �j1x�}G*�/�$��xc+�!
�H�h�0������*@���6�) '(~Cx�pV��H���9��� �z��������Zh��@f
(�L7=�� ^�x����1��>�ho�vw:D����dfS��ve����vma{�� ����1�a O[�k�9�T^0���/K�n�7onZ�j%���1��5a��L�"I?W_}��6c�1c��A�N�5�� I��}�6LfC1��[ L����P� �N�:I�y���������Ev�6�Jf���.�G�*�XhCg�I(��/40|��{e� �\�%1���p�_>3�O�.�w���b�N������5k��E�$��k0����H'������
��<�>+�<�e��(������b!��[�u��h�z�gU@���������Z� H�/�����}���������.��B,���sd9:\�#�)1��+�<�;����F;��M���7=K� ���b�h���)?
�;��p�"2'm��X~W�/`>mb� �A�M����w���a"�T�()�`8J����1B�,3�����cG��,�;2n��p {�1�$���Og����,�#^�O=���a! 2a��x/�� �Y�
��� �����a�bP1a��3u$t6aH<<���h9�[�n��5��S����_�->��#$� ����H^t�h��6eJd��S����H�X���c8�����Y�x��J���� ��u�T�BP@�p!����B
� �-��@!*�`�[U��+ �J����`8���UU ����XJd�����uk��{VH]����?1;������:���0�����0%k����h�86��0����k�
� �.�8�eJ4 `0�X����j���V���m�f� n� �<�j� ;5!�i38�u�,����z6U��
(.�fzD�x��� tl�������!����N���CD�(�H�)�{�=����+�\ ��Z ��T*/f���'�l1f���MTo:���{N���c����Hb6��>( :'���q�!��J���O��u*V�h���Z 5HXk�"l5!��w�.eJ��6c`C^���qc u�M4C��V���Jx$3��m���S����QP@�pZ)u�c8�F��*�
�������3GO��k�D%V0�H��
�A*�/0�� ����c����3g qp�����w�������f��U�� ;����.���4s��DM$""�(�k�vT".q������D�L�,�iY������G�6r�f����0��Y��6��M^�*E G��BV�;��� 2DB+�k��t��E�'+&@�Q^�\�x��]��� v��|����L��Ry�0k� ����s�9G�9��a;� 0����K{��!`�� h��E20Lg�0� 0��?�Yf����N��Nx b:��8/�n�0`��1�:Xf`�J�~��Y�|�xS�����~)�o�����c��a�/�P��ca���M���JZ���� ]�Y��_M���� sJ�qb����k�����V��0��v��Q������B�D�*�d�1�����H����,�F��l(i<sJ���w��0���b��Y�lA
����e^���y�����.
-. ��*��*P�
(.���g�x�s����]�v5���OZ���K8e /%`'`8U�9��Q��waQ{f�F���~`� @�� N���P��H9r��gf���u�z����H� �/��hG��o�Q4d��G }T��^���}BL'Z��S�a`2�&|�"
����D
�c��Z -[�@���X>�/����2��>(�+��m;��z!�1<z�h�N�������U�
n��3~ �0m����H�T _N����O��5|���)�����O�,�>����6����L�s��0 �,N4F�o�B��q���x�f� �z�y���g����M��1��l7���#� }�a�����m=:��}�F�'�u�O�Z�Pd`�������#CQ�<��a��,��`8�P0�����*�
�(�`X��������; >������d���c��-:0���@"��� w�q�@�A$F8@2^� ����'�`��c� �N3�D ���m������i��w��7�;T��s��Y'�2������yY�2��r��;��_��_R�s�=WBsd�_��a
�O��'B����.���_�v���?�,mI�B�{�1� 2y�7n�������0>��I1���p���j�5�i��Y3������*�G��^��/4 TtZ�����7�'2I�0_~����g&���7O��:��S��'��uU��0Nd�w��'�DD�4)��k�.��0F����{c�D�s��� l��b����%v�^M�����:( �������G�z��f�����>C�9�c��c����l�����y�����]�/�_9>c(�g�M[z���^5����*m����61 ��W��v��i���b�w�=�$\��j���z g�� �
��@q+�`����I�3�K�.�9f��������x=F�Q�FUY#����;�|�������a<t1hb�����n���x �5�?�`������� ������9LY�}�Y��.�$ (S"c���a�#��}��q�);`��b^c���������Msb�f-��� tC�`R�d~���2�<���rO
��9( &�v���p���hX�� D�u�s i Yb����� �1X���j��@��v�_�
�����Q��J��&V�E~q
�1��-L��
�m������k/����� `�16K���p�=w��%����['|�����8�J������������&D*�G�������g�� ����s����o�~�;y�vd\�2e�
�q��>}�D�S0���� ��k�9���*�
��:�Z������u��K���.o
���K+W�O���_�k@^<}1(�'���K���^{M��e-a�����������>��3��� ����Z��_ 3k��!3�?�p9��i���
$4 ��A���2c6"@��g��b��Y'�UR0�k ��
�]]U��c��
��kgV0�B���P0 |�*��a% ���j��� �R4�| �������`������`8�]�U����8{(��&��
�3UN�STU��
����e��������k��E�I8�*U��%d�_�9<3V�^-k���V�*�8����x��e�����+!��~���?7o���x�`�"� 0�0��� � ��+k�w�y���% 2��#�25o��\x��~�f�M�����.�� d��
�C�@iK�p�B�|7�!o�rO���X
��uq����
�])L>
�����Y �V4�� ���3+�WR���.�lU0��zz�*�
��P0�[ ���
��� ���o3g���:x ����w]C���N a�w��)��Y���I���7�/�W�J�$�2��~!z6n�h���/�f
I����e a���y� 3g� � �R&���������~����LM�6/� /�� �`X��o.�o�Fd�n�X� G��l� [%���`��
���j��� �R4�| ����*v�hp�)N{WgV0����a]���`8��XU@Pv+�`x��)C�`�S�N�s���m�,;��% �O<!��\v�e��'�Lv�|�i�&3{�l3~�x�d�\�,��S'�����:n�8�n(i������ ��U������d
(��� @N�8������nY�� ���u�I �p�B3y�d�8�����Y���\l���c�E�9�^x)v��j����z�)��l���������a�Y�f�0yQ& ��G mO����a�Y_D!��� ��Q2(��� D �!
�C�(I���p
��uq����
�])L>
�����Y �V4�� ���3+�W���t�a}2��`8��U@P~�����j�[��� �m���k�����,�Kb�&H� :4�!��G }$���N�j� E0 \ ��������L�#�<�T����J���2e�� \�zuY���OcvD;�FG�'<4kj� t?��c � �%�n�ZBR[�LXj��0a����7R&�2�]�� �����e�x;�����M|�5��z���� ���
�3�tBy�5"+e��](�iK� ��y�U8��r)����V��+v�h0�����9j���f�� �������$��8���3q�i��N���Ty�;���\� �< ��i�]0���,tp_�7I?�g�pX���/� ~���~���`�}�*v����
�������lw����a�F�9���k����-`�Y�d�5�
ELheB1s ���� /cB1�N�g�}f*W�l)}���Jhg �>��C �z���� �}���Lxg����}����K-6lh*V�(^��D�i^�Y�fM�%%%�B�
�I�}���W�\)mhf�b�
��n�*!�)��]�$/�D�� �e2���
�3�lB{�5"+m�U0�i��� ����.��a� ���j��� �R4�| ����*v�hp�)N{WgV0����a]���`8��XU@Pv+�`x��)C�`8�,2>,�`��X��+��"����CB1S �D����?��S����p�x�z�M# ��KH���7��L@-��{����<�a����7on���' ?^x�=���0eb<����� (Pf yQ&�,v� +vy=��5"+�%�;����� ��
�����C����
��uq����
�])L>
�����Y �V4�� ���3+�WR���.�lU0��zz�*�
��P0�[ ���
��������� �s,X �����5���Q�F�q��&z�=������/F�5k����U��x����R�����iS����d�~�_�DS&@��
��-�)��" ��E>.��)��a�\���Y�p�[��#�x�
3{�
���-���=���<���gc�7M�6��a-Y
��� N�*�oK�����j�t��
& ������
�]+\~
������ �+�`�_�l�*�F==VPT��
(���~�P��c����{.���; O����Wkv9e~4�!@���;�V�Z� p�e��M�;@��m�u{Yw�<��,�A��"���%����L�m���� ���W�T�P6>�������L^�% ��|����I���C~� ~x�|��B����l��� k��
���2�+������� ��e�_.���)����V��+v�h0�(Fw�gU0�Z���S0 �����`�_I���d�5h0�X{���������2�3�q�ai?M�U`��E�v8j�lb�f�L� �[`-�*�C �P�b����_7�W�������9��s��2�ge�
<��~o<&��>����G.��0^��38�`8]Xm�N�(O<��7WI����\][A�k��
��P��9 ��2�� ���s+��S���.���>]��+E��G�p0��>��a��������wuf��J*��%��A�a�(aq��%��_0�}�vq�9���L�>}L�.]��� �c��c&L�`^{�5qx������ �U�W�����p���J(j +.� �5"+�v�*�v���+�JD�]��*����V��+v�h0�(Fw�gU0�Z���S0 �����`�_I���d�5(0 �: 2d�y��g��������,X�N ��o_��[�l�W��bo����h���{���s= ;�T3��
�#�xZtU�PP0�`���mkDV0 �VU0 ����W0l�����a�6T0������OW0�J�`�Q0 �����`������`88�]�Y����
��u�fkP`�1 ��
0�L�>�������!@�~����c�1u��55k���zE{�������K�����O?�d4h`n��v�V $v� �RR�) B+j T�S@����B���Y�p�[U�p����^��U"��
���P���.���>]��+E��G�p0��>��a��������wuf��J*��%��A��o����[�� <���7���w�3]�v5�Z�2���3 x�a ?���T~�<u�T3~�x�s �FN�2�����f�=�,� �P0�@�\�
(.�f�J��V@����p_��+�5"+.�na�[�p�Z$��(�L�0 �`��U ���j��� �R4�| ����*v�hp�)N{WgV0����a]����������j y��x�b��>������9��#��� S@��^�@�P0������*�B�
�S\"���� G��~SX���$� G��| �`�W�`�_W[m��`������`8�]�U��kE��O�pp��:��a% ��������W_}%����K�,0<v�Xs��C9$�*��1 �e�
�_�������b�
�j��@���~����?_;�@���I +�������Y�p��wyF�.� ./��i���
��U0������OW0�J�`�Q0 �����`������`88�]�Y����
��u�fk�` <r��R0���� >������ S@��^�@�P0�� ��C�5�G��^G y��#�Y�f ����F�
�sse��5"+FWgU0�J�`�Q0 ��.��`�_M�����j�t��
& ������������^l�8���� ���������� dm�=����Ud>���^`�C������;��*�WM���.�l-/f���?�o��F�w��e�^I�]��� ������O��l�o�� r�������o�m�O�nV�^- ���z�i����/��� ~�9��CK��}��������S � � ��?��?� �����+(O���w��i�o�.��$?���*U2 t����b\L�x.����l[7�����n��r4�������2 .4s��5� g�q�8p��;m�����7QW�� L2� �}��R�/� &��~}��1w�y�7K��
�
����sW������� �� ��\�jU3s�Ls�I'%�K���
� G��MV^kDV0�L���`8�m�N yX���k@����L ������a�vR0������O�^b�3�!�?
��o� �w ��/_nf.Yn�l�a�+�j��M�[�_c���!���;M��SM������
�.y��a�c����M��w\�Z�p�a!��T�[��� �T�re�&<��S
N
��� �Mz� ����;Qm�l��k{��m?��s�=��$��_|af��e��;�T�r���f�l� �c� ��8�v�Z��+��u���O?�T`i�
�bW>���M�-L���
��&���y��>�t�R�@H;z� ��U������M�����(�]p���+���P`�O?�$�4 �z���6lh(O<@-�0��?�|3e�9o�����2�p�
�m����N(� ������ ~��@^ <�y&�P�
�3�<�P7�{�D �8q��
�� ������`�c�9�4o��\y������*S�����J9���7o�1�m�c�=��u�YR���'�T0\FA���P0\�������v�<xpin��1�0���{�/���f��7���rt
�& +����)�Y�p�$�K�
��"s�O�`8���
���V0������O�A���>
�]���| �^�|����0� �1���f����_�[�� �!��s �����/�����{�d��jjZ�����Q#��H��
���R0��K6[�� C`<`��?��#����0��'6k�.k ����{�9���e;^�3f�0�=��x�&*7�|��W��={�nL�Y�~� �����ADx 3��� (3~�2y�N�:�����k�N�������/���y����������I��� w�a.���2�X������%����6m��@t��l���^�h�>���7'�x�|��9~����%L�%�.�����H^��}��7���-[��|��'R����� <���-[�4����x �Z=�)���E����9�g<����Scv��=A�!1�U!���{��Ko�����a��t�[#���h����h��-��a�D�� ����a]\m�}��aW�����`tw}�b�����6� 3f�������u�K��Y���9�T������sa����0��"p��a�FR0��K6[��@�
6���^�#^�L@ a )L�y� �%�0^�\p�����9���>�� �B/-`�x`'����U�&P�V����_/���J�������}��x����+Vx����U (v�L��x��t�����t�V���{�� �%� P����r�, � x��(/<����0a�q �^{���� x'�_~�e #�e�������'�;��� �` [�^=�h�x�v b �e��v�,�~�I>hA� ��5�m��8��ai���P0\�������!C�� �H�����C��6�X,�>c �&U �
� '�6���� G��v�Y��n-��I�p�[�l� �����`�*��w��+�������������;�������������1U����s�X<�T�����S%��*�P��������3��; 0
4H@ �r�}�5%1o��u�J8b@"��x~�! {��C � b`_�(!� {�1y��=z�h�8DY[7���r�}���.�� !����r�)��[�n�e-��D�`|�N_q������� �6;��hS%�Kf@00���i;|�p������k������{O��f ��N;M<���s2qi��������K�.�{��� ZY}��e����~Z�m������y)�� ���O�76�!���bp����rP/ 0mL[�h_eT0,R�E����"n|WUg�k5������`Cr0#��#��A3 �5�Y��]�*�U@���a�������`8�-�`8��gK�`�*�w��m�`�_W[m��`�����c
���p0��:��a�������`Xd������1<]��� Z20�3;c�Q�F�G+�����<���c)��{��7+����g��)6m�-���v�*�������<r��R0<v�X���a��g�6L�Y�
n���9��sL�f�J�i�/�b�#F�0�
�n�>���%���Gy�����'Qb?������S���X�}�n�����/�{4!���HY��p�_��5�n�cm���Y<{�l����3��&�@ S��b�������'O= �$4���{��w
��<�'^�xk��
� �LzS�9s�H�i����`Xd���X�E�����M��=�9 zt����/��RfU��0��dm^�1P�jU �A(�1!5l���|�O�P0�`8zWm�[#����E��Qh��eT0�Z����`��� ���j��� �R4�| ����*V0���r���a�S������d�[���W� ��J�����_�O�
e�^�������9)~�C�!�����^zI�9���#����^0 R>���e�0�� �u��q��F��K.�D��������2���/���b�oZs���so��L���k���T�\Y �=?���Oi�3��_��}���-�M������x O�4I�� ����z�c,f�a ��L�R �/������Od���{��vn����������0��@��7
\�� [���XP0\�-��zs�g�� �)�: ���C��!��q:@1�&4h 7r��� ������ �9j�g�`X�p�/�r�� �[�P �`8T��qa g,]�T0��$
��uq����
�])L>
�����Y +v}M9�O����`X����U,�d`���x�e���~�� �&K����) �x�����o_��&�.�CY�v���rJ@j�N�d�[�2�T���0���Yg�uu���"�9eC3�d���4/������ �?��Y��$�_���^����
�$���s6l������������3f<���l�������M��*U�����o& �.� �`�T�P�
(.���U���c4�`��+fq��Y�v����� ����E� C�
�~���L%�VC�����O�
��wUf^"kDV0���a8R�pZ!�2(�^����`��% ���j��� �R4�| ����*V0���r���a�S���a���Xf��0, � d���w0�����gi��x�b�*�, ��*���8]0����� �7�xCNy�M7 l��%�3l����� y���-��;.�m�C��-������2g�}�/���c<������wY�{�_� �@F�p&��e%��6��9�5��8j���>>�6N�{���y��x�� �p�7o�|� �^��s1*�`�[=�u�����}��!|�� ,��1��w:Xf�0+������/f���wBS�^���V@����B���Y�p�[U�p����^��U"���p���& '�V�����n�/�w��me��+�����]�p��O�
� ��J��,
�E8�f��t�a
%��/�7�%������fMY�HW ~�a�~�1)Y4b��E�Q��O,
y�.�._��<����yV�K�����I��hd�
�������;�C]���/�w���5u����)�y���_z��w���s�b��g�q�D��j�5�� �[�e"S�a�M���=x�`q6c��v������'6� o��F����`���u[1)�`��Z;$u�fN� �p ���o�-k
��3!�y��MgB�Y�n]s�)��N8A:Q<�X2;��4W3�B"Y�C����B���Y�p�[U�p����^��U"��
���P���.���>]��+E��G�p0��>��a���)��)9 �����U,�d`�(��^z�����t�If���q��Y#]0�k�.q�H�9Lxkl x�9��c%��l����Vl�x�����b�����}�Y3q�D9 [�y��'!�)S� ��`�����.Y�o`l��d�~�k��%px�����P?^�-O���%kA?���R�3�<�t��Q�CNW��o�Y��P0\��G�-D b�F�N@_k����m�6 �A�^��`;Ofu�D7�G1��X\ `�l�DkD@
-��
� �\��d��
�#��Rp��n?[z;�����X��XC�V���+�o#�����j�t��
& ������
� ������`X�T0�`���*�Y"0�� ��v��A�~�8y�Yg�����Gk��H Sl��-2�?���4i�8?q~���nL���%�3K+ z��9s~b ���;��� �@���+M�^�L�����i�O ?k�,3e�����������@�n"�q� w�q�_"8�
�)H �76�u���K�.�.r*�p9 ���A��S���
��*�^�
(.��Y���XO��/��!<;��� ����N��F�G��N��>G uT�j���F�
���~�v�5"+[���<
���WX�V0 ��)� �k�`�_W[m��`������`8�]�U���a����� ��
� ;�]�2K��8]t�E��,�'[�j%�����-O���6O *������q�F�sc����+(��.07���M�6�(���c ���M�&�{;���� ���j�u"pk��X{=������3[bk;��2a�I������%!i ���4i"^�@�����xb�5Jo���x�3 ]0L(j�&z����V���M�����/�$��<�����f��
����~��d������W@�����_��k`��
�wk�O
���j�-����j�-
��[N���.���>�{ ���<�1
cZ�a�\��U>1���� ���A��`8H���[���awWSrR0,�*V0���� 36 Zg���0��������b���gf�����-�3|�;�J 1�1�@m@1 �.�v��9f��R��x���.9Y/�<�0�x��� 't�.�����u �m�V^g�}�x{CV��9`�z ��p��]e2@�`�P��a��T0\��} Q����R' -t�t"t*���9s��?~�F�p�4v\5 +��$"��5"+�t3J���S����t�yf��]� ��m �����1���&f���������O�� N�26� ����g�
���k�`8�F��a�t�O�����
�(�#\��)c&��{���m~G�� F ��� ���[o5�6n7��j��G�����J~����y���������u�]�� ����6v��b�1��0��� ��/����a\���a���D`������ |:5j$Q&O?����Q^�a{Bk�f��W3���7o6�����
h�ScLcJ�Q 6l��J��8�D �q ��d������G� m�;1�<�u��u���P�;v��13�3V>��s��!Cd b�Gt*0��
����Ej������K�W\�����w�-k ��G
�������+�`8�MXx���C��i�t�t�|��
X���D�fmf9i* +.���� ����h�*r
�����^�p����@��U������z���������<��c������U��:w�0���R=�� F +V0 �+Y��4��a�����0�Fv��]�6!��w�}�9U9x��f�W/��#�45j�H �� {� He,�s �������+W�+V�O?�T&�]~���S�N?�y��3cV<�������Z�4����% 3�t1�[�O?�TZ/������n����gO��ClS*0�v?���:t�����P�7�tSZQ{���p�
�)�z [���XP0\�- �zs3�S����m�t���` 2������b� p���`�������3 Q��(*�`X�p��N(�Fd��l�� �`8m�B����P7O�
�`�_.�����j�t ]@�U�4�T9>��A����:M~�����~�i����9r�������l
���y.��`X�p.�+gy*) +v���wF��0�E�Ff<����x��i��x�S����������� �k��%,5��DT��+� �IDATI ���Z��3i�u|� t��i���- xK���K��G/����u��{`�����z�)��V����.3��{�g��S�<�*x�FO�
>��c}��~B?�����s�^�J��E �������]����0;2~#6 �0��+J��������k�1<:�r�-���).�_?�
�������
5�
�E���saa&x�����t������U��! H��
�)����� d]���)�`X�pfWN8��Fd��l�tK�`8]������p�OyJ�`�_-�����j�t&����v�f�/���?���=*�:M~��r��g�<s���o������?��)@� �R���� \V��T0,Z*V0��G�9%��y��#��d�[ � �j��z%����O7�&M23f��0}���T�:��c�h���{�D�( }����+�4������L����Z��s� T�_��x�^}��v�r���{��g^x�3p�@�T�cG���{ �S��$�y��g�+Oc�Q���/i>h p'l5+YG{��w�)������a�E ��p�� g�a�;p8Q
�9v��e�
�����E����!�' FYf�5���A�h0���t �4��N�hB] s�1����93��� ���pP0�`�p�f
%](m�`�0ZR�pa�#�P0���
��uq�U��+%��G�p���:��a�������`XdU0�`���+f=[���Q��+��"�>���M�.]L���}m�� �
4�L�:U��D���������Km��z c����;d��o��F`%p�e�`�%�� ��k�1�'O��V�Z�go��]�I�
',l�O>���8q��� ���?<u����+�����.�B~g�}�@Y�$I�YVm���2���aI���[K]q��� �� ���2y���)S�/���4����O�]� ^�<��@j�6�
����^�
(.���S�-��u�� ����Y\x 3�
( ����!���3��Zlv���4�
� �n���1 �VU0 ����W0l�����a�6T0������OW�aW�����`tw}V�
�]_SN�S0,r*V0��w�,B����C ���}����������Z`,����f��q�.} � �lS�`��O>1��7/�����i������]�v�Q/�k�E[e;v��3L��^�z�K.��4o��!�wl��u��9��w,��SO=� �/��R� �zO� �"��9S`��E��[=�n�����q�����~<{���'������oY?�P���m��-[��s r�!e8 ������+��M>x��q*�Nd�������@r��NR0l���bU@�p��|��M���N�- 3������o��2�<��t��^^tL�� BZ� =��S����h�*�`X�p����S[#��a���5C�y�;g'S0�3i����a� ���j��� �R4�| ����*V0���r���a�S���a���Xf��0��.=~�x3a������h�3Z�R�!��Wc�&<1Q-�@n^w�u.���)]0l�,!�������M�65^x�����~��g+6�/����^�Z�%k$E���z���m��b'�eH��1�\���/u$]t�E��z���d#�P���+ �e��?��������� tg�G@+P���H>{�l�`�����vr����I'�Tf��+V�i�� F`3��h��� �~�]w ���c��5��1�
g`� y��Y���^�I 0����a�S�+b q������`F *k����YB@b`1�7mf�� �: � ��`: ����@RSq)�`X�p!]����`8���`8��gK�`�*�w��m�`�_W[m��`������`8�]�U���a����� ��
� ;�]�2K�9 Q*���o , ����3 p���\��`@.a����W�>}�-���N;M���O �� 4 6l��^��x�b����)�������0�d�����2a������#�;�X�v������������)��[9Y^�Z����Lbb{��
����P/^�!lm'@i���\��g/k.� "�7�����#�<��� � �%5�D)���� �l<����Q�Es� ��h���g�!����`�5�
���� � �, �Bo��q����1s����A ~mXhB=T�REn��@"�t|�����M�P0�`8b�l��Z#����2��K�pC���KW�g�l1���aL���v
UE�����M���hXK�[�������i]��m�`�_W[m��`������`8�]�U���a����� ��
� ;�]�2K 3^c�_<UY�v��5b��,�L^�O`%�j��A �l����c���t�����Z����%f���'�` �� � c;��'�p��K (Q43y6��8i�616�L�~���@j6B1��;�Nz��U���o����myc�}�9� ��qcq �/ck������)���N�^��r\���%48mp�m� ��_|��8�Qn�����6�S�6�� ����]l
(.��A}�����p���f�� �K�fgD�=�XY��YG���;n��T�
� {����Fd��nI��n?[z�V���+�oC�����j�t��
& ���������;!7'm�n��3����0f�?�>]~���Sa����7�}�K8�D'fL�6v��b�1��0����v���V@��(�`X����^:`�sr��S��~�[�EK$K�������G&���S8�'k���S�+�����o���u�Yb��
��87!�����q�b I� �1������<�L �L�����>�G�D�{�=�H���;��^�m��1 :t(��BY�.�R7<v�����������]Bin�j����A)����o y�a �x�"/�m��{w mm c����������R��vv��j���� %�����+�;�c�]�����R�U��R@�pQ5wn*�l� |P: :��$<�Y����
�o�rpc��������k�P0�`8j�l��Z#���d*��;��o�tJ��%��x��q 3�1 h��
���I���.���>]��+E��G�p0��>��a���)��)9-||e���I �a#DM�>]�}�e����xK����sg�J_n1W������~�.��<��WzOW�� ��OV���h�:�\��L����}��]I�a�S��a���s3v�&D�i�1y`#N>�I(d�I�!� ~��RGk���/�m<#S?��M����5�Y��:R7�,��\��
(K��F�`~���'0 �u���� ��Y ��3L � y�G[&K�io�=x�p�x��l��1�C���I(F c�;��m��&�6�l�q�?����������7 ����}���N�e��W�
(V0 �������`������ 04�t��w{>�0�`���A��`�_u�����j�t��
& ������
� ������`X�T0�`���*�Yy��=7�>c8��\�L�V �@K��\$�=9/��rp~@��� `�f���4���J�~��'��`�V^h���h�&�] �xh��m'��������9}��1 K4��@Y ��C��@�D`��-�
;��� ��&L0�k���^7FS�
��y����� ����
���R���`8�>Q�V��k)����V��+v�h0�(Fw�gU0�`��5�4?�"��a�NW��2��������*P
� ���\j� @ IDAT�� ���� �-*�� ���)x�E�(j�� E�/4���6�1jO ����
BD9<A�(�{��"�y��o%5���3�53�3��|vg�����_UO�<�z�������I�(p�����__ �?u�������[�N��oDg�N��� 6q�D;����6.������_�|��r���Ak�����W��Z�j���v�K��������W�?��O<x���zO�����m���W��U�Lk�� k0}��{�=����W�L�L�q������ ~�������Sm������ ���{M������f=��k{����5o�<���A���'w?rO�?��S�����>[30�Cb���?���2� ���g�k'������������Z�je��w� :�����5kl����r�J�������I��M�?��������O��m{�������Z=�:�������m6����{�g��Il�������{��)��kSl�������f�{de'V�������W�e6�gw����m��F��b���]�����.�� ���~<�F�f-�4
]\~������?���w�]9�D6lX���L�`��u�h�g�f�4�h�
��?������6��a;mi��w��o�>��oq|�r/2N��o<��\��Q���]� ��7���q�l���v�W�����_~�|� >��m�v\?u�-|��+W��������Y�~�+��,���I) *Pk�3E`8���h���o�Q�F��[�li���]����Bs����9`!0����o���,0�o���'0 V�BY.������T`8Z�P{��.0 J���.���K����vr
������I)P�
�j��nfZ����-fn�h�R�p�\��8R`X`�8z���;�������n?_{�a�D�_���P`8Z�P{��.0 J���.���K����vr
������I)P�
�j����@��q���j��,0��t�:A`8V�Q�� W[���(0 �$�������t��P����pat]����p�>�����S`X`8�}����phEeO
H�RU@`�T[^�-b�����p��g�U�Nd������ ��j�jWF`�����D���&��%�^?���M��1��P���������q�J��raH`��*0,0 �����I)P�
�j����@��q���j��,0��t�:A`8V�Q�� W[���(0 �$�������t��P���"� �{�R��C���������A�����C+*{R@
�������n)c��c�=���w" g-]�N�UsT�2���.v'
G7��p�.���1]`8����#0\�C�*0,0 �O�'0��z_�5&0 ZQ��R�T.���uK�+ 0,0 ���u��Y`8k�bu��p�������t�;Q`8�I��u ������-� ���� �T�a���}*�=�a'����p��j�1�����������p����[
�X�a��w�������YK��c� ����p�������M"0 �K��~L�ha� F���
��SA� ;9�� ��Lx���;q���v�8�j�j��������r��6l��8UOu�R@
$F���4�**JG�a��b����,0��VNv��� {%��*0 ���������t��P����pat]����p�>�����S`8`x�� ��S�5����n�a�� 'c��^c ~\eKZli�� ig�uV����H) ���pb�J�������p1�v�DNv�
'��|�����nC��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0� 0���_� /�`�<�����;A�r ���U�:t�h� s�
2$�UT�����W@`8�M�
J��S@`X`��z�w" '�U���~���^���
G���p�.���1]`8����#0\�C�*0,0 �O�'0���^�d�M�<��}�Y{����v�8 �k��:� ;����XE�I
H){�c�D��(=�����{'��p�[U`8���k/0��H���pt
G�j� ��C)Z;���=t�����TP{�NN��������+��bO����Y��Y�-�v�X��j���{�~���������?>V�Se���IQ@`8)-�zJ�R@`X`����w" '�U���~���^���
G���p�.���1]`8����#0\�C�*0,0 �O�'0������/m���v�+�l���f�>8hw��� ��Z�}h]�j��a��
�U�T) �@RNJK��R��.�������nU��d������W"����m(0 �K��~L�ha� F���
��SA� ;9�� ��}h���4;����9V��|a�>�f]>}���&0 ��Qe��H��Ij-�U
��������Y`8��*0������J$�U`8�
��u ������-� ���� �T�a���}*�=�a'��p�����l�z���q��Z����. �d��"0 ��K���(%�K��u�R !
'��fTM�D�H�� $0 ����b�Y������G`8Z�P{��.0 J���.���K����vr
��0&0 \R�R�4.�v�UK�X+ 0,0 ��e��Y`8K�bv��p������p1<M`8�Q��u ������-� ���� �T�a���}*�=�a'����p��
c��%�A) JS���lw]��������YV�;���.f� ��A�Y ��j
����E`8Z�P{��.0 J���.���K����vr
��0&0 \R�R�4.�vO�U�~��������S�Z�jY�:u�A���Ik���5l�p�c�G�NV������,0\�Nq���p�[(�� g�S��n%��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 �����ppIeP
H��T@`�4�=�W�p���v�������00x�-������.��b�Z�2��-�
'��F��;����I�^����T���\�$�W`8����u ������-� ���� �T�a���}*�=�a'����p��
c��%�A) JS���l�D^�?�`O?��
><U��;w�l�52��8 w�uW�����I �7�R@`X`8Q ���z'��pB����1o� �'0��P 8L`8����u ������-� ���� �T�a���}*�=�a'����p��
c��%�A) JS���l�D^���oc���!C�����Y3;����E������� ~h+W��n�����^j<�*r8%Wb� '��fPQ�D�@� "0 ����j�Y��C��H`8Z�P{��.0 J���.���K����vr
��0&0 \R�R�4.�vO�U���gg�yf��[m��=����cG�?��S����m�����#�X��u�v�����&
'��fVK�D�L�� %0 ����^�������[G`8Z�P{��.0 J���.���K����vr
��0&0 \R�R�4.�vO�U�Z��f�������
7��N:�$k�����h� &��+��c�1RM��S� ���( 0,0����Q5�Y`8#�b{��pl�&�� g%W��n ��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 �����ppIeP
H��T@`�4�=�W�f�[�t�-X� U�����v�mg��������?Q�D �n�������I�N��D( 0,0����a%�Y`8C�bz��pL&�j g)X��n ��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 �����ppIeP
H��T@`�4�]W-b�����p�;h���Nd��,�����1k�jVG`�����4���F��%�^?� �R�0v� �{�R��C���������A�+� �T��(M�K��u�R �
���fY9�D�R�� .0 ��fu��)\ O�n��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 �����ppIeP
H��T@`�4�]W-b�����p�;h���Nd��,�����1k�jVG`�����4���F��%�^?� �R�0v� �{�R��C���������A�+� �T��(M�K��cu�� ���/v8���U��5m��j�� ���2�Q@`X`8?=-?�x'��p~��U)��R6�v���w.K�VW`8Z�P{��.0 J���.���K����vr
��0&0 \R�R�4.�v��U/[��������� g ?����=���7n���a1�X[�+ 0,0\L��;�������n?_{�a�D�_���P`8Z�P{��.0 J���.���K����vr
��0&0 \R�R�4.�v/�U�\�7o���=�f��i_��}����r�J#z�/�[l��]{�����[���nO<��5j����zk����5k��4hP��P��S@`X`8w�+���Y`8���,Q`8����%0\8�C�,0 ���p�.���1]`8����#0\�C�*0,0 �O�'0��z_aL`8��2(�@i* 0\��^���N�>���L�bS�N������W�^�N�o��� ?���k����^h?���m��V��{w���mm��q���9Q���E���{��,0��V�c����Sm������ ����{a?��l�m�������~��Y����{=�\`8�r��ptC G�j� ��C)Z;���=t�����TP{�NN�a�������K*�R@
�������j�����o�_~���1�E
S� �W �l�`�c� j ��-_���[o=;��s��C��]���w'�WQ( 0,0\ ��������?v��3�z��ULMy-��,0 )Obv
'������p�D������j�*F �R4�v�l�wN�W2Y�������Kmy�c�:���B���'�G��^u������o����F�� K;�?��o��>�g�f�4�h�X�UIV��0>?~���7�a�a���V���e���9}?����������k6v������Sf&��1R@
H���L �/� x����=��c6q�D[�x�{X�v���E��0����/�� ��n��^~�e���O����uk;��Cm��A�{���T�04�|{�����t}#��s%�gH����V�n]�W��:�e����Q �m�y����n�=�_��o�A���Rj�����d��`�3���kl��Q��g�l�N��8�Y�����.���=�������������#������o�k�6����_��� �nhG�K��T�y�N�:n���K�� �<O�Y�,;,�R���f����u���v�i'k��IM�&�|�t� �WN<�D���������5����k���z3^����bw�u�������� k�g?<c�����p��Km�^]m���+z0�>�_|��������)7�5ki��Q�="f�����j�,[��v����������|-X���2z��;�#��
�*�?���6��n'��� z�ol� v(PE�S,0���>���y6�� �])�/4���o�����O>�������|��������=���������4|!J� 3�k��Y��H�J� >�=���,k���K�.u~i��p�
m���3�sy~f �3�~��#� �s��u~ &���cD��w�}wg������z�=� j������n�eR���R@
U@`8��2� �K�,���G;���E��q��np�e�]�c���������W^y�1x�G ������o�[o�������y���[���m��7�*Z�� 3�� p��p`�Uf>j��|^�S|`x�Z0|��l/v'2� '2_&��+�T�{��'�� t�������5�+���M`8]�d��n�\�a�\L�����{���Aq�-v0�����z�Hp�l��hQ�`�� ;Y�>}J 3�_t�EE���� 3q��Q gx�����|->���0���{���/=
(`(0��.5zW ���om��Iv��w�o�a�5�{���v�qG�6m��)�s^�p�}���.� � ��Z�r*w�yg�|!Kr~e���f|�s��q�L_���0�x�
6�-���M�dID�� ~�a7�8
0>��3\F���������P@`8���:
����s�=����;��`�Z���z�['����?�� �fY��ao �������7�h_}��1�����nrkg:����k|����^�z���[� � ��'l�
(��a��g0�=_�W�R�XSI?��%�J3��2���k���� |��9����;���pXqO�������d��UH�"�]�b�k��vd��#��R}.$���pn��F�.]�8��1��>?�5���#�g d��'cz1F �J�����D*��1���pm"�� nW�z�
Vq� �O��RI���%�Jz ;���\d`Lk�Z|�2>F���� �V���(|��'\4�"�t�,�0Y*���*����;�p_�L��+�=������ <:���m����m��62 �c�������7���� ��f�[b�O��=�Q� s��@j�����LH)Pm��-�N�T��������N7C�T��� ��G �� &N���0�.3�pv}��,�����g=\�k(e�w1 ��0�k��qZ�� ���.�/We����|���U�`���k�T�0_�� �Z0�k{7a�y��q��jT?��{�U:P�`��-
h���v���������b�D�;��Z��b^c�6),�y��9���� rb�`����1��)0����_i�a�����&�i�a�1��B�� �������f,��D����m���M�h�������>s�+�d��m���;�8;���\4q� �+1�L+@^�����Q��
-� �%�(���d�\�l��p�N!CR@
TG�����s�R��w������Kn�%���rH�L*6���0����/?��s������k��}���d"��&0����<8V�y��K��p�{�w" ���*�!_���X��%�}tN+���z|��'0 �F�������t��P�����U�`�o��E�JZ`�?���m�L�@k ����RK sO2N
��-�F�7N���R�w���^{�E�����
�0���~�%h������aRP��9�e�!r�%N>�d7a�c�F?g���o�����}����]������?���� ~��>��-��wR6E {U�*�@!.��%V&�� �Lb�>�����
^�L�0N\nfn������I�Adi� �9�&\�a���w�u���������?��d���%�;���M`8Z�P{��.0 J���.���K����vr*bX`8�}�� �������&��M�6.�h��!n��������x��� ��'���,���>����{k��u*�4>�g�y��;�<��n���u������u�]�E��\2~l��I9M�5��<��<���:R�) ����p�/��� u�G���6��
��`n��]J�L�0'�s����s�=����t�Iv�����Y)�z�h�����T�;�����?��`TW`�a�[`8����u ������-� ���� �T�a���}*�=�a'����p��
cY�a� :��C�Z�[n�e�z��[2T|�xI�g6� 1b��2�K�;v�X� pw��A���T�e�Nc���/�h�]w��&��0�h�R��
R�)��XO�u6�xc;��3�:����[6`� �<^x�w:����]v����k�Nx^����,0��,��C`8qMYa��HY�S`8����u ������-� ���� �T�a���}*�=�a'����p��
cY�a|����6I�0}2j��>��y��.�4AN7]{����W/*��5� z�!� �������K.����j+�~p�m����{(?����p�B�����~) ����p��.��H�q������� �%��n���%S0����K��������K%}��g;���s��=�I���������;����%i '����+0 �K�
G���p�.���1]`8����#0\�C�*0,0 �O�'0��z_a,K0|������ �2YVU������j����[+��3�t�]�v5"����
�������[l��
8�.��r)L���6|�����]|��6{�l�����~) ����p��.�����[7��6�� ���w��A3S0�� �^x�KM��&���W^i}��qk>����u�������Y`8��*0������J$�U`8�
��u ������-� ���� �T�a���}*�=�a'����p��
cY��a���QG �R>WU�I��7�8���� �����7����� �G� ��%�y����O���:�*�.���E�l����RIW)��R �
�X`�7c��l<��c��[o�l`� ����K��#0�?�]��0i�$[�x���w�}�u���7n,��D�a��"���2�Y`8��*0������J$�U`8�
��u ������-� ���� �T�a���}*�=�a'����p��
cY���.���<�H���C�UY�z��{� '����������o����@ ��1c $�w�}�����6�=�,Y�|�o��������,��������V@`8���`y ��f�r�0<��#n��3���: �5r��A�l�����������m�m�5�g����� f=�p�f�l�=��������m[�:%(y�/Y`X`8��8��Y`8M��N`�ETY`8B����n8��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 ���X�`����wi���iSeU�w�#�8����{����r�)v����'�3�/��%M5�9���d��<y���[�E1 g����R
�BU�\G�`�����#���o� ���n��������o_k����m5t�P{��w ~��G]D0P�u���"����K7�������p�8H�N��#�
'����w" �&a�/�L^�0y�������`��w��$��?��l�m�������~���u���(��������m)0 �K��~L�ha� F���
��SA� ;9����W� �|��v����������z�������V�^=�4d�� "���������� �n:����� ���,����LT�1R@
�J��\)+��(��:w�\{��� ��d'��Fmd��P�vm����� ,�u
�@�������E �~��5.5��H���ys��)P$Z�a��Dw�2��Nd��2�$�O���5X��@���n4��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 ���X�`����;��c�Ua\!8 x��k��������
<�5� hz���-�K w�q�7]�q� K�.u����n����pU���R@
�T����+��
�\��.\hO?�����������h �%��y,�����4o���������W/����;7� �O�������\�w" ��I� '����Y`��&I�#0 �r�������t��P����pat]����p�>�����S`X`8�}��,���a����D�V��K�K�;���[��iS;��C�:����. �����{�I$`��3�����m�����������*e�R@
�P�� �+�� �w����������
��' _�<� ��_�6�pC����[��w����f�
���(� EG��Ex'��p�[U`8���k/0��H���pt
G�j� ��C)Z;���=t�����TP{�NN�a�������d�<��������*_|��M�2�����={��py��'������� d��� ����s���l���]t���S�R 5i� �:��sm����U���R �
�Za�_G�/�u�0�9 %����}�����g����8\��NZ
`0 �m����kWk���[O�Q�FJ ���������p1�h�DNv�
'��|�����nC��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 ���X�`x���w��/��b�TY}�y� ����] ��M7��HEM�J 0�-��bc��q�� &���W_�2\�&qEK'>��S6j�(�������~) ����p��V9� z�h������ �:�8[���Ha�0�
��02i<�"�V�
S�Nd��d���p����^`�+��W���6��%�^?� �R�0v� �{�R��C���������A�+�e ��)��7�� /����v[�U���� /�`�^{� b�7����v�u�����9o�� ����� �E�c�=���Nr�-7�d�(���=i�$����m��Y�l�2E G*��R@
�S��|������H�a���:JB�Nd���4X��@���NX�UR]��hq��u ������-� ���� �T�a���}*�=�a'����p��
cY�aN���� ~��v����&M���a�Z2�|������o����]D/�7�|s�s�=]40��@`62^>���.�4���`�L�r�)��K�����.�Z����������;\��������n |���_R@
H�|* 0�O�U��) 0,0�QGI�A��,0���������� k�J�+0 -��p�.���1]`8����#0\�C�*0,0 �O�'0��z_a�`P�R� p��D ���z�w�/����M���
�:u�� �m�}�����4�m����/�y��W]���~��������8�����m��<������;���^z�x~$ &��H�����z��X�
�) �@�����\ ��� ��(�[h���1;�A�Y��z�j#�4)8��s���:�Y� �U@`X`8����Nd��j��S�c�5���p
������
"0 �K��~L�ha� F���
��SA� ;9����W� ��w�E��h�-���E�{Y���o��o������oK�,q�� :tp���Np��� L4���n/���� =�f������ �z���O�f�\�|_�&� �����; 6�' t�A6p�@���$�R@
H�*�B ���
0Cj��1n������f�mf]�v�v���5|:�LJbPe0���l��9��eK���[�����c������p2zjf��Nd������Q�qm���%0��^q>Z`8�u��u ������-� ���� �T�a���}*�=�a'����p��
cY��#�<�E���1�>��3�K�$���:R={��%��U�V.�����w�=�B:�:8o���.�@��� �u���It2����8�>l����������m���������c�=���N���R@
H�|( 0��K��/���9�cf�$���'�����;�YR��a�}`���C����r3��y��^D��\T�/0,0\L �;�������n?_{�a�D�_���P`8Z�P{��.0 J���.���K����vr
��0�%f]� w��xF~����O�l��`���n�����& 4k�n��V�P���+��d�$��w�u.���'R���������.H���'O��R[�t�I���y�������R /
�E��.$ 3�20n����.���X�o�yF" g$S�N|'N� �DN%�o��hU�%����N`8Z�P{��.0 J���.���K����vr
��0�%f}_���&���Y����D��V�m����uk#�4K ����HEMfKl����E��t�Mm���w�I5���S�L��7��=��������,�_
H)Z�����^9��0�/�n���Ri��� ��;�� �E ���(w S��Nd��d���p����^`�+��W���6��%�^?� �R�0v� �{�R��C���������A�+�U � ��������5~�/_��0�7m��-{H*i��*�H!
,���&� 5��g|"R����<�=�,����a���w�q�]z����O �++S��R@
�B��\�*��(�������8�4�� &�4�*�YY<<Fm�Q��>�a��b����,0��VNv��� {%��*0 ���������t��P����pat]����p�>�����S`X`8�}��j�a"vCl |����9�f�������N.��z������� ��'�|b/������s�����o���c �nQek��R�"�+RF��)����9� �={�����n0d�d?k*�r�)n����G�/0 )K��.�N�����nU��d������W"����m(0 �K��~L�ha� F���
��SA� ;9����W+0f-�g�y���� wi{���
2� �D Gm�V�rk�r�-(��q�N�l��.�t��u�N�>) �@N���2��`x���3�5k���p���6w�\c�$�G���m����=����1Uvp.�>%0,0\L=�;�������n?_{�a�D�_���P`8Z�P{��.0 J���.���K����vr
��0V`0�z�&L�K.��V�X��$���?�%�����|�����e�\��������^r)��� |��v�Yg���k$�R@
H� �@$ R3��0��]w�e ;vt�#��H�A1k4=�`z�G���k�t�M���k 0��(�W�a��b����,0��VNv��� {%��*0 ���������t��P����pat]����p�>�����S`X`8�}���a����z����z�3g�-Y��X6q��vsQ����3�*����������+��� �u��7o������k������k%�R@
H�J�D �+�Q`�s��F�h"�_~�e7n�=������A�.������ :X�Z�\e���I�� ���fS?�D�F�� +0 �6�N����Z<��n��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 ���X��0���wn��k������� �W����S��� ��g� ����?�p;��#�O�>� ���� J) �P@`�
����+P�p�
�q +D O�4���y[�p�DI���gO�g�}�W�^/_�����oC�u3�X���c�f�V<
Oo6�%�5d������n?_{�a�D�_���P`8Z�P{�C)YX;���?T�����RN� ;Y����_�\�)W�\i��M�w�y�E��'.z�5)��D����[��m]���w���+�>mR@
H�B* 0\H�K����02 �`�{��G�l+Rl��k���[o�U��n���}����E�w�����{'��p�[U`8���k/0��H���pt
G�j� �1 J���.���K����vr
��0V �x4q�D{���\�C� ��w���w]����P�� ~h_}��� &�0��I�&��U+��p�.]�Y�f����I) �U@`8[�t|�
d�1���u�I)=b��7o���cv3��;�<7�/[�L`8�VH� ��������;��+�)���{ eV?���tJ�Q���$0 �K��~L�ha� F���
��SA� ;9����W� /K���E���p7n\.-S0�� 8l-Z�Rq��������5j�J���N;�
7��fd)�t\Z9l=������Z�Nd����CMK����8_`8 ����*
G�j� ��C)Z;���=t�����TP{�NN�a�������K*�R@
������y��L����w��_{�5{�����7���s�Z��um�����]�����C`��V\�������Y`8��*0������J$�U`8�
��u ������-� ���� �T�a���}*�=�a'����p��
c��%�A) JS���l��^u�`�W�b"�'L�`c���%K�������~�J�����p1�h�DNv�
'��|�����nC��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0�g0<�#�^s�~����2�� ���[�� �m`��l��a���*#��H��Ii����`�K^�|�}��g6m�4{���������+W��NIQTo����C{'��p�[U`8���k/0��H���pt
G�j� ��C)Z;���=t�����TP{�NN��<��^���W�u�=hw����[��Z�_-���'0 ��Qe��H��Ij�����o���/������RA :���f[��3�" 0��~���]jiRL��w�A���$L�A������T�;��+)�N@#ePE�� DJ�!��
%0 �K��~L�ha� F���
��SA� ;9���/^���{��������6�8hw���5?Y�U?Z�f���}����>:V�Se���IQ@`8)-��z�Z��=� ��6mj;���m��FV�^����u��N�j�G�6 ��+l��6�;�����k��- o����C���w" g�[���[�T�>���-�g G���p�.���1]`8����#0\�C�*0,0 �O�'0����_�R|oNy�>]����U���9N�~������l�����{�m p@����H) ���pb�*�eM`�?����
X����V�� T8 ;K�.uPx��5.�j���L�B�yY�����$�����InE3��d������W"����m(0 �K��~L�ha� F���
��SA� ;9�������s���{��}�]��^���9N�~��Z4����c� ������N+����$��"�@�) 0\tMZ�d�(�+p�n��YC��V)�W'0,0��^��x'���/�$���p[�|���k��=��-'0 �K��~L�ha� F���
��SA� ;9������K?~�=:�Q{���d�&A�s����a���|��Y�����X�1 ��Q]��H���j.UV
��������Y`8��*0������J$�U`8�
��u ������-� ���� �T�a���}*�=�a'��p~�����l^�y���A�s�����O�b�
k9�� ��#���8����R@`8Q���J��P@`X`��z�w" '�U���~���^���
G���p�.���1]`8����#0\�C�*0,0 �O�'0��z_�5&0 ZQ��R�T.��x�����}���)��n�����N���W�ZeS�L�e���� �f����=����6m����@�a��t�`U�Nd��`�������y�l����������� �5-�v����4E����O�������7�~��Y��=�R��S`8Z>��h]B��c��p(E cG`�0��.U`X`8t�
jO`��)0,0 ��ZkL`8���'�@�* 0\�- �� z�!{��'S�t�b��v�m��V�^=[�t�]��6���1���h��.��b�z��C���( 0,0 �n�
��,0 L��6{��7l�����O��V�l�N��y��k=����
����[c�W�h,��.9� ;��S�Z�����S`8Z�P{��.0 J���.���K����vr
������I)P�
�j��n��u�]��������w�m�l��5l����� ;��Cl���cB�?��3��S�P&e'
� �}�N,�
�"��rK��#���X��w" '����^z�%����6��v���.f�;$�Q������j}�����R�v�;��s���*���hy��u ������-� ���� �T�a���}*�=�a'����p��j�1�����������p��|��� v�a9������ ;�"�*����a&W��;�>��#�4u����u�P8�`�{����}{#���7�DNvK � �m�6�j�v��l���n�b����Vk� ���v�Y' ���H�& ��&0��V�c��pnu��u��\+� �����i�,E`� '0,0\�;����+�F��R@
d���pVr��(�{�9�4iR�_�m�� q�n�;��?�`��w�EBon��
4�Z�l�� �� @ IDAT��P��u�0N�W�N����c����������H(�z���f� �_o��w�����Nd��~�(Z`8
o�������]� ��D� ��m��|�]{�I������[�V��IX :�<���56r�H[�r��d v�4iR���D~LNb��Rg��_�H�;�a��X�_�a�<����S�����������p��|���Y�������XhP������a��?��T{��Ef��6k������e_�M{���� �� �g�}
]�������9�:� �������pJ=�?Y�D���N�� �� �gs�y<G� ��k�1+0�tsC�bE`�-�N
H)P�k���M)���������`����s���1�w��&[����W�/�a��-6p�vv�~{ �*��N�|a
�O�c�� x�z��U�������c�]�SM#0l�T�����7�9�5�F��.yN
�I�
eT`�))0,0 ���v��z�R@
�L�����������`x�����_��;��e� ��'���g��56�kk;��^��y�x5/F`X`���(G�S�
�:C�� �H�<����9*N`X`8G]+�Y�a�����p��+��h�wR@
H��( 0\�tn� �G;�e�����������wk�����
4���_���~���T@`X`��z�w"+�t�[U`X`8�=X`8�4������7~LW*� �'��y:�� ���������O`X`�f7R����k�=R@
H��( 0\ �tN�B�j�*{�����M��}�����o��x_}��m�����M����u����5k��9��x.��l�����nU�a����`��T� �:C���1]`8G����p���q1��9�b53/0�����T�l����h��R�:
WG5�S-p����6e����m��e��qc;��#l� vH�\�t�}��G����{��g@a r��
����7w�g{��i;����8N��7E�����pQt��^�w" '�U��c���SM#0,0�� 9z��t�� �'��y:�� ���������O`X`�f7R����k�=R@
H��( 0\ �tN�
�z��6m�4{��gm��9���?Z�V���+���������w�y��z�)���� � ��^�f�K=M�i"�;�0;���m��6����g]'�_����;���w" g�]���S\�.�)9��S�!Go��.0�#��dV`8OB���a�� w���v� ��F*��pyM�G
H)P ������J`/_�G� m��_l��y��8RA�0��Gg�;r�H����\*i��n����r�J9����k��l��Av��Q���G�a�����J%],m)0,0 ��,0�j�a��Tg���� �g��y<G� ��k�1+0�tsC�b%�`��������� ����qR@`8N�Q�uY�z�� 5j�=��#�b�
[��m�m�u)�O8���� 1b��u�]�y�;w�^�zY�>}���~�w�}� {�1[�h�m���� {��W_m�l���"�?��E����p�4����pl���p�i��S�!Go�s$l��
�Y� '0,0��������Q`X`8�
�����������N�j/���[F�� >�D�
S��R .
��%�� ��/w�t�|�M����K�.��S'�x�� ~���� �'�|�j��m@�����k�F�5��� k���s�m���;����w�����%ui������Y�������������Nu� ��c�RI�H�<����9.F`X`8�]�f���~��5���� w0�x�b�<��1cl��)�x�GX�
�_��H)
���p��/�b���[���K�����o�q��\p�����A�T���~��'\dp��M���.�!C���X[�� ~pQ�C�u3�6�d����d�{��-����d-�k.������nU�a����`��T� �:C���1]`8G����p���q1��9�b53/0�����T�l����h��R�:
WG5�����3�8�&O���U`���� k����:���,>���m����l�2�m����3��~�����_� `/��������������>�z���* 0,0 ���}��Y`8{��t����p���:uN�!0,0�� 9z��t�� �'��;%�<O�6���_j�{ k��G�j��b��d������2�������6�h� ������5v� G� 5�%0\#�t��R ���pJ
���_}��K � ��7���>�<��u@��5k�t�v �a��v�1�� �H�o�=|��'� /��v�;�0�i�����+ 0,0���[����,0\N�D��m�N5����p�3��� ��s$p��
�I� #0,0��.V3��N?�a����H��.���H) ����puT�9Y)���_�QG e��O��7��~�������k�l�M����|��K=c���q�t�v�mS��0Lz���� ��
rk5a��8.������Nd��d�����pl{��p�i��S�!Go��.0�#��dV`8OB���a�� w���v� ��F*vE`��� �����w�&M�X���]�J�&,������� Y.9�m����M7��5jdu��-{����Y2q����f���f�mfK�.����������k��s������1W\q�����_���sj���������]=x��O?�7\k�:u\�������(
������������� �,zR�?Re��]����g����m����23����<�R�hH��GPGlc����?��,>K�6��-��:�nm��qm�]��������%���*���i�%K�����%mN��>����K���7l�T&�e��U�6Aul ����Ee�G����?��&mHy��@U
W���_c\N:�${��7m�
7LE o��v)��~�����+v�M7�� �>����N�n��`� B41k�!{������n�����M�Nv^����,0��.I��|�p0�i���6f�B[�g�Y��I��_�������f=��k{�t�=*�X����m�f�m��=��i��H���N��z�
�"\���IE������<��U+����u����W��a�������Kg'J)n~LNv� '��|���}_�����k�t Eb�v+�3�j��q6w�\V�}��E��� o�f��yM�Y�cZT�T��� O�0� z�!#����=����+��B^�&�����z5j���\8����v���������s��qZ�������;�(��_�w�y���d�b���(��%9������~��g����:��`����g�b�-�u������������m�� ������{������o�m�&M�>���e@.���eK����[�q� vpefr-�q���������Y� ������6 �K;��V[m��+�>���7��������������k�������F���{�����w�=z��=�����Gm,=���O���cS ��s�q�;m�4��?�!����o��������Z�>}\yea-u�7o�=���v��w������������3�>~ ��:=����I�\r��'���M
T���pU
��5V�[ � $| '�3k���~����� �(3q:w�l��_��h�'�_|a��z�D,.��B��o~c D��C�a�������
�DNv�
+b8�=X`8�4��Nu� ��c��p���Y��< ��b��s��jf^`��'0,0\�����a ���>j�=���+�U��o�����������J=z��US���Np��I�����.�����~��B�x��v��}O=���H�~^�{���{�v>���>:�_�g�(5y�d�x@D/��@[�$� � �:����_?�7��<{�5��-����k�?�x��c��m����6&�o)�<�P�g�*� ��b���^���.���@\ �1x��G
��M����,��z���>�E��L������8� ����{]�����> :����z �= @���k���J ��?��3��u�Y. ��g�q����0}�� �fA�T� ��, @i>f����� T�q� ][�{����m��\���n�^z�� h ����k��C�*�����U@`�t�>oW�� c>0�0&r��SNq
� `�������>���d� )����� ~ X����$�������[�C��p�������w" �#K���m�N5����p�3��� ��s$p��
�I� #0,0��.V3��N?�a����H��� �����������8� c�B�e�F���3g�m���"A���L��|������C�7�L��w���A;�������^x�e�$2���"�p6�����a $�+����� �H���:�
�$3'���@S�c?��J����>�l���z������������ |� Q�2�?>��@ &
� 0�hYx��3���lG{��� ��H ,� �O:i"n���e�]\Dq�����k��� �DY�b��� �$&-4<�@�>e����0u"s�dTZ�p���v�>� ���~C�`���g��� �~�����r��� �z�����. ���>��3�<�>��Cg� <�g�}+����������Z>p�e��EJ �<�H��;RG<��nF ��Y�Jz�>��&� � n|3,jVOA.X��X�a��w��Nd��5J5�"0,0\�n��S�S: �:C���1]`8G����p���q1��9�b53/0�����T����0�>���� Y��Kv�!C�8��n p �$R��#�X��kg��~��3����>���� ���e ��c�=��B >"x�M�)��l Q�+����T�� �������IJ�*�v"SI����RORK�'e3� �dY���;���t0 �~� �%%3p�� Y�_<���� ���� �n�L=�7 -)�� J�0�H���8[ d�� �����O=��~{���J� g�N���~ W�����H9��<z�0�a�����C l��������\������:�2� j#7��?��9��f��y~#�5��IG��&5�7�Q9/j#ZFB�0�F"���DV���I�L�D% S# �P$�?k x�P'�3�T� _�v�
�rH���1���d a@c "3��Oz��� ��>���H�Q�urA.h \�w" 6�������2/N`8�_�yvdf|����������m@?� �V�\[�����/0,0���V�R��p<��\� ����?Tk �m���0Z��=�N?�t���W&����
�� �D��G�8�9�~�,IY �d}\�����\���@'6���(�3���� pP���� H��o����C������KQ �lr���ZDQ�w^���e�t0������(Z@1)����D cXJ�4i�9�<�����>}#�����s����KD�.�T���hH; ��N �?�xO���(���y�`����>� �����K9<�EL=�����6�GP>��h���p��a� ��L#J�y0��6����/}�kh�6=9���$@�r���[dR�z�7�f)
h�����#��5��Lg=f��r����M
d���p&*��)�;43���1� �`�7D�0 ���@� >P�@������ �|Xs �c�av ������^������pr{o��{'��pym��G`X`8��U`8�4��Nu� ��c��p���Y�4����kSl�������ks=f��1O����5k�[?r���������{p��&0,0 ��-0��G`X`8�}Z� "4���
� s�;���fe������ ���}t'�h�9-[������Q�F��� 6��V��7 2� �y�`�=�1���?>�W@)�6��� 6I�L���eA,~x�H@&k.sm�R�JN h:�N��=�� �%�[�n�Q�����?����O��S';���\Dr�t�D ��$>}t#��M�����:P�9s���2�6�fh S� x�����e�s�g#�����7�
�~��� ����&�7u#����`hG}hS�Sm��)�0Q��3��?�c�k������H�I���S��& ��u�� 8p�K+���_y� ;v��l'x=�����] ��M
d���p&*��
0k���o������2>����C�-'�|����� �����|�3c�`�'�8������* 0,0���[����,0\^�$��mN5����p�3��� ��s$p��
�I� #0,0��.V3��N?�a����H��� z�!�HE�v��w��^|�l|����+0p��Ar����:� �t�4 K���%K t&�u���s �_�+W`�ha���hI#
��|�*����^{��e��{�Yg��=����`(K���/��E��Co�� 6q�D�y>�OO�j�a�7���{"[���b�&���m��E.*�u��� ���z�Z'8�,�1 ��H���K k��r
D����������{ ��i���a����[��tP�~=�5`8�L�>�]?u���\�K���D��&�7i�������Q��'���@�7�Sg��I3^Q]��z��
����9U��2D
�&*��>�I�A�|��� T��/��b ;�et���
(@��Y99�����y�hy*�;���$x���Q���Y����������7~L���y2+0�'�s\����p��X�� ;���kv#�?�20��I�$��l�mx��1D��� &"��[ou����@9�v0��
_ �����(W�����]��;���r��D��v1�+�:�I�*��Q)���x�"�����/����`�H���w��R>�GZ���=�)Q�\3� e�������s�:�Jf����T���.��q� .P����={Z�-R���04�^�@l���N\��Ck� h���+)�Y���,�2!�t�0�tp�N��/��F�0i����.��s�K)MZl��/���!'0m�\f�l��o>x=r�H�7��&�*Q���?_�R ]��t5�>�
0���[f��n Z��aP �G�1��C���F<f�0(T���� Q9U@`X`8� ,���Y`8��.N`X`8p�
gN`8�%��<+�,�> >u@ ����md?� �V�\[�����/0,0���V�R��p<��\�5��90F�K.��c�T��7s��k2�Z � 8�f��i��a�
{�>����1{��G���\����z�`%�����m��\1 �
����D���&` x�z�e�^s�Y��5�o��Fy�o���@P�VI��_ h�mV�t0���?�����:Q��V. ��V�����Q��0�� ����� *� P���5z��� @h@� <�;���:�e�0i�Y� ��l#��haR��7� ������J��q�C�oRt�
6��C�t{�!��I h?���\�n���K�)P�����I�\) `��|��.�C&�0�� �g� � �Cvy%���8�#3�������[ �8O8'+��a����#0,0��=��c�Y`8)-]O�a��������F��D`���Z\�Z�rk�1�����8�pXm�ZfM�4���D���t��D6_���))�F`X`8� X`�5��G ��_+��@��f=`�/���;�8!L �D ������I�1�
� lr��.v���C�&�Kd'���? r�h�\�a�W�&2��)���^R8�| �� 8�s:>`�����2?p:�~������W�j ���� ��|��y� �q"`��{"�y�f�I"�_[�(��2���Y?��CME�Wv �'��~��.�8�5�3���f+ ��M��s�=#�HN/��c��u�]����D3Y��~C�g����M������pNJs��q<��/�o�N���5{���LgH�+ f���� ��=f2+�� ��� \I����+@ @<0s�d�������� � ,���m� ��� C�l���r~�����x/0,0 � ���w" ��E�W�a�����<�%0�Y`X��S�!Go��.0�#��dV`8OB���a�� w���v� +�t�n��gW�9�(��I��_���'�p�]{������8i�$��dm���?�E�Y��` ��9�'0�hO��d�D 1b���.>��-W`�g�?� ;v��������!0 �1i�� �� ;����*���`��DC�Z��-��fyI~>��3m�f� i���37�s�0�i�`���z���W�Q���� ����~��|�i���{��N/ ���=�����e��ly���� �[n����L�1i���5�����t�MN�Z�L�����LD3�G��@f��$:���W)��J�U
�Q���T$��af-�����L�� PeVKRS�Z���]~�.����i�R�l��� ^��g(��� - f�����- 2 ��-�N�g��_���t����;���t�TX`X`x��?�S�!0,0�� 9z��t�� �'��y:��x�1�Dn=:o��x��f��N�n�(�������Y���l�{���CNti$+* 2i/o 1������#�6�^g�"��C����������0�O��%2XJt&��q��n?�� �aMV �G �>O�@86���_���e�b�1)��H����K=���l�f��0u"�#k����%��
l����$b�� �q�������q��e�t0L4�N���L�0�it�>c� ���D���is �k�����\#�Pf
��Q�|��.�u����k�������3g��| ���?���a�s� ��;���
�<��D������]|g� �\������� 5�:�Ds� AcDG�^6�yeu����W@`�+�W)�G��%
���`�f��C���u <��o��@�@
~��'�� N�-� �
��Pb�:���]
@R ��X+� ��f�uc����E<X0k� vpkD��U����k��v�w" ��e������pv=&�G ��Nu� ��c��p���Y��< ��b��s��jf^`��'0,0\������a �O<a�_��_������ $�m
������7O<�D�
_+��<t�P� $j�w������;87w�\������-�`���o?0B� ������+f��.��IM������7���I� F���L�0�d�����K� f x�+� �3�6����g��.�g�� ,%����gB�0��0�{���������\�y�v�����F"���}T6��e7���D��T���L���� � D���30���d�����@�
g���� ��A���`�C
���}�y,�����I���5l1������ KY�n�5k�����)f x�����0P3��c�e�r ���ml�<�<�A
X]QJ�����ARW����� ��y�{r������n�N7k�:���s����������>{�f�%�����w" W�U���m�N5����p�3��� ��s$p��
�I� #0,0��.V3��N?�a����H��� Y9a�;��S���l��f
^��o���a�lg�q�K7MI�s��g��L�����w>���G;�Gd�_��H��- `!� �g�!Z��� �I�~�A�C���X����NO��K0�3f����g`/���Y����H���!p�`*�l��� �/��2�M SU�!c&���
�NZf��(�4�D�3G��6�>D�q� ��w�Z�
�������9� ���
� Q�W^y�kc�>��z��F�WH�|�:���U@`�x�VWcH��������` (�2*�������g {��7��5f\1`�u��yL��b����r �} ��BZ����L,�U0p33�Ys �<0��AZi���/�����a���Y]�6�aE ��Kq�����qh���A`X`���'�g
��9�g���n����
��Y;
�
`5����Zld�a�2���)������n}��0���=O�6���_j�{�u�v��� [���#�[�:�lp��n ��6�(���A���3�&�;D`�i���e���*\���d����� ��G��)�F�(���� g����������kM�e� |�?�<�����6f� C'�TC�$pJ�1}� $�����v�S` >\��Q[>�p�r� ��K�0~[��<�@\����[7;����?�@ �� u�z�)�v.�^|Ga]\"����~�{E�,����S����l�0v��@����_7�L"�> ��k 5���������&���Ov ���D�E"����6�"�� ��;��5�������nD63��}�N������� �^�/~����u�1T�@<T0���Z�P�2�e��&M��Yf �K�h� `�1��"��q<{�E� ��[���={�{@4hP�t� 6H[��������+0,0��B1�z'��p�[S`X`8�=X`8�4��Nu� ��c��p���Y��< ��b��s��jf^`��'0,0\������a�b�$��@6���4|���������@�t8�?�T��L�m��mk�F9����v�5Dl�9 n��+0L�F� �(U�
�~Q� �H"`�{�9#1���0�/� M�K0�n=����f�@�v���c���?Cw"b��F 3�
�=��C�����G �I���~�[���-|����Fe�0�=S( g2�l���F�9��� �j?|�L"��N��7��H_
�F&1 v�a���o2~j��Q@`�:�� )PCp&�r�-nP'e�
���"��"���e�`%���s�6�\f1���#�vp<k3�`�����b�����+\��A������g������F[�}����81=�;��c�@VK`X`8������S� �:C���1]`8G����p���q1��9�b53/0�����T��L�0�@^[>��s�v��"i��� �h`�]�|�R?���� K�a��[���r��XQ|����?� +���B�O�3�XE~�7�x��H�����?��������%�
F�����G y�k��HC��_�(x�%=v�`= ��-�v��������w�y���I�5��^{����n���a�R �Lg� 9=0������N:�^x�E S����w\��3j#�6Q� uR����5�hO�7��u���WT~�m���
����R
0��P��?���@���^�uj 0 R:w���,�W��b� �
�%�r�q���@�~ �H5�`�1��r'N�h�Y� E��P����p��;��,0 ���~ ����{r|��pJ`�a��Tg��?� �H�<����9.F`X`8�]�f���~��5�����)&�6K��t�M6c� ���dA��Z�>]0%%LP�������,�G� 0�g�S>|���X�f���+0���=��c���w��� �&z�"0���d��ZI1�jm��}����C� �%h�w�}����;�<��[�w��Q�h�)�&������&Sf�
�:�y2g��>d��?�q� .�8v���a��������:����� ,�������u�u�J���$��fb���u62�Rg"��df�a�we�;�N�'�NWC��@ ``|2��p� �C� p2�:8����t�h�kL ����Tf��d�M `�,�9s��t���}�������h�"7Hc�s�GL��P�0�
8]� ����B�y x��g�� �s����w�Y��q�v����<����][�q����N�'%��y~� 7�j�
���}�Nd&�L�<���Zh�� 0��;��������f=~������K�l�+�J�v�}6�ew[���Y�n���b�y���A����.��ka|��5�C~�`�/�9H����JR���Z�5�W�X�5��.�B�+V0<b��R����g��8�p:��'���9�q�����H������)7�5ki���8Z�����?���w�]q� 6l��
��wE��x6�g�f�\�9�NmZ��#w
���v��U���]�k��J��|����x6�����~��'����5|� �] �V���/��<�����^��k��i��}����1�?s����w�1���zS'�y������HY�^u7|��#&{#)�Y/���Ik��4 2R_���x� ��������9ti� ��[.��� \�+)���f"���@�t8�����NI�6P`K;���7���,}}d�^��F��
������=��iK�C��)��M��F )�����}���a�m���{f�o�/��*���g�M��(���7�B���$a'Y����@�zY AQQ������.�"A� ��� kD�E �QY�H���'�J��os������gz��z��|�9s�������{���������
�����ZZc-�Wb~���M�G���O1j���-�x�~�7���N:)Z�Z��
Y ���#-* ���@�3��a/1�gVh�k�(u����GKD��������;�
�8u��� li�`�.�T�~j����.n��
I^�c�����N��.Z$��N�����"ki-=]�MbX/� |�Y��*+�|T�������(������v���Ea�uP��tq���������������l�>fm��Y��*�K>�����?h�7�|stQ����/�+V��V^�71|������w�n�)%�����7M���>m
������n&������o}k� ������lm.��;�u������i B7�i���bXlz����lYk��s�{4���;�����l���szN���oY����~�_e�<�����jc1���z���-�/ �o���b�������Y}R?�Q _������F���9Pb��7�j�{1\�T��(�� f=eg��MA �jzVe��q^����&�O���M�f�.��Q,Y��Z�O}>��;7��1k�������oU�D�c%�����6��L�3�0�P�[Z1�v(����]�jS;�D����>����l�I�J�JjI�x�����wK������Ea eM���s��Y ���?�_9�s�=g��@�jf�D��C�Z+Gjy�x�X�8�
+��&5�*i))+��'%>�������[����q� �g ���]�V�f� ��3����WBTiT����zF�n��}�g��bX��cU�b��-u�{��G�,^����;���5>�������Z&�������bV���|���!� ���u���/�h=�P4���E��k"����k�4��4�L�mZJ[���� LKHKH���.q�@ W��D���.t�������>�%tu��L\�c���a���:�ip�O����N
:A�d)Y�%P����wN5�R'?]����?���~�8��rs������V�y����G������M'9-;�;�4�*1|���VmV�:�*�b��R\��7��.P�Q �V��|�a��������Q��d{=��NS ����P������?� D���a���1��=_�� ������;l��Z���Y�����bx����_�F`n�P�g��[,����5nZ�Lw�ky3]�F���c&L�]���{�6h>+!k�yJ ��uM]Wbx�����+�g�i`���� u�k��>����
T6 ���:��Q?5X_/3��j������>��h��bu��f}�a��������@���1<��'��]?e'�<6�lk�BrYn���q9�&���6~��_Kvi P�[w� @�p�ZO+�U����f�J4J8j�4��`�X��k�iv�� ������6����F��J��[��I�k"�����s<UT����Z���c �5{!I��Z�X�"��c��w��j Y�}�9��j�b���I�Sc��
- ���]�aIl-#�e�%d�d���5aI3�u>��ZM��>�[�(�'
3�5�I����{�IX����(�&�x���S������D�[�H��c,�U���5�:�����V�6���X (~��e�%�����}[��~�����L���la�^V�mj�� ����&��r�������8�@ g�GZT@@'4�]dh�M';I_�u�� Q :I��%9� �Y��8�t�������,� =������t�hiSY�0���#JBX�6�r:Y*/�e( �;������.*$������W_�����T��g�8��[�/���tG����e1�=�xSy����(�� �g' �i;������J �.X�M oy�����0� W����71|�����Cv�>sK�a})����l�v���=�����? ����� jB@K�i�B��������u�N�
����jr�� ��j=��.���$z��� ��f�����z�� ��G���] 7o9�(��s�'�������l��_?/��I3��n'��#1�������p����
�Nj�d���$��_��+� 1����]w�e�<�HT��L%�4��|9�82}���W����V2��q[�������3s�4�j������j�W�VB/�z���f��i�Wc�Z�Y7lk\B�FhLV�h�X�W�5f OIH������[��o.������%7��&J�f�X�Jhk �T�-I��I������h�����������x�����a1��o�MK����$���%R�Eu,Nb�6���F��a}��x�X ��&2i6��3L��:��M3��zQ�KTk�.��Nji6�.n�_~y���z���E] �b��i�f � b�v �t��Y�z��NN:!��"��t���J:1�"BrXKNk����N2�������;vl���3����E�����]��-��;�t��%*$�u�����������N�:9)] �}����N�������QKU?����������q�����o���h����G�0b8����Jjb�}�� ��KA �&�/�0b�_kKYb8%���Mcg����q�bX�z-��G�h�X����P5����:������6��*��c� 4q>{������-�,i�%�5v��\�T�%���p������O>iOn���i�8~����,���?�`�����l5#V���L�Ea�����a5#WS�Rnj����.�,KVY���t����%��&+i�W�#����I]bV3k5�Z�h�KI}��k������q��E�1o,�U?z���j~����z�,d ����<4��4����%�U�����b��s����V�����/��V=����+��y���Fu��[�����ZMS���{��b�0���SO=�IT1$~f"�����(��f�j
R=+A:�.�0$��w���� X���A:9j����E��7��:��9�:I�ta�(��D���� T��������%���.0�.�TF�e)�����~����I���O����J�1 ���O�p��/� 1���B0?��TE� ���b�g�� �]��?b1����, 1�T}���~��+�%����&���lI�j���H�i�g��qR��#)�Y��n���(���j�Z���M����G��qeZ�R?�|��]u\��!M��X��&)�coi��$����qj�x�kK���h�p������n�r��Q�����%��^\����,`�V^�G��t�8jL[�i���#"�:��3�c1��-5�I����4s���&
���k���w�8����b��%1����Y���fB���Zqk�]�����h���m9�����Z:Z�%r^xa�~�'�@ g%Hz�I@VbVwiv�N���:�������A'y��%�u��f����;����NfZRC:yi9;���eBZ�t� �G?�Qtw�N�z��.F4cXy���g �~��M'�����_�J&+�43�uL:��7�iy��uW��hk�b�>Z b�>�1��%#����Z��C 1����R�� �� 1����R��N ��wC W�~���� ]�����f���X�J��_���n�(T���*G�X��S���? O��b�3mY��OqjlX�$�5� ���j���<^�!��k����������I0�$��9��h�= _W *O�I�krS����X �Q����� ��+V�+��zj�� @ IDATPy����x�O�8�
qy���v a-���U-g~�)��%�\��r%s��� 1�pL���D@'@��tW�NZ�BKn�t���E������G��g���(���<���S���1c"a���-m:�J�j�i-G�����oGXw�)/����6�,��;��lZ�K';���t�6����4J�a�R>������ -���$�����u����z����N9�����z�����z�?�Qo��}�%�g�V=h
�.��LP�=cx����}w������b��et���m��1<x��.�����$�L _�����@|_��t~�g����x7 L���z�%���.]�U������1���B��Z���u�W==c���1|���r}Z���7
��3D��������oc��-Y��V=�}� ����:5��]�?j0]������e�L�1\ T��^��x�N�}+;q�q���Z���\���O�g�M�e�>k���5��sa��U=T���4��������|���1����S�"��//b8`u H)1\��VJ ��:��hb���� IgM�$.=�%P����9"�v�l!P
���;������"Oy�u�Y�}�a���s ��b�!���}.z���hi�d����~��h�����p���@���[���I������Hji��V��K����{n�NN�M���5��>7A_�t1����Q � 2�N;���-�����o�)C��@=����?h�� ^xfM�o�<�g����71|�����w=zttWm���MJ?�� �����1\ ����b��s��s��V� ��u��>[�>�K�n����*�����������C�%���N,�uM���{G7"����4�A�N�Pu%��������x�����=
��=?���u'���[dg �o��R����~.��^��� �S�k��9��}�
�-�6��Y!� ��k7�%�6�?�����7�Y4 � ?u�M�l��� �����1������=����xB4�R�H���U�I�'��)�t=��k�2C\t3�o~��h)[�p�2*����]=��1��;��W��Oj�]c0���C��;��z�F��F�5O�HK@�z�� 8/I���5CWBW�XZ�B����f����f(k�h����`-!�;�$��w�����GKi(�R�+� ��7�tS���QG eg�}vK�V��X �p� � ��2�y"]h�C"� ��Y� ���:�U�a}9��z�Z�F����}����z��m�kc d'�xb�;���d��[l��C�!5�fb�����j����>[����c,�[����)S�8��>�pn�bX���>��t=�a-Y�n���+���Q �h���<|h��:�O�����g����_��#��6�1\�S��(�� %����>7��V!���'�o�6�
{Z �5��{��'z�)b8n��D W�� S���c ��I���Y�:&�����o�R�z��e�]�����f=��Z b��5@����/5�B_���9�����P�~O>��H ���Z��O�S����r��~��k��J����+��_�:z��n��f_��W�gkpKy��1#�'�Ke�����g�y&�k�������K��t2|��G 1�&��W1���mbX���*�������'�E �Q u)�� ��.�������o.�.��=���t�NzY��H ��������bbX��h�bX�<���6`�����,m��'��z�&LhUbX�g�m)�X kFI���%T�����93���b8�+�&1�]A �`�
0���$������a]��V1��?=c����F W��!���N��1,) ����1�G@�Fh�������������y��j@ W�$�@ ] �^3|%�5 �g?��>Js�]wE3�%��� ������SO=ez������~�a����?���x��h) =�@�Y�����'�������{Z���M��T��K�J
��-�Z����}��l�pE�(�D��`�"S �1�F gjD�#����3�����FQ�_�s:b��`=g��W��F {_����,%��r��Vv���������ju+���&���UbXX�������r��)����s�9����m�]�+�#� b8�� �z'�����?F394 1��\��>8�����:�_��la������_�j4CW�iy������ ��?,����|&���R^/����{���]��Er��GF�$���������k�� ���;/�������: ���>��c���t����F W�-��O<�\�KI3cx��>G���*�a�����a���_(��M5������1��,��K|NG W������;)1�v����)b8"�f�p��T�b8&�O�1��B=���o4��kU,��)_��{=N�g���(��!PM��j�$/l��f������D���������{�E�\�j�B���N����3gN��]������I�j��fk�J�Jh�����'���3
����� 6l�}����N4Z�ZZyM�<9:������8n����|+����?�Ow�k�� ��M ~�������%;����� TV��1l�
J,%]����-�py�0b��18�%>�#� ��-b�h�� ����X���?�0b8[G�����f����qi=�Pc��>����'�h;���u���*h4 ����%Z�SjL^c���u+�h9���/�h ~�a��I-!��.���;�lz�ps_PN�� �R���� x����Y�����z���������.�5X�g ���)S��[�e��t��%$ ����7E���w�}��$q���H��n"����g���\Id�, 9rd$�5����E�h ��t�SYZrZ'VI���G��5KY�(/�`����-��h� ��e4��w����WU� ����
��o"���#��M���/�9 1���l��@;.1�v���e���!���:�?�F �3�@���J����Z����n���+���o_�m������Z�y����>�����:�,���Ob��>��$��L�f�n�����~�E�����if��A,��r��hV��l��m.�uHo��F��a=�Viw�a�H���$�UL�K�$�>��C����j?� 1�����K"#�����B #��j0>wF 7�F #����_�s:b�`O�"�=�v\ b1���e� 1 �C #��u�N��g&��@%��P#
2� � D�\H�j���$Z� �f��yZ�Y��?��OE��5+W������BW�V���'����f���+z���R zN��I��1"Z�By�Mejv�f
�����/���{���r�z_yIdkl������jn�a�p5�S��������l�#���Z�����&��a�pScp�K|NG ;�)[��'���A #� 7�l�#�#~�a�b��/�������9�O��p�H �u��{���z��v��� -�A � b8��!�:&�g�j���Q��k���$�% 4�X�$��L a= Ak.�cDJ'�����OW��aC���f!k�i�#��g���3����!�l��hik����zF���V^{��g���;������m%������<�V��������� ��* 1�nj �~����aG�=e���q1�a���&�-{�p�1�W ���;��U�X�Czfk��^�x��~k�
lhc����
� b8��!�:& i+��b����������,]��?I�bB�9 ���*�1k�,����� 8p`�Lu� ="���\��R>�?�XKPK�����Y��E�|u��bC #�]��Z�"#�kU�)1��NKr� b� *b1�� ����� {�1� ��b��a�M,[���� b�����_� �^0k������M� ��5��� ����� b8��!��B@d����g k�g=�W�tK����$.�Y"���a-�����t�Mq)&�k �x��f1+&�bWb1��m�"�x1\ ��+1��^k�rN��&��a�pScp�K|NG ;�)[��'���A #� 7�l�#�#~�a?bX�'z��&v,\�0[��Aj�OjM�����s2!B � b8�*! @ 1���^"#��]��a�p�-1�T5�a�pScp�K|NG ;�)[��'���A #� 7�l�#�#~�a?b8[c%5 �F���X� 3'�F �D�
/ DF ��-������e b8&=�B���#8\�l�T`����� �����l���6x���Qj���'F�� 2$�W�'i�[|NG �������8z�0b8n A�D G��F �?
� 1L#� �#�F �(3"#�3@ )b1 @3l9�pf 3c��18�%>�#� ��-b�h�� ����X���?�0b8[G"5 �"�vE�|! �� ��7� ����� +������2���]�M������������v�S��aO� �F ;nb��G G����l ��� �� b�Y�� *&�F W�xL"#���2BB #��h.~wE 7�F #����_�s:b�`O�"�=�v\ b1���e� 1 �C #��u$RC ��+�aWd��� b1\q� 0a<���r� 1�.����1��1�nj �~����aG�=e���q1�a���&�-{�p�1����H
@ � �]�%_@�b�a�p��'��� 2b8��)#$�0b����wW�po�0b��18�%>�#� ��-b�h�� ����X���?�0b8[G"5 �"�vE�|! �� ��7� ����� +������2���]�M������������v�S��aO� �F ;nb��/C �-��?�L��<n�\]�v��X��)���ic :t�.]��~����}{�b�-l���+;�u��eC# x!����B �r ������������T������-���"���#��M���/�9 1���l��@;.1�v���e_�����l��)���O��9s��[��6l���~�_�I�_�v��?��g��=�����o~��e��. @���'��'�F �o-��"#����R"����GM�C 7�G #����_�s:b�`O�"�=�v\ b1���e�� 1��[o��7�h��y��/��i�N���Q��Bx�r9[�0�y����z�j;q���k����6�1�ID� @ 5"��x�� �@ #������"#��Ww�#F #�����~G 7Ub1�� ����� {�1� ��b��a�M,[�e����g�W\a�VM���fC6�V6��B`���Y�����������h��zk��"��FF@ �D 1� 4�@ � ���[K�{��������T��a�p��Q���M������������v�S��aO� �F ;nb���@ ��8����z���lRW������e ����k����
U2� �P �C� �� � ��M��~�����L�0b8��n��0b��18�%>�#� ��-b�h�� ����X������1 @% 8#�v���! �J �����������k'}L�a�p���yO�pp�0b��18�%>�#� ��-b�h�� ����X������1 @% 8#�v���! �J �����������k'}L�a�p���yO�pp�0b��18�%>�#� ��-b�h�� ����X������1 @% 8#�v���! �J �����������k'}L�a�p���yO�pp�0b��18�%>�#� ��-b�h�� ����X������1 @% 8#�v���! �J �����������k'}L�a�p���y�
������]o���s��)n��e�z��Dfm����]�Z� �]�v����B�Knw����p_x�{���m��6t�P<xp�y� ��[g'N�+V��!C�|{�����~��t�p�� 1�����G #����O�p��RNP��rh�/ �� b8o5F�h���zj�� 2b8���F �������|o���������� �&[�r�I
4�$V;u�d�������l��f���b�-�s���/����T�*�)>�#�+FDB�p��9�0b8s#r�b�%]/y#��`�@ ��p��S, P� b1\�u���x1���k 1b1��=�{bX2d���v�u�����?�-�� �Eg:K�y�a� � _�� w�q���;�}(������ >�#������������������D�a_����v���! @ �� *� �$�0b8�"��* DF �� ���`[pbX��?��C���+l�o�[�����fm�=�����+����v����I�;�
T6�p���J���ea ng�ppURQ@�a�pE
�W"��/���A ;CK�� �p �@�@� b1�l�~"#��]��a�p�-�L1���+���&=5�������W�`����d�m>ss;a� v�I�V�|`���V���n9�� 1�������a?-��R�� 'b8�� @ �>�p���# �� b1�� �< DF U-e�F ��h|%�T �i�
�� ��%b�WUm���?4X��z� �!�7��V���t�p�j�:�"������ ���n�%�G ����7�y�%b� �J �+%G:@��0b�Y��A�� 2b���X$b1\��T�����Y���5�������N ,����VL�a!��e6��#���vPb�T�� �` ���
� b�a�p���g<���wm"����`�p�USn`��r���?>�#����f���j��]^�a�p�Z_���) �� b8��!:@ �F 1�� �! ��a��fY������UAU
F #����\d�vA�&y"�k���B�s:b�,l����J*
1�����J��E�Y9�agh�� ��H@ #��-"���Ad�p��1��#����rC �K����9 1��}5KD W�f��B #�k��R��N)�]�a��A �@6��l�H
8 �F ;hV5�2 DF ��
�R0b1\���"�� �5�1\�e���ea ng�ppURQ@�a�pE
�W"��/���A ;CK�� �p �@�@� b1�l�~"#��]��a�p�-1 l��b�\b������a���Y"b��4k�b1\����d�p
Ha���~�� �@ g�Gj@��0b�A��Y�� 2b�fUP������4$� �]P�I����`/�����. [p;#�����B #�+j8�!�}�vVb�Z2� � ��B� ����d����x1��zD #��m���`��������|NG �g_����Y��������%#�S@
{�p��Ct� �� b8?RC ���U�����5����F W�!��1��jM�D �{Y���t�pY���1 \�Tb1\Q���1����r����1 @ 1 @% �$�F '[D�_�����|�#b1 l F [5��.�����s:b�?�j���&����F ����(1�R�� ��� �� �l����p@ 1�v��j�e<���YT�`�0b�*
�E&�aTk�'b�&��*4>�#������������������D�a_����v���! @ �� *� �$�0b8�"��* DF �� ���`[0b8��)70�p���� �����W�D���� l��96�|[�zu5�z�K��v���������e l�Wl�����t�GU Z�`�f�d�&��]|��v����^����o���o��}����z�-�?ox&�� ���!���� ! @ ��p��H �F�a�p=u�x1��ZE #��m���`��������|NG �g_��fS�N5����?oKY�6m���K^lC�K�����.\h�w����Gf��l�ms/1T��p���,C�p��W�`�p�H� �H 1 b�Z9�0b���@<���w�"����`�p�USn`��r���?>�#����f��a�� �~~�O���o��m>�~];U���6���g�g����V�u����f=��C�
C W
e�3B ��
���J��� 2�p��Clh����zj�� 2b8���F ����VM��!��%���������%"�� }�Q���+m�� ��]��.=7�&b�y]��7���;��3.4�q/�����P��U�D&�� �!K��,�H @ �@ �^C��VH 1���f "#��]��a�p�-1 l��b�\b������a���Y"b�1|��W�A����n l�9}&oc����YvG��� 1\��B^Y ��� "-b8�j @ pD 1�,�B �@ #�+o=�������M9!������"���vYb�%��� �����Y�\���Z��M����M"�����TU�@����w@ ���z�E�uF 1���& "#��]��a�p�-1 l��b�\b������a���Y"b1\��T���U�Y���5�_�����H � *�p�5C\h����zj�� 2b8���F ����VM��!��%���������%"���lOU� 1\U�5� 1\S��( 1\
��@ �@ �Z3��VL 1���� "#��]��a�p�-1 l��b�\b������a���Y"b1\��T���U�Y���5�_�����H � *�p�5C\h����zj�� rk�k����K���?lo���m��!��+1�z�j� x���/K��������ry<Q�k���=���6��A6z�h���w��y������7��~Cm� ��v������� b�#l�E!����F��9 1\
���1��]��D���M ����� UV��"�[��_! @�> ��� 9
��0b��t<��Z�������>��/��f��f�u������� ^�l���|����f�v�Ef�������F ����G�nN#��#���������JE�F �j 5}1\S�U- 1\U��� 1\ �� @ � �}�� @ 5�0b8uc���� rk�
6k�,������o����i� �Z�g.Ya�.Zi�[��V�}���c�� �;���k(]|��t�r�b8�J��������T��a�p��Q���5�_���U�Y�����N�� �� b�i�� R@ #�S7� �"�v1|���^o�f��s`j�x�/-\j�|���Z���} 1\��x#����uA�a���� ����Y�2�0b����d����xr�&b8W��R������7@ ���z�I�uD 1����l� rk�W^�}���v�>;��z��_b|��&�_gK�x)3�s]�u<b�N*� 1 ~U��t�p�uU*B�0b�T���{�����j������Ef��ZP�L@ �E 1��4�@ � ���Kv�������+KIW%��[���!�J2����Z���
,���W�N���+ ����U?�-�i���#�N���Yj��<�l��� E a�[kv��6��"����v�q�Y�^�Z���?=���v��W�A����n l������7�3m���e�-;�B� �2��;a�s���f��� ��ZsMq#��P� �!�p V*���@ #������ "#����E�#�����A �3# %�p(5Q<����Z����+m��yv��7�K/� ����}Grx��u�p�B�h�r{�S/[w�����n��� 1���D�^}7�Rx���/���A ;CK�� �p �@�@� b1 �����o���/�;��c�����G�[�x��s�=���?c ���f��1k�>��d6�}����������I�l��E�����f��e,%]Q��@ ��}5KF W�fM�B ���[�^�d���=����R{�//��^��e�������Y+������5g^`v�I��A�;!������&b8��� �p�H@ �!��MU(Z�0b8n���z�������/��Nm�X����m��^��z����� s�m��z�� �v8�����{ 7�e�|��z�1\75��*[�� [�����.����j���]��V"��.\j��[n��h����l��%��-�0b1 V'E �U D�� I @ 7���*�@�!�F �������'���v]��
���zw����� W��K_���~�)��s�2�nW�9= �0b1 �gb8��� b8#@�����/���zg�}g�� ��+��y�����]K����!���v�3�~� 1����U b�,\!���V� � �E 1\-��T� b1 7���������v���6t���u�N�[��9w��������?��|�|��wC �����_b|�1���"������ �3�� 1�� _�} b�{C
���x{� ���`����-bx��. �&�F �Q��J�
1\.1�� �<@ ������ b1 7u����+����������h�;�O/������+8�?�7��~Cmb8����.2F���Cr�0b�C3����f)if W�w\�B �"�-_��7�@ 5 ��t�� J@ #����I�����1��S"������ �3�� 1����*+1�F W�w\�B �"�-_��7�@ 5 ��t�� J@ #����I�1�F ��)���E�H�zH�F {hf��F #�+�;�R!�]���/b�j
� �@ � :EB � ��q A �$��F #������p�"c$��� =$G #�=4���@ #���� W����z�1�
5A �@
�k �"! ������� �c��D #���tJ�p8u�1�pF� �#�� �YeE ��������T�aWd������� @��5�N��@i�a�p�B�1��~"����p:%b8��� b8#@���a���"��a�pe}�U*��+���E {CMA� P��@�H@�4�0b8n!���D`?��a�p8�1 N]d�1�����a���fVY�a�0b����*b�Yo�"���� @ ��p
�S$ P� b1 ��pL"���a�0b8�N��.2F���Cr�0b�C3���0b1\Y�q�
1����|��PS �� b��)(M 1��[b8&�O�0b1 N�D �S#A g�!9b1���UVb1�����J�vE�[��ao�)� j@ 1\� �&�F �-1 ��'b1��S"������ �3�� 1����*+1�F W�w\�B �"�-_��7�@ 5 ��t�� J@ #����I�1�F ��)���E�H�zH�F {hf��F #�+�;�R!�]���/b�j
� �@ � :EB � ��q A �$��F #������p�"c$��� =$G #�=4���@ #���� W����z�1�
5A �@
�k �"! ������� �c��D #���tJ�p8u�1�z��� ��'��7�|���_��Vm�����+W����{��8�6�� ��}����MPYK��-�:������l�� 6|���9.\�.
��o ����Z��d���$� �B ���� R@ �F����/�a�L��Yc��-� x�~3�U������s�� ��*�_��3�&_jg�3�N?�;���J��.��vo�)�g��iW]�?���7��}v�]�U(y��%���[����~g�&�bS�
�E��*��*eQ�^�v����{v�uW��?M�?` ��P�@�&+��b��=���s���|����� ����Cm���e��q�u�����m��6d��(� =zl��&_�p�
����l��3�G����]��&�������yK������1��l�� b�}>�1lw,nkK�^d6��|����H {�����[=�G y�~>�*;h�|���
��[�}���w��jubX�u����n|�3�pXM�I ok�}�
4�h|�g��+���ft�a�v]d=��,�/o�#����� @�?����l�@,�G�e����������
���� ;��tb��{m��5�}z�6];���?Z������6}������o��nf :��x�p��.�I�&��#� GCC���5�������;3�EI����k�� g�m�kg ��� =�z��]4�� { 1\�N
�h&�/��������������&=�.
�FoDb�� 6j�QV��6m�M�2� Xu1�|��Lbx��q���w���V�� �U��� �c��5��g^�� r������zok�>�7Y4����O7c�G�E���
��F��:�>?o�������g ��mJ ?������4��������~��M�O���g����3�tN���qa��E(���1������=%�O����O~��au�T��bX�{��ts���s��{��7�x�.����������! @ �@ WJ�t��3�cV�]F� i�o���rB�XbX_"�,�z�1|�^���#��G Q�4�h������#�K���f$� ��w]j7�t� ~��EX�xq$�����Z�wg!������b��� ����N8����� �q��M���m������v��AS�?h&�/>�����:�h�����m������Y�?H w����7�N����� {� ����M�g� v��^�t����^��~�Y�������k�������q��S�}���f��1<������/1|����a� V�n-Zd�����t����{� 1\��7g �n�}��}�����^�z�C��L��;x�B���EA�~� ��J �������y��W4��9s��Oo��&N}��EI���H ?eg����e������
D�������:�d
�50c�()�o4�#� }�-7=~Bb��{�5�s!��%��� �� b�Y�� *&�/F�� |�����g���>s����oR k�r��w��u���;����f �|��%g�bx�E���a�Qy��b�P���w�O<�����_��(,%�� �������6����o-$�W_�M~v2b�(���� ��'b��e�%����:�z�E,�+YJ��?�����_��N�m�^9��U@���_mF��a��W�f/=cXbx��G7{!�k�Y�b��bx���a��1� �I1<��q6l��p�@k��F1�}���_9����o
D��������^E %U�7g � ��>s��M��+���������UWK�n,�o����v+�7]{��7�~���D7K#�K��M@ �H 1�6EA ��KIk�Zk_JZ�s�Q ��_8�P;��#K6
�1\O��,���R�|�g ���Z.9^J�����ZJ���7����i������e�}��R��z��wy�p�j�d���1|�I�V�|���1����+,%=�0c1\�)�}��R�&L(y�����/�l�/��pN���a�5U����1|�q���ZY�����������R��Z$��6����j�%6=cx�TKI���M��Cs1<j�Qv���� ���5c����f)���� � \@ ��K��@E�f�3�������E<c����.b�&�_gK�x���c�qv�3b�5a?�#��p�P
b��*1�Ng #�O�R �A6�a���@[v��f;�e��w
*�
E"�� 1�,��TD�a �+gGJ@ � b8�:"B�:�a�p���1 ��'b1��p�1 J�D �R��@ gF�-�03��5��!����h��o��~� �]v�?b�9b
� �@ �>EC -@ #�����I�1�F ��)���E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I�a�p�"�1��~"����p:%b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1��[b8&�O�0b1 N�D �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��q�@ �$��F #������p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�0b8n���D`?��a�p8�1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�F �-1 ��'b1��S"������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� �����E �c��D #���tJ�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@�@5�����m����a��d�9y����Yc��-�� z�����=������k��99��|w���K��}��>��<��v��?!���EP�!����p�$b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I �����m��%��k����+���������������7?���onv��g��N99��D ����jk>{�����u��� � 1�F ��Q���E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ IY���� �����]w���Kl��������`���n�:�������k����3���f������F ��c����?O��� �`1b1 N�E �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ d�o���=��?�O.���, ����K����zk�J�c�� }
3����l��sv��E #���:D�]����qv���� ���G[������c����o�����"�pQN��@ {G��@��+���1��~���#b1������ 1\e���C �gN�� �� b� kJ� R���� �h7��2;�G;;q�>)Ks�G��o5���:�-����aV�&��h�;�O/#���l+�v@ {C�� ��S�>3G ����,�0b8[ r�1�F ;�X�D g�FR�p�@� �� b�
Wr� 2����O~h�l��N X�9��{��=�Mi�b�k���$b�����7�B�p8m1 N]d�1��^Pi�AUG�`����
�o"����Z���e"�����;���TaB �@E�a# �� b8I1�+�������A 'G�^��:?������6i�$1bD�Hl��Yv�����j�b�V��[.b��<k�b����,1�.���� 1�F ��g���.�V��"����� �*@ W $�@ �#�N�D #��-"�W�a�4�-���fG� �B* 1\!���!�����A W��wJ�0b�w��dy�a�0bx�����a��]�vA�<! @ ��Pj�8 �&��&�/�a�p�E�
1��c��.�� ��u����J�0b8���F #����������h�@# �� b87UE�h=���F #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1���F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�p�b1�l�B #��t�����E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I��$�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� ��I �a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��<���d����> @ IDATb1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ��H@ 'y ����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�N�@ #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1���F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�p�b1�l�B #��t�����E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I��$�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� ��I �a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��<���d��b1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ��H@ 'y ����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�N�@ #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1���F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�p�b1�l�B #��t�����E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I��$�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� ��I �a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��<���d��b1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ��H@ 'y ����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�N�@ #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1���F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�p�b1�l�B #��t�����E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I��$�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� ��I �a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��<���d��b1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ��H@ 'y ����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�N�@ #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1���F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�p�b1�l�B #��t�����E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I��$�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� ��I �a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��<���d��b1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ��H@ 'y ����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�N�@ #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J @ I 1���F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�@� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *� �$�p�b1�l�B #��t�����E�H�zL�F {ln��B #���������H;+1� -C �@ �T!@ I��$�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B� ��I �a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � $ ��<���d��b1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ��H@ 'y ����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @% �$�N�@ #��-"�W�a�0b8���b8��� b8#@����a��-]Q�a�0b8]_��b�ig� ���%c@ � b8�J �D`��
�n�:{��w��?� ��U��m����K����m��v��wo�������I��a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � �PH
/[�,�/������k���o����#1��G�H
2�
���}���j b8I1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *�
���W��/�l��{������t��h���������]�x�=��SN9%�A��s�� b8�1�N�� ^!���� :bc��p�"c$��� =&G #�=6�tE!����t}��^�a_����v���! @ �� *�
I���{��x� ��a-=x��H��]�6�I�}-Zd={�������8��~���z��$N�0b8�"x�F #����! ������ �3�� 1������F #���_{!�}�vVb�Z2� � ��B�@� ��~���������z�v�i';������-���$��{�={��gm��6w�\�|����3��}��'z�p��1�$�F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�5I_-=a��h�p��mm�������}�Z�6m������{���LKO�z���<����1�D�F '[D ���a�p �1�p8u�1�pF� �#�� �[����a�p���k/��/���A ;CK�� �p �@�5�%K�����n��w���5+�!��/9z�p�N����gO�2%Zn��7��C9�;��hvq��A '"����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @%jM`��y� `�nhh��w����c;��c��M�:��|�I{��gl�]v�#F��Q�Z���?"������d��b1��#6���.2F���cr�0b�csKWb1�N�W|���E�Y9�agh�� ���&�g�������}��6l�0;��#�_�~-�6{��H
�s�=��O}��<��H$��sD '�!����+�0b1 @Gl 1 N]d�1�����a������(�0b1����� 1����r����1 @ 1 @%jM�����_�����+�X��=���������z���bh }�Q�,�[n���w�nG u��u�Y-�[� �Ij�a�p�E�
1�F �C@ �S#A g�19b1����+
1�F ��+��B �"�� ��3�d @ @ P � �Z�3g�M�<�4x�����|�36h� �b�-Z m���������z���c$�����Z�6mZ���?*�� ~�F� i�o�}���������z�n��'vJ�v���e�R�z�g����9�����X�p�
6��������}z�6]���������G+���z���W[��������C�C�tL1��������M7�d��/�F���O_����~��� �iy�������K����>d��v��w� P�� U���{���[��m������v�/��i��4��s��~�����v�Z{���m�
����n�?` ��Pdo�����p�W���~#����l{��G�!L�>� y���q C�������� ��[g'N��K��^{�e�������c��6���k������K�5�o�n��K~���(����������/Z� .�W_}�n��G���Y��!;�7o<?o�����cq[[:�"���!��cl���]`_<t ;��c�W�^-�[���~�;�p�5v���6�{�E�������L�U7b��������� �>��;����uN��7�k����n�.��&o��3��3��g�9�x�}���������*{��K��[�� �������@s1<r�����rK�5� �v���F��/����0����! @ Y ���#-����������w�������'� =g�����+�����q��E������uk��]Q9�a�[�~���xS����������>���v�M��O�k��z���v�����}n�V��O����;�/ V�o������v�+O�'c-�A��Gv�������R��J��� �������{��9�sW���=�������f�����5�bxAA ���m�� ���s;��C������%K�/��_~��{������=�<c��RD��`�-;��� �)��6g��]cv��6���l�~�D7��D~��'��I���m�Z���8�y�� p���r����_��u������v���=P��?���n�uv��wY� �[�^����~ ,~rq$���w�}v�gK"�L�y���m��v�O��U��: ��\72��u+_"�f� ����i��S�P[u���Q��
��/>���y��m�I�f�{��9����X� ��^]��:��f��^{�&��2:���^�5�\�}ZA ?��������f�����[ Rb��?�C b��a{G��)v����s������,��
bx�������� ����vW�����~�]���5�$c������n7}��{�)v�9��P�t��}��w�/��a��?7�"��{� l �x�n0�;}��v��cm�] ���l�G�<��N���.� ��U���s�������~�� ������r��?�����o>�`�_/��b�p�� � �@ ;�J����3g�5�\cs�������� �b�~ |����?��I�j��f w��%��- �.�5�d��-�����y����}���K����������p�����1���$v��B����/������
�;$�� ���6f�����R��� ����Zh{�?��A�������m�>�� y�Y��`j^��Y��=i[>:�~������ Z�Ej�����Y���s����f^�7^Y�������_���k���� �*�A��&�A �����c�=�����@����'�f�7��v(��?�J��M&�VX�w��N������3K�a��%���p�������7��v���T<��G`��K��?
��x�}f�g*�/��R$|v�y���a}�k���{� =J��#��DS���n����j�;�W /,���������Q���/��*��v9����
H�W�d[=y�]}��%o�Z�h�����vsA �|{�}e�|����p���pN����- ]����q�6�p���p���7�!]V����8����?)�H ��z�l��������k8�RA$W����i[yb��(}W��y��d�Rk��������;_:-��],*���R��o��&=�W�/nX��/v�y�� �����jc� ��+-�uc�u�]g/�/����6�g�< y�� ����� �����V�(��t����)S���.� �C ��+�aWd�9"�����'��
��y����6��f����D3%��� FbXR��_�b�|� Z�-�Y��a��?�q�d�%w���6z�h0`�&��x�?���O{�n,���W-��7���.�z���e6g�Z{�Mg���0k����p� Xa����l�?v����GE��.�/�K{��l�?f[����k��
��/�O���5�h�w���lVX"�m�R� �{�WZ������w��K.��C�`�/_n���� ]n ����/U8o�*���U6{ea v���n� /��a�����A;lc' {D$����b�n��������o�����������A����n�b��������%;��3� �n��
Zo�hw>x�u����������������X�v}m�a��������.7��?����~��1,1��n�U<c���o�� ���~��m���u)�4��me�����>�%}w��� 1�\�jmrzN_���-_`���g\p�i �b�n���������W�b{�|�������+��5��#���C�C��:���� �ss�qv��\���g���o�d���i��]e�wl�;\��I�S��gs�nf�v jmt}��S^BO��n��_��6�p�}�Kc���NK���������<�6�c��l�n�u����f���_ 7D�������~x�q
��v������S����[�>�������#(���]o�g���8.��[���3� z�!��?�a��r �? �"�vE�|!�#��a]�J _~�����s��o����C��_�;%�����,��,s�R��f�����;������m��5&�:���;\bN �U��#��,�Y�k�������"�
y���A����z��3�k�DmQ�� 3fL�di�8V�^m� w�uW�\���G�E�����:u��������ZV�����.��k�C�\}����z\�lY^�U��G9h��Q�F�\�A�C�Q��G �f�� ��E`�����;,z���>���� Y ?����y]b� ��� �u���y?���_mO���q��7<�Z�*��D����3��t�5�S{���c]���\q]g� ��S�M+F��]���s��mj�j�y��:��{���#K���v���z^��2�����j����o�h�������S�Pt����5^��]��"��R�l����n����a�p��� � \@ �&L��
���6��X������L��� 3�Z�$z���+��2�����|�+%����X���e�OZJ��G ���:*���x3� ��/��y ���n| y ��1I����$w�;����e=���B����M���c,S�������T�In�g���g��$21���~��T��l-��sH�Qu� ���Pb`S��-E ����R���7��M�������� _���P� ��O}v���������q]�:����y|:�������� �/K�M����R���y���>Wu�,w�g-b�\j�@ > �}P� N@���8qb�����{�I'E3��-)������d��� y���3
7��'-���gGwT�y��n�m�du�_,�6>0q�����u� ����r�T
�� +��~���)�����5q=���������}?��>��D�]������ .��nR��Q�����3f�c�=���R�ZFx�=��}����������D~pN��Z�R��������Q����\���� / ���������������y��2}���M�
���� x�4���.�V�j�,8f@ � b8�� ������<yr����-��}� Y�^-??r����������;���<��� ��
�n�X@E����F��������_����4�&,hl�!X 0"*�DP��7�9��������{���u���������g=�Z3��I;{_��M-�����?��>k�=�� &����� �g-�����6��(2�������yE^���.&��m�D{4��d"{�b�.*���_-��������� ��'����%���c��}�����sN{������}V�xnS ������C9������j�a;e�]w,�������O���-!��_.z���v��������c����W����c����c{_� ��+�� ��
y/\u�U�{���6�bxC���A � 0KB ��� ��w�����SNi.���f��vj��}� 0�98n9���oy�[����}�s��[;��J���|���{N��!U$�!�hw�h1������#j�E���p��n��&��"j��w[�$8<�M��[���g' �Z��. ��� �G�$�������1.z�V[����rd��W��j,g�}v��}��k�#A�r�$�=����k�/���1}eH]�^��7����Y�51�+�fL ���4�=�e�����J����,�{~�-'����� ������8�>��S�o|��1����V� V%!�W���� �6���~���s����y�c���+_����3�<���?� s�1�4'�pB���]��[�+ bx��g�NJy�D�Q
1�Q�-�I!�N$��������eZ�fL����'��|����C O ���bx�2����G0�p?.�6� ��@���� -sG��k�m�����D���_����x����m�g���~����>��l�:����#��=vc� 1 bxc�fQ�IyQ%�a�
1�ax-��!�U2��������eZ�fL����'��|����C O ���bx�2����G0�p?.�6� ��@���� -sG���s����77_����[6���/m=������@���T?���Zb�{��^�S��}h��>��a��b8��&+�U�$� c�bx�[�SC /�06�)!�� 1�������>-$�[O����?�������'���e��-1��`��~\�mA � ����AZ�� �W�������E�u�]� w�q�C ����Xj�
7��\v�e�������s�m���@�k���UB ���.-B=I"/�6�
!�7 �E�!��"Ha:m1��c��~\��m��i!9�zB ��i]=�����=!��/�MmA��~C ���o�@A`���� �� �0|�3�i�&� _�B{;�C9�9����]v���)|��W6��~s�y�5�_}{�i������7�|j�1 bxj�� %�� B�BB O��"��aJM1�d��~\��m��i!9�zB ��i]=�����=!��/�MmA��~C ���o�@A`���� �� �0|���o���|��k!|�{����]��H�����.j6�l�����gs�1�4 |ps�;�q�}1 bx�
5���D�� �t��Sr����� �x���`���eZ�fL����'��|����C O ���bx�2����G0�p?.�6� ��@���� - ��`��+�h�3��O|�%���[l�E{�h� l={�m��:��f�����na`�1�0F1��$�<��� �)4!��@\�*B � "�p?.��6c����o=!������ bxZH��������� �p?�!��q��A � 0B �_iAX(<G��?�qK�~�{�k����v��[E������t��6����=��w���=<��1<m��g}I"���];�����gM!����t� b� ����L�����Br����/���z��i!9�zB �_��������%�� ���1<�A`!���[�����j���� w��%���)<�b8���tk �&�<������t 5�����f��~\C ��2�o3�O ���bx��O��!������ 1<lj B �#b� �|�@�G ���e�� 0�@���#*1��I"Z|K�1����� ���j|����X�bx �����>X������L.bx&�����s�}�
1�g��~\�mA � ����AZ��!�C ����?&�<h�-5>���~bx��[��!���c�C��%(f�&c�L`]�JC /;�3�`����:�JC ���^4�p?�!��q��A � 0B �_iA#�1<����$�������KP �M��A�o���^ ��!��������3�u�+
1������!�g�\*
1<��z���p���%�� ���1<�A � b8���J �c����R�C /A1�7!�-�� bx-8�>�^�b&o2���e�4���C>� �� �s�4��\`��EC ��b� �|�@�G ���e�� 0�@���#*1��I"Z|K�1����� ���j|����X�bx �����>X������L.bx&�����s�}�
1�g��~\�mA � ����AZ��!�C ����?&�<h�-5>���~bx��[��!���c�C��%(f�&c�L`]�JC /;�3�`����:�JC ���^4�p?�!��q��A � 0B �_iA#�1<����$�������KP �M��A�o���^ ��!��������3�u�+
1������!�g�\*
1<��z���p���%�� ���1<�A � b8���J �c����R�C /A1�7!�-�� bx-8�>�^�b&o2���e�4���C>� �� �s�4��\`��EC ��b� �|�@�G ���e�� 0�@���#*1��I"Z|K�1����� ���j|����X�bx �����>X������L.bx&�����s�}�
1�g��~\�mA � ����AZ��!�C ����?&�<h�-5>���~bx��[��!���c�C��%(f�&c�L`]�JC /;�3�`����:�JC ���^4�p?�!��q��A � 0B �_iA#�1<����$�������KP �M��A�o���^ ��!��������3�u�+
1������!�g�\*
1<��z���p���%�� ���1<�A � p�E5��zj��V[5�m������#"J���[nY��b�-�-���������$��WCr#���|�����U�Vv�T$��I�#������l�k��N�%K�S�� ��<�d�Vc$y��1����j��������j�8�����d���MozS�����z��vk��&W<�
�5�gL � ����1�Qj ���
���s���\+���t#M\�@�������l���n�->[�.7�pC���~��a� �W���������H��@A`�1�`Is�@h�����Oo~�� 4�_����D����n����nw�[��v��jL���o����� ��v����=��l������� ��O~����k�m}��������.C���n7Y~���l��^{��l������U
� ;/�� �����v��]s���u��U�X�C �����j�~��o�d����'ZbXL"��Z����8�����w���s�=�I3�W� �q�5��c���|����_��dy�%��r�g�}�m���%������#`L�����#�1�.w��� k_j��B�(������c�=��/�\�a.6��y�3����j�#}A ��@ ��b�!�A������� Jp������^zi�C���|ds�;�i�C2��#/^���5���o���>��� � ���kN���#���n�a!�rZ��������~�<��On�*r8eX���?l����F� 8��c����q����l��%y���� c,TKiZB���Nj�������o��~�T��/��]�� ������� {�&C��o}k;�?�Y�jn����p� }�k_k.���f���n�>��au �]B����t��O~�y����}-�Z��<��a�����v<�� �@�tB o:��!�)#`E��I����\����{�*x$�����v���:���*��r[��?��m�*���)R��_�r{g�G?�����w��u"-nx��_����s�����o�N���#�=J$��>�i����lZk7��RX|4���v
�C��4��?��?o9����Ol��Y�5<�8����s�=�����xDs�a�
�iq�������c�K^���6�Y�5<�����_�b{G�4 ���
�iq���>��������<�����wX�����s��U�
� �0�^Q�!A �~���s�%cO8���D�G��EF@��������>�y��� bx��5�mn z�9����%�?�� G��EF���}ms��W6/|� �7!�YZ�m�,]����|�9����<�)��� 0e�m���|eK$>���ov�q�����eR���;�;��k�`������5����W���� ��G���YT���7g�uVc'������2L���y�{��=�y�c-�L�a�1�A ���@���-��.��@�� �5]1�2�bxe�Q/B _�!��/��� ��P%�v�C ����?� ��nm{�����zb���kA ��E ����&- A ��^�bxe�1�����^� �,C _�C�A���Jn�v�^�!
1<d�������a� �p!�� �@��@����MZ�@h1�2!����c���!G�1<|Y� � ����C����1�6 C�bx�����!��/��A��B"�A � ���+��,� �"bxe(B���!��+C�zbx�� 1<|�!��*��� bxm<��)����wk�C _�����D^�@A ,.!�W6iYA�E ���P��+C�!�W� �"���ebx�2 jB Urk�;���x �S��!K���� � �!� ��� �@X\B /�l�� �@�@����!�W� C � 9�E����2���e8� � ���nw�������C���m1<|VB y
A � ���^\��eA �I��}�{�/����#�hv�u� 3@~���4'�|r��^{5��� w���|��������.�����K�+���9�������ou2������m��������k�p�;4�l���{�:��� ����o��� {��<��]�@�����1�
oxC��>�4�z��������l���� ���!�w\|���u�]�����o��w�M�0g�
�SO=� ��?��f��wn��z���%�8,���� �1��~��U�����Yg��|�s�k�=��f�=�lv�i���)
A � �F ���x�SA`������n��m�;��l�� ��4h� �����6QRx�x-����/lr�������Z�v!���nw�f�-� �� �-�!�w��7�6��V[5��� � ��W�j��dL_�gs
�9 �� ��@h68/G�z�#R�\) 5���^���on�Q���g� ��������~���d���0��V� ���F ����oz��
@@����D�pz�-��d�]���p����+9f��pe)I�!��+��E�X�u��I���@wL�CR�� �Q�z��a��Z�1���k��^�l2e�X����jj S�iuA �lB �l��wA � �@A � �@A � �@A �&�p� � �@A � �@A � �@A � ��1����� �@A � �@A � �@A � B G�@A � �@A � �@A � �@+ ��+\��^A � �@A � �@A � �@A �pt � �@A � �@A � �@A � ��1����� �z����\v�e�W���f���o�y�{6� vX��V[�^P:=���|�[�j���/5w��������9���:G��<���[� ���E]�\r�%������g����g�f~�k�������\q���nw����o�x�;6�m���������~�|���nm�Nw�Ss�{��9��C6��A�w�M7�r���K��o������~�Vf����������W_���.wi=����3�L�T������Z����}����������"�+�����7��\x�����������cg������;��6V���w���fw��Y\f��~���jQ������������xO6���}�]v�e��]���o�z).1���}���h��6��E�f��9������Rl��N;-P+�� �@A`Q1�(�H;�@SF����ns�9�4���������� ���IOzR����N�J���;��N���~�9��S����G �<��� fgV`���_��������g������n ���4~��W`oo������4�x�;�����n���<��h��g�f��:P�����w��]
�����j ���������nh���w6g�}v�����y�3��&��l����?�9��S�� x`s������1f�h@��� /�L��E{G yd��'<ac�[q�!�?��O5 ���Zb�Q�zT���=l��s� 2���?��6V��� |�P��^���s��>�S��q����^���m�!��p� �l+�|�3�i��� ���7� {l+��7�|%w{�o7�xc;��ZP���)A � �@E ��("� �@X!��,�������!�C �[']?�p����!�-�G ��xl�bx2>��5�p��y�����1<N7�}A � ����uL�MA`E bx��V�����������m��w��M>)�.�a��u�M�4�rp<��[�e�;��VQ"�-�����x�V����!�W���s3E ��dx/���v����-p�����S���G �Y�g|��=��c18�V i���~��� �u�Y�����6{��g{��]w�uF�,F�!�C /�&�A � ��@��a�)� A l0!�'C&i����������zNR+e1����������<��v�g�'��H �%�4�� w��}��g����,�$ @ IDATb8��J���m�������3B-�����w�������!��_����~�� 4?��OZY`���;/[�q�!�����q��8�"��z�� 1 bx�����@A �tB �t �A �ZB �Z���F`���������1����W=!�W�
�"0$bx��N��!�C OE�RIA �U�@��U"�t3���@���'��x�� � C � ������@��E�J�&#bx2>��k������iCA �� bx(�J;�@,�^{m�,.��������� �h�i{�����_w�u�m���{�����g����gD���zn���f��6k��z�f� vhv�m�������K�������k���������E]���wh����>�, �V�n�W�3t�\d��7��b��6�4p'SdB6}���<���+�h������j���������:���xG �<7m%s:��{�s+?��.��v�m�n���>��_�.�����n!�F���c���sK�pe;[l�E�w� �ld��~���7pV�������GFu;��������� ������� ;j�[n�e[�[.z �c�N�>bX��E]�H�������
���U��z���]x��m{�k�}�]�\��[��e������~h3���������kK�KN�bKn��/�5}��%��,[�������]["w��q�����{��^
�����G�r�&�C��q�9u�G8���O���=���<������z���%�\�>W���|��5�\��%� {���F���+e�|{V�����aXz��qz�<2`�^���!=��3� -�m ������ :���'|�k������3 �e�|G����E��^���o/�6���|��azCW��o�V���t���������.����;?D���h 1<��������C���t�������������I���F�~�� �����+�S��3�;��t�|B���+� 0���_�H������vL�L��7�������9L��+;�q����X@ d���Y\5�)[�V6n\C������������a��qv�~���^W��[�<��z��\�� ����{���������1�9�w�qU `[2����[2U9���2X��b2�T[`��_|q{}22��3vTu��]��3%G�Te�~�� ��A=|�Q��z�O���� ������0?W��\}s;u������1����;�������n����|fk��bS6�I�)=) �S0$3�`�������I� ��|�R����U�����1:�?ba:@ 7��>��v��������y� A � 0dB Yzi{�@@b����j����4���}�=�Am�S��T��/���J$ w�q�� ���O&O&�&:��{n; ��4)RL�L�$b�s��4���;�&DB �P��G.���g�SN9������n ������T�\KF\p�-�u�����d������$Z;��v�)�1Z� �C�HD���D�&��H����=[�JdVA8~� _h�z�!�4|���>���KTi� ���6���~�k��\��q�����J�XL����9��3[� ���4~����V"J���_�j{�?�a?�d�L>���C=�M ��[\��$�$9�����*R�d����`�]L��M��h ���4x�Z9|��_l�W�d�]d.QA����xKF]bXR�)OyJ�h��I'�W�C�DR���w%E����uV����?���]�zWK u�Q
�P!'6S��O_��~�-�����K��-��lIR�����-�=9um�_�0c��� �/9�uu�-I���R �O�|�;���?�������s�9���$9ItZxb P��G�$!K���+_�J�I0�P���$ou��z��h=����s�6=��h�-iG�d��l{��W��A ��Wt��NO�o��k_�Zk3E��g��g�������[�w�7��y���d�t��Y�����.VAv���n>���>���o������=K>�Y��N�]�\��u1���8���C �P�![��'?���(����������NK�����/��J��/=5�MzJ ,�����J��B������}�c-��z�����Q_�X~�xM6]_�}��@���/�&���@x����.�9��B��_��_K�����
cA�$]_A���+�Up��?��zc��� ���$JF0d���{2�[-�0n x���1��]$� ���O�H�W� {����p,[3.v���~�# i�1�����2�6���\ �'n������v�]%o������G>�����f���y��;�u�Dw���Yi��[tZ�)�;E�������t��l���X��CaSXT��v��8���-��72b�(��;_������}m���� m1����%+�� �Z\!��vu��#�<�������Z���v���$�Q{�?�z �<��g�.�c�54���y�R��v�9���bSx+��A�\���w�2X�k����Y�K��s< �������
�t��k�����=�dXz-�) �����q�s u�az�o�[� � {��Q�]��5� ��jF ��j�~� ��� `��%�%�MZ�&�&D&@H�"�M�%�O;��v2gf�m"m�k�e�j�cbkB�\�����,^rG �W �# ��;x������&�V�K@��nB�����>�� $�TbL �� uKL �$a��QbXb�NH��K8:_�5��|�y����N��I���,��p��G KU�@��� �2��=2�\� ��HrG2���Oo���#r�����z� [;��#Z����1,�(�Do�H}%#��JnI� � $���K Kb�K2������9^%�$e��/q��,�2��a�.�^�T�m�-���`"������%��s�-���� ��*������� � ;�����X�Q��-���"��G'��S�MW�$������ku�H��
DU�O=�N=��m��|��F��_��%��$}���E�k7 j����������U6����0u���H�}v|�n� {l���b� ���5��o�)��k�>���1=�.�U�P��l��������W�������'�51�\�_ ��s]b�~G�=�|j����;��?�c|r�NK��l����*t�/{����.v(���W�k��}�0�C>���I��4����Q_aL)_����W���% ��.1,�Z _���>�%S�I"m���y2G ��3��H��{��?��i��c�_���l�����[�������\�@���?�Z���*�
me��#�8 ���{o\U��M��Ew,5�*��K ��c �L�k��1O��F��1�{�Y�2������_ ]��w�s����;>���n�u�� "�-��`J=�H��AF���\�B��]e��,�F����7�j0S \��"f6��k��Y��l �HFH���z�z�5���^�F����u>������� \��-h������p��<��1L�,��c_l
[m��u$j���kY�g���� ]�g�`����,���J���|��+;�z����Lj!�:F�a���y����D]���9���]����3%� ��jG ��j���?��#�%�Mp &,K&�&��!&T&�&H��[�{�Z&5-�7�K\��K��`��J:��I�(!�'�v1 ;��%:$'� � �gk",Y 9I�(�p�1���% *�"�ka�:�Z���%1L�M�%'�&�V�[�M�]J��]�azA��oBL�J�~�����nIr
��O2o 1 {�u�>2��$A�����(@ K���o��o�6��Q$%�f��"@BB�d$�"�,�@F�� d���@Q*�-�&�@��vI:��$����#������u�a2���%��Q�E���Ii���z$���r�K'��T��EW����dP��/p����aWeK=l��]�������l �eK0�x$'�eK��#�E~eK 2g_�$QI|0 S�D����C�D�����c\S�$���#q�� � �+)�W8^��/�Z"����';���N�)�.1LN�Eas��o���F�VH zP���w>��J�t������k�=y�S����^q-6����$"��T8�8����x� ���3�)E ��?�d� �g�������
�FO\/e����a�O�����a�C_�"?�g�+��.��JG�[l�>�=cT�;���H�+�=��x������ �����v���|�1�|��������+�._�v�
������6n��cL�]_�6>u}�1����|��� ��W'����84�"6!�>b��46��<N�~��2� ����l q 8+l�XOFd�w;�1�i������y2�7R������%���d����?'K�4����t � A���-��Enb �5��u�y�� ���\��<�� �X��};��82�������
,����m;m�q��5f�����Z��?2bk0%#zO��^���C�8�K �#<�G_���c�
��y�vZ '���%���y�A���]�-^�;�M��I ����W�o�����O��f��um7����)�!5�Q���ax� ��a� 1x��t���9�����M�S��< V�(b v�=��H��&� O{��Z�`c0�7�N �v-���q�1�M�������+���K �>�+ f,��� +~���Oz����V�h��|r� �@��@��E�H����C�K �$�LI:J^��d������d����n�&s&j&ZR�1A�T1I��4A�`1y2�C8(&�&E�U&��b&FY-{��M"��^��p$� p���#JN�PK(#.�AcW������9oy�[�1�����di�Lvv����M��k��+]b�g�bI d�$F���5�1�w
�##6�"� �0��0>}�._2��] �_ ��Z���I ����P*qFF�� ���lE=l �$�$���Jb���v��������d�W2##��&vI����Cd� �I Hmr�d5��vI��d�M�G�L"����� ER�!a*9��V���lI���+l U�)r��~8N� �aK�Y���z�[���L��eKdoId���������~vksE@�e��-��]�$1
_��ux ���l��HF1,!��^�L��������Or���G����������_�G���L���s�!? ����# ����~���.���a�9�Uzj�,%�$a�^� y�8�������� �{6�:B��!!l��k��*���E������'p�7�ac���������.i�R��>��V�dB��D�Tb����1���}o�>r��\�����[H��q� >���-��U��]$�����6YNo�)]�9�O)=T/?�3��/:V��!���D��}�0?����l$��|E-��H �������(d�'���x������`�|�C7�)5V�����-���_�U���(_��S�&����w�7�b<"�>b�
s��b~�lJ|�n?0�7�j������>j���������)��k\dK� ���[���f��lmEn�#��8��z�����c�y��1���u�ac����+qc�y�`��g', �10�u�����TK��b<�*,�O���1S���[c�a���O~�?XW}g�+��,�0v�����
C�h�F�������5q ?�8z@F�l��C��[�X���%�pm6�O�G��^=b�"#����������_l"^�����Nt�l]S��S�����]���E��&7�?�����O�q������<������5���| |�!� �R�A�����l����(�����X
�������B�tq�W16b��i ���:���f�b=���a@��I���TL �� �@A�6B ��E�� f�@�6���#N� �d�S&F&�V'��9�&�&;5�43!��@K���I�
bx�8'�V�K�I I8H�H��A��p:��$L<M:�Y&�&��h�&��%%K� �L�%:$(�����S��� �������KrT]��d�$_�FbM�$�V�W��.�y��a���������B����?u�M^%(�RM2&
��;Ir��9�yN�l���|$�$�$*��c%H��Jd�]m@x�����$#��n��u�EF�m~+b�,�������mU��I4���>������&��#��-�����l@r��%6RE?$��Z]lE��NN�@�9����p�9r���O�\=d%�XI�J�V��K K<� <r��ENd^��t&k��e��0��&�&QU6^���}��/^������?��8u}�Gm��.���!�/��/-9L�/��[���Z��%�%��g/|� ���Bm�vr��D��T{���G�lK2��X�$�G�%��/[�GO$�������/���H �< � 8���89�A a:�1�/+�����dd�K��O������=KR�,6t�N�dJ��^�@����&qN��^�l�](V]�*���aH'���=�q�X��������>b�|�����+���I5�C~ �|�q���[ ��b�
��+��]Y��oC���d�al���"�]���+���.z���~�r�|�fHe1����Moj�2�����';�b�\���p�bU��M� �n go�UvY�fa�`��AN���P�������B���!�g�?�����2��8�%���~7�!V�3������g��@���cg%w�=�#���d��n1��F�R��?g� �^��H�"��1�7"_������z�����7���cs�,bXL�|s q�x��
+���d���0y���R��������S���l���C���3�"���O��
�����#�G<���y��!����g<������,�����'aM�]}'�+6_��%/yI���������q�cb3���5�zb�
}�'�/� � ���� �`�@�-�'l�0wM�m��.1��������Y���U\��#��7 �y�r�$�
A � ���^d��mA ���X���������vb,��-�#%&�&��OZ��F&�����m�L&�H$�_ (��mo����
�I���bX���$�`+Y�-�GL�L[m�]�E ������.` uHr�(��X�sI0IdIO�K �<# �����p�����K�����K� �H�H �CbF�G��&�HHrB$IDI�HDTR����$�%'�GBJ�B��` tA�Xb����e�#�����##D��m�����L$ ��5��&�
:V�a2��.1,�e!A�N�k�D��2'�|rK J�8 )U+���z�X���#��t��"���Q�����,�������VNp��d�W$ ��3��7d�t�aI����v�MtF]�P�-I�z���))%�)y=��]�]"�%����l�b���7���%����v��)��&���Hbj���z���y�z��1L7�(��4 ��� ��F,�v+�GW%���K��� �gu��s%� �)��l�=kK�s� ��Z�`��'H�(]b�<�����h���1A/�9{1�B�����"��i�G�����WlO�E������o��|3��N�F��� �L� `��?RnE��S>�&�6n _a<�=?>�+�%���+��c�d�����b��h��-�x�][�� ��0�h_aa����
~����\�W��C*�"��#��>��"�Ged��N�ga���L,E�d�|r��&�> OF�B��.1,�3��C$ ��I fL #�C\�?'�.1������� ��X�����ytC\oN�w|_�6��u�dT�s�� Q �-�##��?���2vb.b�c�[�7$ ����B��[�����%�����.}��Y�=���9��%�������Gcq2!#m2���l�>���1�D��{Y�8Z��<����k������h[��� tSLi�����{6#6�����=�y-)+V�'^}O����v�����\�v�������������u�g�6�����bB�c����=
��O��fw���H����+���� �vJA � p!�o�"��@3A�$��9bb�Hz��^�k�37�293�56Y�U�}�3��`�{RB�$����LB,�p j�~G.&�&�&� H�����R�Ib�*y�`�" �� �d�Z{�&@^����+��+�$���xt��3��m�L�&NV@#��I[� �/&�}����?&��!�$M$:V1L>��Y�/I��V����`;%��
��M�������2��G����:�^�p����_ <��kT"[��A�b���Nn 9�B '���h�FLHDi�h���U��C$M��v��%���i���q�0 /��/�%��WI���$[��gK�H7X��$[���#��b'�xb+'���������������d[�C�Nz&��t���e�]�]d�����)9�=���] ����������o����Qb��DDH�!7�� M����|%�"��A��������$m%�����D \+x��'�e�5�(�'| =W7n�m����D1��K��{&[ HX�]�]��a��gu��}��w�����v���]����� �"�-�bCb ;G����0>�!vH������ CV���C��� !������>b����~� _���;�3r��l�����+,0F��W ����}�s[�U����K�
:���t��0���]�b%�� �.�6����i�$# #JF}���9��O8!r�"�� �s�i��&_��W���q�>����d$ ���p������8E�������'v �H���9b��/���\��F_�6��I�����,r�/��,R!#�-i�8�����6a����1�� �� ["����U�j}�yK�6���������bN�%���e 1,�S��� }��l��2R]�����Kz�=�5����;A�' ����������u�a6Q�)��������8���lV<c����<�\Y�n�cG�'|�8=�������1�8������
�Z;����i��L����|��7�u���XK�������t\�i���@A ��@������ ��L����&����b����m��d����k�eB-Q)1"9�Lt����>�o�n1�l/rXr� Lb�&�&�&�&������&��
������_�F:� �������"�%Z��{H�Zb�|��0���n��B L�%}������4b�$1dG���H��
��@���l�kGd$$9�|�G�K�� G�%�����a;�\��k������+lSL�q1,��~�D�*��(uI8��D�E"��q�l���l 1���%� �`K���LeK�@��%~�����Z[�����'�b�o��o[["�����j�d�M��� ��� ��=�1[�#�%�`n��b\�K$2%c��vI����7�7� q;�qD��z��}����,�:iA��K4��"d:]�s%���]J���c��e�l��y��J���!l���@��t���{Fl�i�%,%�]C���D"#�2������1����~�%�������d��_��v�
��v���/u�y�����$��0 1������x�G [XS�B�[��/-_A��|��
oxC�+�����+����+�
cP�1�4�+�����|�b�� S��g+������@���W �s�b����n ]�7�Y$��������� ��W\�9HGD��L���}1l�'# 4��3����Z����#��I�/c�Xm\Q�x���<5��[�6��
�~1l��{�" �W����o��v��ob�� +6�������l�p�v���� ��������-�G�d$v1,ft��^�PK��W ���I����G���=W,{�RX\���2�����������:��.1������h\q ,���
��v�7�l�%F#C�
~IN_���*.��$~'3�� �� ,�����b�-�2�AO�"�Y����<����h�����[�g��Q��9� �@���� f�@�6F�I��!��k^��v2�w��n�{� �M�M����Dp�|!aB �!v�w��a+���x�<A�����v:x����0AF��N!I_���������+����D��P^+�� rDb���I:���B K0�$q#� [����?����ABI^�)X��D� ��D�]�d4�8Or��a8?���l�KJF���@a�$ ������ )v K~����[��g�tU�%�$���%����^�g 1 g�[���u���T������d�k_��6��|D�����G��-I�y�9��"���@�����/1���$ 9nG��%�������r�;���%W��E$C(]b��n� � W����,��-]�����qw�P`R>�+{f_��<��BVv�[���M$�����u����z���
�r=>\�����Ce�1\r�0C�x��Y �a2R�!�uR�d���Ov'�$�1< �O��ac?b�|�I>�>>�O0~ �`��?��b�|�����Y�U���Z�~c�� �~D2=1��E ��8 R�j ���X��3���>��?'r"#�*baj��E��2r�rR^��W�������A�ac�8���E �a2� ��W��WZ�e��x��>� �M�<���q��s�����&���d"f��-��+^l����?���y��t�Im b�?��?i��Il��z�&�"���9{�+bN��.b��u�1��v� �HFv�"�][\��� M��E/]b����bS�7�8�c�� �&6����'�*��r��������q�s,2P���IO,��G���L;�W�o�K�� [;��.��������sA �~B ���o�@SC�K �0�l��H1�����Z�����r��O�@uj�n��U�,�p���O"� )�+�h��WD�D�I2R��Tb��V5KpH���D��Y�T�������G�,bX����n;�%�%;-.@b���B �;Y���
�$]KFp� ����]*���$�%�E ������I�#�����<c%���H$ ���!������D�������zH��%K���)�O"��K�l����eKv���d�����
b�%/y�:m��az�^�����0�?�[��Z��|��v ��J�I �+�F�t���S"��5�R�d��(1,INW�r\���3���E~���� ID����� �J'��s�3���$+;���1������'�e�0=���|8�����Ce�u�����'g�?Twa�t}]wt��@g�H�������Q�v�v�X-�k<�#�a� t��n|V��_��_\��*_�p�#���0���I���|��=b�|E��~>|�"�
��Rw �db�����T|��W|�����l�9�] �s�0B���rx���1�|1�[=���6'���z����!��;L��H^d}C��"��g���k��~7�����\�x�6�dD/�H\�\##���o����cq�~�O��u����:���ar!#~���F ��]�����/1,f�8d������ �0�,.� �����/o�Z�������u-Z�3�Op�7bS�1�4������]�1�?se 6jq'r�m��j�s�������O���m���E � �-��7����_���uB /���}A � �(�^I� A �X����&[�J} i%�b"k"*�� �TL���\�S$�B �Gm]�p�iri�,YS���EF5)�z �!��"+��."�������&���C�s�L�&�&��Y;�C ?�D��j��D�������Q% ��b�
?�~#c���,� �lh������
9LF�S�D`����&1L����c�=��.�U!�N���(���6z-["�Q[���`bK��l�b �b��;)�!"Q2q]��6�n��LD��}BN��v O��cn+j����_���� ��(���YB�>�1<N�
#X�SBN�s�_��i|�%�%��,�jq��"!lg���$!9KZ�g}����g�v�q=d��<{��<���[\='�pB[�8,�3y {�)M&�$�pIce�v�a�}D����6��3~�8�� s�k�szX�H�$�C �"}��O��u?`��CkL��G|��*_����y����|�$��A%���^��-�-$S��L<Q�"�����I�;Z�� ]�H|f .!���8F����"#;�-�"#�wE�����yN����|��^��G�� ��ZCZ `�Cr�+��Zc R�1���q�m���C ��A��������\�@�s��b'���3�� �� �c�������s`b�&���gA ���d4�f�|���o�����2�h1�r�ld��B��f^����l�OW��~�?�i��|`���u\q �� 2��`g!C������c� ����[ c�� cA�L�����q�&=1�1V������F b�$�� �@�A ��tpL-A ��l1lB������� ������E7ie�����b�k%���IX���bi'��#��EP �L����� �D�WxK���J���y� ������&�Hc$�D��uz�+n9�.��?���[����p��*��%#� ����$9#��#������XC���9�'#;}�I�U��dT��i��B�����v\���S�.
� �5����������]��4�����e_�%�"'�j�+r�d�3��#�HaKX ;;j���$9I�y^�"IeG89IfM��T�g���>�X�`�[kj B����RIDIc���Y� ��a�9�T���.���}��vz������{��5����Xte9j������ �GO\����a{��M$�� ��#������*�����]nu+i:� ��$:�����K��Z���h���.�Id��)l��:�p ��5��� �+��|_���+�I�9�}�� ������oL�#��W������W�"��xb~��������Zc?H�bQ_� ���hdd�b���J���������������T�,�a�_�Y���k���<� ����7�o������/1�^�KWF\X�e���P�EF�c7���5��_���� /��"t@�~H`1-!`��� �������#"��]L,!#1�X�|A\�� 2�U,��#�!��M��n�|��gU��E kK���q�qbS}�?2����[�I���)l'1[6P�$~G�{,=����kzRwh �G��EX�\��ao^�s����1� ��&"bx��A �u!�!��I� �I��@�:�J��"���0!31�� qb����#�QbXB�d�o&�p��I�%L�%�`���S�g�Q��NbD�m�L��
D8��6.i������B ��])� Yb�Vb�~��d����R6�%�$��� %�����HC�*v� ��y��#���d���� g�qFk�����[��i�o��4-���6K���7���mBJ���gI��.�:~�p%��;9�
���"Q��%���� 9I@�;�$�������L��''��
["#�X��-H m����a�*�$����h�$��w����n�<$�j�["/&���1�_����\��%��,�GW%D�tU{��"�jw&y�N������w�` �DR��� e$|�Z�����'v�X�AO��sa$IM7%��BR�=g�9�
1\H���
!��3:(���2��[aw���ER�]`��~���W5�v�a�nL����
q\�
c���"%���%K���c���m����b��a�h�����t�Xg,G�A�F`U����)q�1���b�z��E;0'#s!c�����sc�k�K�`,q-:�v��A,5�[I�i�l�C�u�Wm�����t! ��_�,��a�������KF���[C���G�I���H?���e�c���qw�5�q_qM�*�nd��a3 b�\�����v��(�pmwn@�K����b*q�A����^}�O����q*�bA���<��.1l�E����v���u��T��v���y/�'�1�� ����u�����0"[�@|l�=�?�D�H�
{7fZB��������C-�� �@�xB o<v93� �^l1lB)I-QbBnRe���(�cR.�hRn'�I��I� 1�5�E�� �I8Z�zub 4�6���$�q%�$�F'�d$����CB����l +I�*F�H Ya/�h2�% %&$���3��p�0���1|f{;1���!t��"I+ac��haS�@2�;b��/~q+K�Kd�Qlw R��$/�����%���<���cI ���$�%��v'�A�]T@7�I��v=�3�s��a���Q\��Y��ar�����tZ�����(���$�cw[���6wlI�N��$��},[��9�D��H�� e��$'�H6�m!���e' =#'}E�[� q��K��+����E �n���v�[("ifq����.]b�� 6d�-��P���J0��~G����q� ;1N��uT]����@O��v���LD$��e����dp�3�����8��naX���m
�L�t+i@� �3���s� ���~�W�1�)������Z5���@qU����}�BL&��+� ��^��W����8� ��+�)��Q_��#Gc?�������!�o%���j�r'��B5DY������,��; L����A,NF�B�}��D�q�"@��?wgc���q��D��M ��� ������K�@~7�'x���dk��_���Y�JF�S�LF����b���1����-�82������l�1[%��#�+�� ��k�*��1,>4/��NW�H�JF|�y��O�+��0��Rl������ ��Io�������H�y�_��E��`���w��1�*�S������Fl�8~���������OO����:�z��Z�����G�xo��E�l��
y�m)=���� �C o�/�YA � &!bx:�-� 06�����2;�L�L�$LFM���J�C��� 9�61W$1C b���G K�� #)��Jj� ����Z��+�1������/l� ���L�I>��� �pWq
�q�1u� dI0I-%��m��$���������`#���ErRB�L%�$��A�4�8&#� x�M������fO�>�HHW�bG 2L�D�4�a:D���D���S�)]q=�$��(��?u\{����?���D�6J��]7=�6 !8�&����%�!� [�X���4�����H%'���$Q�X�+�)�ONl�n�7 b��5r�H�Ft^�$���$�""$������"L��N
Im�t��*�j��N*�&�g' ����g�*��v#� �Bj�[r�t�v�k7B[%k%*����5�u�2��:��-����g�Gv!��HL"|� ��|zO��*l���g8J��D�c�����~%��%��8[�����I�K�3��{t^���C/�����t�Xl���v����@��]b�o�-�F�kL �}������79�s�2����W�|�^kv;v`����A�l�
�*!�����da,2~�u�<�8����k#-pk�Y�#~���?�.��p���X��##��9�����.�W�"NV�W2b���1�������1���P��2���z W8��u 1�������vd�}b�n�D�8��Xp�Nj����=�qo ��1����R�c 2� ��$~�@@ �W����iHy2�;
��B >V<��C��t���v��K���k������~�#�������|�0���S��� ���
�� �X��6��N�`��A��'_q�8��$7c�qM �S��q�q'+�{�G���P��Vp���s���W��^�&�s�*|�z�|�����l�_?B Zy
A � 0=B O���@�E`C�a�D�(�Z�L�:�j�+e�gM:��i�&��3a��6�WB ��c��q��$s��d ��V�Qc�j"����[2����� ��� q�B��" ��LM�M�%&��DFf�R��$3�����>��v'I��Ub�S��I2���{` ���
��VB�>H<�C]�e]� ���G����$?$9�JdO��,�'��9lv*�mz���?>����jm#g�O���v9�K��l���[�4#'�)FN��eK�aKl�-!��%'�/[�4*[��G�Hb����2 b�v ���!����q�A$��g�.�E}@4�C;����"I�������n�otUb\�L2���lO^�]bX�� ,�� �I2��A���Ha���E��
IC��`D��O�� �CO�`���2=��T�X����"�������=�]=t���*{� a� 9��-�+����|? ��'�M�=����=���>��%�>�_PB �0��W��(1��Z�F�>���+�a|������I�%_aL��-6�+��|�����z�
��|E-6������|5A����������C� �� ;���d���x(>'[���#c`�A� #� >vIDATu[T&>K;fV�0}�� c�1K|Vc�q��<s4c ��x� e 1�f�MF�^���9���xb�i�&#��~Q����L������W2*Y�_\���9�E� �If�"��^q���E�$;1����`EvQ�[�p*�.��k�#����|=c� ?o �.y� 2w
�U�p����$�jLr��������W����FO��X� d;\����: �{pa�0����� ��x�^=��?���d@O��7k�9
]a��E ��t+�� A � 05B O
�T�@�G`C����D�Je+�M�L�U&�&�&&�&r&a&�v_�d�]� � x&�VxKLtW���V��$bX��n�������������l� �!&�� �$X$#+�O_s;<����TF�H<"��r=r3�6�FBxo"���ubX���6� ^dT�0I���� ����"�CFG6�X�_�G�%�!9�������:� � %�JdO��w9�3l��H����1����Y��������_��8b���-�x�%vP��-I��_%�����I !��D�/��%\$�+���$�$���,9�������]�D.�&iF�}��+����o���tZr�g���@�/�5�O�G���Y�y��.1�M�EW�IR������X����:��#$��$|.��yt�o-b�����{�G���$4{�sp�'0�O����=���n�IO*��.�1\���W~��Y �o��qbR�����AW� ��^�K���R�~�U��B OB���V}�0��� ���Q_� ���3 Q��� �+��]_a� Sc���p���KjL�+�<�� �����v:���g���!�-9��1 �cX�]C�)�N�g�5?Ll����� �af���������X���&ck�2�����9�- C�|fE �y*�@qc�6�W��<m7�-GadT�t��~����"X�8�����-�t "��J�W}qL��C����\�$s���"[�Z�)�+9F 0� �b �����$��G�����Ob82���
]FD���}��U�c��������� ��(�S�0CT{�7)6o��wX�#�� �f�O��b;�� ?f�=� =3���(�����_��#����Q=�c1 �Pzbx �k� ��JG ��J�p����@@"3�R����A;�H��HK�Ip���8u��� ���[w��V��qVKx�Y)a!Y"q�=�[�j{?��� ��kI��rxI H2X���Z'��$]�a(� 9���&�L�e�1�J ��U�a�\�C<}E2G����b��=�im����E�� $�$d$%$ � �v K�I�"�,�����Z��$���vkt���q �;���$#uw�"#���U�C$<Jv�����l� ��$K_��q +���d�S���V����%[��[_ZiO����v�[RE�N�K�,w��G �W�_F�G������cK|���h�Y���t��J �MdD_Fm �������h��W��aKn[L�$���� 2rBPH�m�c$��n:�-�!Q���=Z���N��5�������E(���<Z�r|�0���!��m�s�������}�l�����^�������W]�q.=A��&_�CO�Vs�KzB>����e�G���<�@>��R�nzK%{-��+���>���(��_)���1��'gc �7��Uze�{�����s��1]\@w�~�
~g3|���q�t� M�ra�}�0��#df\��+����F���0v�� teY����W�k�W�% "����BW�W G�
c�J"����zc5�yE�uq-l`)F���y�pt Y��1�O������H<FF�|��z�1�1�o.�z}M���]�a� 9jls�x2G����{z8Z���� iA����I�0�`���@��uqU�1Q��3�J0��1�8����b,�[����JV���~�y�0��qL�U_c�c.�c�u+i�#�@F������vv�8�o��m>���q� ��68��q�>�%���;}�G�^�+�e;bS}���M���b^)6E�wcS2t�����=�ue�.8�:��lp��C�AO,F��f�hl�� ���&�47w��qn/O�F�� 0N���)���)A � ��m��
�� A �����d�����(95za�.� ������sMj�#� �dBW��nb��V�"�MZ ���cF���>O"��oB
{r����HW��` #h���MF�&����� r��hB�9� ���'1�M*��s���Lq��D��i�v;O����r�����p�D ���)�T����N�L�gx��W6 ��R����CFl�C�G%#uIB�&�`� u���g7���&�r�v�6 #m&3rv�"���X;f�����]��p��x����7�&' ��W�-� V�9��O_a�n�x����%�I�D�r]��z�E��+;� ��G�( �;�/m+]$_�)99���*y�S}��S��_zU>��y��_���p'g�]�JW�����?,�a���t���+{�����3\F =qmut�Y{��8{��\��:�u��__����g��8�N���������g���6��s��>:O\�V�:]5f�7�c�� ��_��;})[�_���r+���q�� b��+�l�)|�8_Av|�?�@��+����k������5��WL�1��dTc���/��{v���AcA������[�C��/^��
�p �)y���9�u�
��^8�� ����9��;�X������u\S�b��1O��F�����n���=���v��c�u`K��m�]���~��e�NF�Z��~����d���B*���N�/u��n=���y �� n}s*�+��v������ ���v����mtB��M�%�GF�8�� K'���������p�'����k9�����)6���#�+6�v�$�[_F��8�v>���������x��-�E�����]��D=�������8^;��u�{�o������3=P�k�� �@�!b�6,�.� ���d�mB�H���3�M�p����s��� ������b"k,�%!�_I�j���h >���� �����J�� �Mn��[��I"���� ���Kz�����K�! ��I��-����H�C�������n���^[*!* � �% ���<��5��������������o}�� 0f�K>�&�F�1��$~���y��dY��t�|�������G�K���z�|:���N0������GR��,b��lh]�Z}/{&o������u�����U��������������K������3��>����~�����.^w q�;p��-�|����\ _����}������g�}+������� ��t��5��}8��'�����^��#�H]EL9�|���[�m��{zS�~w��i����i���2�������m�[I�1^�����p%#��w������O �Cu��������x��eV��jx��6��I����u�e����~*�'.����j{�������m^������������������mQG� ��zB )A � ��� bxyp�U�@A`�p�P� s��I���5{�4GBI��m����n�6e)���[(��{������b�:� A �ED�m�O_s��O~����El�jh���>����������:�����A���N1�9"�so=����� �@A � ���� �@X�x6�g���J9���]�� �����l�����`�����,_����g,��e�� �����@A`C�hO|����F����!7�� og)!��F�����'��(��US�� ���d$.���s_=G��wS�@A � � B G�@A`� `���y����_|q��% G�)|�� �j�X��z��Gy��������6}G qDK8"�S���[����e�]���fKy�C���� � �� ��,�������f�l�#�<���=��A�c�6�m�=���K=>��a�dQ�������{�v�B ��^����en�\2���At0� A � �@�*!��
g* A �EF��K/m�������Z����HnQ��>0I��g��s�9�N�S$��@��n���& ���$�%$�R���g�}��iEA ,<��!|��g��EDzn������os���}������ ����>����g�1���{�1�E{�#�2?��&#w����s��e��~���k��<���A � �@��@��E�F��@3E�s��:�������rK����7�Yv ��� �V[m5����u#���~�%��Bz���o����FFv=l�����$G� �����/��$a|�;���%;���k����%� � V��w^Kh}��hv�a�� ��G����v�-c��E�� ����7�����-���;��cK��������BFn -F#�8�y� � ~����[o�l�����t>� �@A�6B ��E�� ��
G�������D� H- ���#�6xn%�v��X;��s+#$��)������)���J���O A � ��x�D�J�X��N;�w 1�o��6�[M��vp_}��ml���v�e�VFbh��S���;�\u�U���_� ��� ����Bz�����@A � ���^D��MA � �@A � �@A � �@A �)"bx�`�� �@A � �@A � �@A � ��""bx��6� �@A � �@A � �@A � ��@��)����@A � �@A � �@A � �@��@��E�J��@A � �@A � �@A � �@�"!��f�
A � �@A � �@A � �@A ,"!�Q*iSA � �@A � �@A � �@A`���"��*� �@A � �@A � �@A � ���^D��MA � �@A � �@A � �@A �)"bx�`�� �@A � �@A � �@A � ��""bx��6� �@A � �@A � �@A � ��@��)����@A � �@A � �@A � �@��@��E�J��@A � �@A � �@A � �@�"!��f�
A � �@A`�������/������������v�m� x����{���
O��@A � �@A � ����D szA � �@�A����kN:�����Nk.����nw�[��W��9������4� �@A � �@�@��� -�� �@A � 0LB SniuA � �@A � ����t SCA � �@A`���+�����7��\x���
7���^>����m����|��� Q�����K���|���>��� {����[n�l��f���\0� �@A � �@�.!���gj A � �@A`�X$b����nN9�����On~�� 5{���w��]��������
��?�����|g�������������o}k� �-��b����� �@A � �@SF ���MuA � �@A ,.!���&��xl�KA � �@A ���@��� ��!� �@A ��B ��x�B ��&�� �@A � �@X �^ RL �@A � �@X/B ��)��xl�KA � �@A ���@��� ��!� �@A ��B ��x�B ��&�� �@A � �@X �^ RL �@A � ��*F��k�i.����� .h����6?���n���z����w���}����>��g�}�����9�����N;���������n��_����O ��M7��\u�U��_�\r�%�W\����?mn�������e��;4;����o���}��?���Z�2D-7JV���a)�\�\$��f���Bc���.u+KkRZIW.��\5�F�����H�����m�~=^���������� �>>��8�s�������^�w�Z�j:���R������ ���o���/}i���]�6-X� �\�2����i��u��W_�{Qg� R�J�R��5S��
S�� �a� ��=x�;������Q���[� �|���z>�� ���U+<v����X�fMz���� j���^u��Mm��M+VL\���FM��/O�-�[}���L�Z�J�7.z�]��!�z��������>}��F������?�1�^�:���o�������S ��G }��/ub~� '����?����y~�����-k��{��m�����i���c-��N�:����s'��
(��
(��
(���!`0�o��U*��
(��
(�@��?�8��U�V�e��E�HPJhJ�GHX�Z�T�v���y�0 7 }����SO�*��eK��/��R"%���aC�����3*a!b4h��4i�k��i�_���wT�x��t�=�����G8��E�9��#�����~���F�� 2y7�ij�&�m���j�!��R�J�����Y��C=����1g��t���G=���~��]��F=���w������w� �j�N��Yg��~�� ������^~������a6��m�����o�*T���������f�� ����##(��g�}6�����6 � n � �O:��� �8������=�:�4��d���b���YoBv�%t�Q�F��>?�+��
�wy�?P@P@P@P�� ��� P@P@�� � ���w��~���G'������ ��b���u <V�w�}i������� ��t��1" s�1�f��M0L=��MK>�`�CWtiN�W��+�����,�p��]�����#�<�"�%$��w��n����Rj������������&BW�������_ ���a����?����+� �`����J��w���]N��;������ ��� ����?���<������+��
(��
(��
(P ���*X�
(��
(��
|&F
��KH��aBJ�N�8��t��GG0HG'c�3M�-��� zh�����5�l�%MG*�1��=�P����\O�-+0� �����#h��iS<���]�t�N���[�x7��\�I:��L J�/��/�������t��
�����3~������G=���~:Y9�����;���xd����z������g�;3~��Y:�.\���_��k:�����;�� ��zf$6��} ��R��|a0��/~��xW:�{�� #�����K0Lg3n��&P��t��� rH��}{���;d��� ���W��x�vkS�JLg2�t
s-���~t ���Q���5&`?���b������5�����?S@P@P@(���ue�KP@P@�
�>��3��[o�0� ����L�S��` f �8�P����m ��}q�0����c#&�$��c���O������t�n��1F%��5+��%Xe����C��g���e�t�#�'M��� ���i� ~���|�y ��������.]��2>���[�N`�J��0�����������Y���w�7������� {,���{��� x��?���C�3]�����p��7F�M �p�=z��g��k�� !;��s;� �y�X:�Y+l� �yF��2�4��1!;1�� =h����e��#8� �����;�����'�<k�g��X�^��'��U���g�o�"��3�rP@P@P@���}e��SP@P@B�}f����k��t��mKp[�N�]�� ]��t?]:D � � { z �(�������������A���H�3f$j� ��D����}}���#,eT5��)��A4�g�f�<S��^��s��WG�L�p��z��fT��O? {1b�2��w/z��J=��'��j������ �{����t���0���C��Z���������ba}�57�/;��n��E���K"�� ��xjb��?t^������t'��_��k�'@f��v~���#� 8p`�+�)��g� �@�kxwj>��s�g����V�*��
(��
(��
(P� ���
Y�
(��
(��
�$@�x�'�x"~��U�t�Yg� =zD��/�$x�7o^t��2���`� �����(���'���yAf� �B���:uj����� ��%L����5�
��g������ �8i�Q�!(& ���wB�� t�bD�;�|:e�����Gi�a��e��� /�0FV���{��W�����R�����p�����&�1�W�e�]�3u� �q',����"|&0/��2�+��b�H����}|�����6��+�}���Ne�f\6���G�����4���G��G��A�����$B����oP@P@P@����py]�R@P@P@���g�8qbt���I�) ����B�|2v���t��oq�0��1K �n�� ��7��#6{{���3'�{��;B�_��W �����0�'�� M�a��W�c�;��z���������#GF0����Gi�a��#�u (�S���Q�F�N�:�� �#�6�� � ���%fTuv� ����s�Y��]�������|��7���������u��f�65��&��m������O��u��e���.d�c��a�/1!1�����gnS[������
(��
(��
(���I�`�<���(��
(��
(�@�t���y�EE�HK0����/�<�IK
�8�p����K/��B����.��"��&����q���f/]��9�*.��b�IHJ�J���c�pe�3���o������}��� ���7�0��k���?� ���{��'L�f^�]fuv� � ��?��s������mBpj`O`�[:���@������f<6k��o;�cF��_tI�<a����5��c����e�3\\h]�=��
(��
(��
(��_����-��P@P@P`���_�@n��I����M����=_ww�2^� �NW��� ����P�.�+V�i���5k�D-�|��0������k1���'��0�q� e 36{�����n�w���t3�9;6l��F��9��U�T�����+U����w�4�p�>}R�� w�W�7 |�A�'�
���K-\O��A@� ����i�� �y���rq������ g�}v���5��gsq���
(��
(��
(��
�� ��[�>WP@P@>��?�����[o�����1�.MF
��n�X��nJ+#��.��ww�0"2�%p}��w"�&����|����������Q��0�/c���?��d�P������ F�5��U0����k��&:iy6� �����Qa��+�� ������^�t� �vws���s��1c��3�z��}��5k�.�/:� ����� ��&�%���ky���������u�����~;�g� ���w*i�yI��w
(��
(��
(��
|��_���R@P@P@�= �$x%X$�#&�c,q���� ^{��i����n���aaX�@�#�;�@�k�l��+���t�D��B�
��r�U0�S�PWV�4�`D=���L=�3 � e s/����.\�9���#�%,&&�^�pa�����t�����^��aFN���?L� s ��x�CX���S�N���m�6������;�Hw�}w"�f���=������O�[��}���~^��
(��
(��
(���-`0�y {P@P@(BO��+��bG0�XhF�� <���#��Wq�0�!�gt��q��;��a����� _�{ �/]�t���e �K=��KMt �f���z<����n����z�y���r���)�`�N�2%M�81j��`��[nI��W� ���y��1Z�U�V��[o�P�Z�j;-W�`x��� �.�90���'O�n��N:):��b�9rd����b�vY���~z���j���n�(.M����
(��
(��
(���-`0�y {P@P@(�l�2������aBa�������t��q<{���y����0���������nKO=�T���& �{�A���!� ��V�al������_==��#i���QOY���o��f��>�lZ�vm� ����G�Z�V�T�F���!(� :z �9�:~���#&��y :tH_|qt�R+A,�9���M0|��GGo��� �/����@>���sO���G }��n��M���c����=�g������U�VE�Kp�������8j>|=P@P@P@(���}��OP@P@B���`w��Ai��I�z���FWi�>}J�D�{�������x�| ������/&�N�2�� �[��Vj��e��\G�y�A��+��������=�X�S�0�� ,�z#M�J=���7������G=����tG?��sp�s�u0�z��&����h�zB����71��a���W�^���/��������\pAb]��i�sf�o������s�����G���vZ\t����{��7-Z�(�a�&,������*��
(��
(��
(��#`0������
(��
(����t��z�y��� z(F)7m�4�> pK: ����KL8����+V�H&LH�>�ht���KW�q� W�#� p ���Q�0�f���1{�RO������
"&� 5jTtHs^Y���nd����@�q��i���������q���_p��]�v�� �{�� �p��=�E���n� �0R;�:3��� |0��������n��}vx8�����9"Lf/g:�=P@P@P@�W �����NP@P@�0��+��}m���?�*U�D ����F��
/^�8]s�5i����� � S#M�I�HK�K���~5��;}O����3#l,�Q�t���J=����E���^�z��TD�?�}��Yi��1��(i �8iFF��t7�t
��<g��] M��>��B�|�0!<�2a���z���?E7���? ����� ��t+s��trS+��{}�u��� ��c���4c� ��t�A��j��}��
(��
(��
(��_����%�sP@P@P�LL�0�#� ���k������tz>���i��qi����9�H�|�0{��.���J>��cc�`�� 5y)�HSM���O������~��a�'f���=k �o���6���u����m� %�� �zl��Y�| ��6m�}�G���-[�;#�?�������%����!C�c���|G�`�k_�ZbT8�G qD��<�a?cF��2=]����[O<w�O<����� �Nf���5�������7������Y�}����q���A�=�^P@P@P@�� `0\ V�P@P@P��t�2������N�:���-�-]�+V�����?���7o�@�N`�L�> s����w�W-�& �c��MM�4��i�Jp���W_}5]���$r\}���_�~l
��`����"l<������C�1��s�L�/{�h�1L=G yd:�����+#����na�G���"&<����[�g��j�� �����d�i���}{:���� ����F&V�/�E]����?��d���]�v���;u���x����aNj��Q�\K�4�K8�{~�����J�6mZ>|x����N8!��5��qa�e>'�����`�����)��2�����gg���aI�M�0���Lp���
(��
(��
(���"`0����u*��
(��
(�@r>���1*�����Z�j�M�6f�o�>�MK8��e�� x �U� s>�1!`��a�F�f?�u����5k����Gwi���w�C@��L�4iR�sKH�38�{��O;K *��2~��_E���-[F�M�*���AIN=�o��u�� =z�����e \��3��e����6������x��|- ��9MxZ�m, �yWF93�;;��o�[�ny���W\0L���}cOe���C�6�������3�Ap�9�6�� ���7�H��������� �=������b�8��|~�,�y��a���� ;�Zd���
(��
(��
(���]�`�����)��
(��
(��N��vt��2>���0����&0��t��-��^� r+{� �9�@����K�F�L'2�# ����qo:� ��:�_vo�>������5�� J �95k=z����k�v��s�a� �2]��-�z�K����C��W��� �Na�! g�_��{fa-����� �b�u�� �m������o�>�|�>�!�..$ 9rd�t�r�c�Hg�gf�wqG�`��a�q��t�����gBj����h~�j���a��0`<t���c�8�����=x�9{F�w36��t������CB`�K'�K/����s����s��u�\�r���^P@P@P@�r-`0\����P@P@P �������F����1����0�@�`�ZBN�L R 7 ���� ?c[����_�JfOY�7g�� 2����nRj`�2"!A$����t�f��K��%���W��"����n�)�O~�}' � o��% % � �?^�vm�+A+{��^�L�2�41��y&A��y�bb���O��^�Xdc�����zx�QB���U�VQ��w�N�0!�q���t�}��;dt�k���t�/Y�$���YS~��n� ��o�Y�5��vZ����������'��i���� ���������� ���.���>;�jdP@P@P@�%��}i��UP@P@B�p�0�������� ~8�MK8��
rn�I�����f�:B���0�soa����������Y��}�C){���,�3����%��>�O��(��={��b�A�t�^z����-a,us.#�;t�c���%�� :��L�����k���YF#��_?j���7�*O�81����j�V�}�'��������K����������i�A�0�;p���j�wM��|����C�+��m���]:��)��z�A����+�G�U�R���y�g{�I����!��9t(�Q|��g���YL�v�g,�C��
(��
(��
(���L�`��-��(��
(��
(�@����[���^� �����NX:g�p%&�$4����s�,n��I�W��N%$������ts-*a*��t�r-��+]���t������t������7�P����{F83���H�7�5u��K�����9�� ��0�zB���:Be�L:�yF-g��E��=�� arV�H=\��e�M������m����qA���}����w��1�B� .� Fi��[��/ >|x�����o��f�3������' �3�w.�� �H`�+k�hp�l��?#��?�����\��x�
(��
(��
(���E�`����u(��
(��
(�� �������0�� Q X ���vN�p�T7o� A,�!#� K��%�e�_F
g�%�/�2�'������\��iJ ��kO�I�J��Ij��7����zx_�C=�C �C@L}����{�p�a�� B��z�����=�s�5s��w� ��o/����&p 0`@���K��g�t 3����[�u� #��`W���5k��]�s��.��� p��a�������:���?S@P@P@P�� ���>P@P@P� n( <8�\��\�n�;=h��R�� ����[�
(��
(��
(�������>�T��
(��
(��
�?�l�+y��ai��1���������u�V��
�K��I
(��
(��
(��
�����^�y�
(��
(��
(PX�;��{��(k�Z�O/� ��5+�gL��!CR��]S���K�c0\*&OR@P@P@P`� �����P@P@P@��`��+V���G8�~���d�����R�J����J�F�J-Z��i���� �K��w
(��
(��
(��
����p�8zP@P@P� V�Z�f����� �������3�E�>�M�6M}��I�{�N�k�N*T(���p��<IP@P@P@��0�+>/V@P@P@
K`��5i���i���i�����O>IU�VM
6Lm��M��wO���K�+W.5��p��<QP@P@P@�=0�c:/T@P@P@
O`��Mi���i��)i��� ��U+�o�>�l����
��m��5-]�4��p�����A�R�n�R�5rO�{P@P@P@�C��=��2P@P@P��������m��%���X�bb����?���~����cFRs������U�TI x`��Q��z�
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� ����
(��
(��
(��
(��
(��
(��
(P`�����
(��
(��
(��
(��
(��
(��
���p���o��
(��
(��
(��
(��
(��
(��&`0\` ��*��
(��
(��
(��
(��
(��
(�@� � kC�X��rp IEND�B`�Re: [HACKERS] Custom compression methods
On Fri, 1 Dec 2017 21:47:43 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
+1 to do the rewrite, just like for other similar ALTER TABLE commands
Ok. What about the following syntax:
ALTER COLUMN DROP COMPRESSION - removes compression from the column
with the rewrite and removes related compression options, so the user
can drop compression method.
ALTER COLUMN SET COMPRESSION NONE for the cases when
the users want to just disable compression for future tuples. After
that they can keep compressed tuples, or in the case when they have a
large table they can decompress tuples partially using e.g. UPDATE,
and then use ALTER COLUMN DROP COMPRESSION which will be much faster
then.
ALTER COLUMN SET COMPRESSION <cm> WITH <cmoptions> will change
compression for new tuples but will not touch old ones. If the users
want the recompression they can use DROP/SET COMPRESSION combination.
I don't think that SET COMPRESSION with the rewrite of the whole table
will be useful enough on any somewhat big tables and same time big
tables is where the user needs compression the most.
I understand that ALTER with the rewrite sounds logical and much easier
to implement (and it doesn't require Oids in tuples), but it could be
unusable.
--
----
Regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
On Wed, Dec 6, 2017 at 10:07 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:
On Fri, 1 Dec 2017 21:47:43 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:+1 to do the rewrite, just like for other similar ALTER TABLE commands
Ok. What about the following syntax:
ALTER COLUMN DROP COMPRESSION - removes compression from the column
with the rewrite and removes related compression options, so the user
can drop compression method.ALTER COLUMN SET COMPRESSION NONE for the cases when
the users want to just disable compression for future tuples. After
that they can keep compressed tuples, or in the case when they have a
large table they can decompress tuples partially using e.g. UPDATE,
and then use ALTER COLUMN DROP COMPRESSION which will be much faster
then.ALTER COLUMN SET COMPRESSION <cm> WITH <cmoptions> will change
compression for new tuples but will not touch old ones. If the users
want the recompression they can use DROP/SET COMPRESSION combination.I don't think that SET COMPRESSION with the rewrite of the whole table
will be useful enough on any somewhat big tables and same time big
tables is where the user needs compression the most.I understand that ALTER with the rewrite sounds logical and much easier
to implement (and it doesn't require Oids in tuples), but it could be
unusable.
The problem with this is that old compression methods can still be
floating around in the table even after you have done SET COMPRESSION
to something else. The table still needs to have a dependency on the
old compression method, because otherwise you might think it's safe to
drop the old one when it really is not. Furthermore, if you do a
pg_upgrade, you've got to preserve that dependency, which means it
would have to show up in a pg_dump --binary-upgrade someplace. It's
not obvious how any of that would work with this syntax.
Maybe a better idea is ALTER COLUMN SET COMPRESSION x1, x2, x3 ...
meaning that x1 is the default for new tuples but x2, x3, etc. are
still allowed if present. If you issue a command that only adds
things to the list, no table rewrite happens, but if you remove
anything, then it does.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Fri, 8 Dec 2017 15:12:42 -0500
Robert Haas <robertmhaas@gmail.com> wrote:
Maybe a better idea is ALTER COLUMN SET COMPRESSION x1, x2, x3 ...
meaning that x1 is the default for new tuples but x2, x3, etc. are
still allowed if present. If you issue a command that only adds
things to the list, no table rewrite happens, but if you remove
anything, then it does.
I like this idea, but maybe it should be something like ALTER COLUMN
SET COMPRESSION x1 [ PRESERVE x2, x3 ]? 'PRESERVE' is already used in
syntax and this syntax will show better which one is current and which
ones should be kept.
--
----
Regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
On Mon, Dec 11, 2017 at 7:55 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:
On Fri, 8 Dec 2017 15:12:42 -0500
Robert Haas <robertmhaas@gmail.com> wrote:Maybe a better idea is ALTER COLUMN SET COMPRESSION x1, x2, x3 ...
meaning that x1 is the default for new tuples but x2, x3, etc. are
still allowed if present. If you issue a command that only adds
things to the list, no table rewrite happens, but if you remove
anything, then it does.I like this idea, but maybe it should be something like ALTER COLUMN
SET COMPRESSION x1 [ PRESERVE x2, x3 ]? 'PRESERVE' is already used in
syntax and this syntax will show better which one is current and which
ones should be kept.
Sure, that works. And I think pglz should exist in the catalog as a
predefined, undroppable compression algorithm. So the default for
each column initially is:
SET COMPRESSION pglz
And if you want to rewrite the table with your awesome custom thing, you can do
SET COMPRESSION awesome
But if you want to just use the awesome custom thing for new rows, you can do
SET COMPRESSION awesome PRESERVE pglz
Then we can get all the dependencies right, pg_upgrade works, users
have total control of rewrite behavior, and everything is great. :-)
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Mon, Dec 11, 2017 at 8:25 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Dec 11, 2017 at 7:55 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:On Fri, 8 Dec 2017 15:12:42 -0500
Robert Haas <robertmhaas@gmail.com> wrote:Maybe a better idea is ALTER COLUMN SET COMPRESSION x1, x2, x3 ...
meaning that x1 is the default for new tuples but x2, x3, etc. are
still allowed if present. If you issue a command that only adds
things to the list, no table rewrite happens, but if you remove
anything, then it does.I like this idea, but maybe it should be something like ALTER COLUMN
SET COMPRESSION x1 [ PRESERVE x2, x3 ]? 'PRESERVE' is already used in
syntax and this syntax will show better which one is current and which
ones should be kept.Sure, that works. And I think pglz should exist in the catalog as a
predefined, undroppable compression algorithm. So the default for
each column initially is:SET COMPRESSION pglz
And if you want to rewrite the table with your awesome custom thing, you
can doSET COMPRESSION awesome
But if you want to just use the awesome custom thing for new rows, you can
doSET COMPRESSION awesome PRESERVE pglz
Then we can get all the dependencies right, pg_upgrade works, users
have total control of rewrite behavior, and everything is great. :-)
Looks good.
Thus, in your example if user would like to further change awesome
compression for evenbetter compression, she should write.
SET COMPRESSION evenbetter PRESERVE pglz, awesome; -- full list of previous
compression methods
I wonder what should we do if user specifies only part of previous
compression methods? For instance, pglz is specified but awesome is
missing.
SET COMPRESSION evenbetter PRESERVE pglz; -- awesome is missing
I think we should trigger an error in this case. Because query is
specified in form that is assuming to work without table rewrite, but we're
unable to do this without table rewrite.
I also think that we need some way to change compression method for
multiple columns in a single table rewrite. Because it would be way more
efficient than rewriting table for each of columns. So as an alternative of
ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome; -- first table
rewrite
ALTER TABLE tbl ALTER COLUMN c2 SET COMPRESSION awesome; -- second table
rewrite
we could also provide
ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome PRESERVE pglz; --
no rewrite
ALTER TABLE tbl ALTER COLUMN c2 SET COMPRESSION awesome PRESERVE pglz; --
no rewrite
VACUUM FULL tbl RESET COMPRESSION PRESERVE c1, c2; -- rewrite with
recompression of c1 and c2 and removing depedencies
?
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, Dec 11, 2017 at 12:41 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
Thus, in your example if user would like to further change awesome
compression for evenbetter compression, she should write.SET COMPRESSION evenbetter PRESERVE pglz, awesome; -- full list of previous
compression methods
Right.
I wonder what should we do if user specifies only part of previous
compression methods? For instance, pglz is specified but awesome is
missing.SET COMPRESSION evenbetter PRESERVE pglz; -- awesome is missing
I think we should trigger an error in this case. Because query is specified
in form that is assuming to work without table rewrite, but we're unable to
do this without table rewrite.
I think that should just rewrite the table in that case. PRESERVE
should specify the things that are allowed to be preserved -- its mere
presence should not be read to preclude a rewrite. And it's
completely reasonable for someone to want to do this, if they are
thinking about de-installing awesome.
I also think that we need some way to change compression method for multiple
columns in a single table rewrite. Because it would be way more efficient
than rewriting table for each of columns. So as an alternative ofALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome; -- first table
rewrite
ALTER TABLE tbl ALTER COLUMN c2 SET COMPRESSION awesome; -- second table
rewritewe could also provide
ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome PRESERVE pglz; -- no
rewrite
ALTER TABLE tbl ALTER COLUMN c2 SET COMPRESSION awesome PRESERVE pglz; -- no
rewrite
VACUUM FULL tbl RESET COMPRESSION PRESERVE c1, c2; -- rewrite with
recompression of c1 and c2 and removing depedencies?
Hmm. ALTER TABLE allows multi comma-separated subcommands, so I don't
think we need to drag VACUUM into this. The user can just say:
ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome, ALTER COLUMN
c2 SET COMPRESSION awesome;
If this is properly integrated into tablecmds.c, that should cause a
single rewrite affecting both columns.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Mon, Dec 11, 2017 at 8:46 PM, Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Dec 11, 2017 at 12:41 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:Thus, in your example if user would like to further change awesome
compression for evenbetter compression, she should write.SET COMPRESSION evenbetter PRESERVE pglz, awesome; -- full list of
previous
compression methods
Right.
I wonder what should we do if user specifies only part of previous
compression methods? For instance, pglz is specified but awesome is
missing.SET COMPRESSION evenbetter PRESERVE pglz; -- awesome is missing
I think we should trigger an error in this case. Because query is
specified
in form that is assuming to work without table rewrite, but we're unable
to
do this without table rewrite.
I think that should just rewrite the table in that case. PRESERVE
should specify the things that are allowed to be preserved -- its mere
presence should not be read to preclude a rewrite. And it's
completely reasonable for someone to want to do this, if they are
thinking about de-installing awesome.
OK, but NOTICE that presumably unexpected table rewrite takes place could
be still useful.
Also we probably should add some view that will expose compression methods
whose are currently preserved for columns. So that user can correctly
construct SET COMPRESSION query that doesn't rewrites table without digging
into internals (like directly querying pg_depend).
I also think that we need some way to change compression method for
multiplecolumns in a single table rewrite. Because it would be way more
efficient
than rewriting table for each of columns. So as an alternative of
ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome; -- first table
rewrite
ALTER TABLE tbl ALTER COLUMN c2 SET COMPRESSION awesome; -- second table
rewritewe could also provide
ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome PRESERVE pglz;
-- no
rewrite
ALTER TABLE tbl ALTER COLUMN c2 SET COMPRESSION awesome PRESERVE pglz;-- no
rewrite
VACUUM FULL tbl RESET COMPRESSION PRESERVE c1, c2; -- rewrite with
recompression of c1 and c2 and removing depedencies?
Hmm. ALTER TABLE allows multi comma-separated subcommands, so I don't
think we need to drag VACUUM into this. The user can just say:ALTER TABLE tbl ALTER COLUMN c1 SET COMPRESSION awesome, ALTER COLUMN
c2 SET COMPRESSION awesome;If this is properly integrated into tablecmds.c, that should cause a
single rewrite affecting both columns.
OK. Sorry, I didn't notice we can use multiple subcommands for ALTER TABLE
in this case...
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
Hi,
I see there's an ongoing discussion about the syntax and ALTER TABLE
behavior when changing a compression method for a column. So the patch
seems to be on the way to be ready in the January CF, I guess.
But let me play the devil's advocate for a while and question the
usefulness of this approach to compression. Some of the questions were
mentioned in the thread before, but I don't think they got the attention
they deserve.
FWIW I don't know the answers, but I think it's important to ask them.
Also, apologies if this post looks to be against the patch - that's part
of the "devil's advocate" thing.
The main question I'm asking myself is what use cases the patch
addresses, and whether there is a better way to do that. I see about
three main use-cases:
1) Replacing the algorithm used to compress all varlena types (in a way
that makes it transparent for the data type code).
2) Custom datatype-aware compression (e.g. the tsvector).
3) Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
Now, let's discuss those use cases one by one, and see if there are
simpler (or better in some way) solutions ...
Replacing the algorithm used to compress all varlena values (in a way
that makes it transparent for the data type code).
----------------------------------------------------------------------
While pglz served us well over time, it was repeatedly mentioned that in
some cases it becomes the bottleneck. So supporting other state of the
art compression algorithms seems like a good idea, and this patch is one
way to do that.
But perhaps we should simply make it an initdb option (in which case the
whole cluster would simply use e.g. lz4 instead of pglz)?
That seems like a much simpler approach - it would only require some
./configure options to add --with-lz4 (and other compression libraries),
an initdb option to pick compression algorithm, and probably noting the
choice in cluster controldata.
No dependencies tracking, no ALTER TABLE issues, etc.
Of course, it would not allow using different compression algorithms for
different columns (although it might perhaps allow different compression
level, to some extent).
Conclusion: If we want to offer a simple cluster-wide pglz alternative,
perhaps this patch is not the right way to do that.
Custom datatype-aware compression (e.g. the tsvector)
----------------------------------------------------------------------
Exploiting knowledge of the internal data type structure is a promising
way to improve compression ratio and/or performance.
The obvious question of course is why shouldn't this be done by the data
type code directly, which would also allow additional benefits like
operating directly on the compressed values.
Another thing is that if the datatype representation changes in some
way, the compression method has to change too. So it's tightly coupled
to the datatype anyway.
This does not really require any new infrastructure, all the pieces are
already there.
In some cases that may not be quite possible - the datatype may not be
flexible enough to support alternative (compressed) representation, e.g.
because there are no bits available for "compressed" flag, etc.
Conclusion: IMHO if we want to exploit the knowledge of the data type
internal structure, perhaps doing that in the datatype code directly
would be a better choice.
Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
----------------------------------------------------------------------
Exploiting redundancy in multiple values in the same column (instead of
compressing them independently) is another attractive way to help the
compression. It is inherently datatype-aware, but currently can't be
implemented directly in datatype code as there's no concept of
column-specific storage (e.g. to store dictionary shared by all values
in a particular column).
I believe any patch addressing this use case would have to introduce
such column-specific storage, and any solution doing that would probably
need to introduce the same catalogs, etc.
The obvious disadvantage of course is that we need to decompress the
varlena value before doing pretty much anything with it, because the
datatype is not aware of the compression.
So I wonder if the patch should instead provide infrastructure for doing
that in the datatype code directly.
The other question is if the patch should introduce some infrastructure
for handling the column context (e.g. column dictionary). Right now,
whoever implements the compression has to implement this bit too.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, 11 Dec 2017 20:53:29 +0100
Tomas Vondra <tomas.vondra@2ndquadrant.com> wrote:
But let me play the devil's advocate for a while and question the
usefulness of this approach to compression. Some of the questions were
mentioned in the thread before, but I don't think they got the
attention they deserve.
Hi. I will try to explain why this approach could be better than others.
Replacing the algorithm used to compress all varlena values (in a way
that makes it transparent for the data type code).
----------------------------------------------------------------------While pglz served us well over time, it was repeatedly mentioned that
in some cases it becomes the bottleneck. So supporting other state of
the art compression algorithms seems like a good idea, and this patch
is one way to do that.But perhaps we should simply make it an initdb option (in which case
the whole cluster would simply use e.g. lz4 instead of pglz)?That seems like a much simpler approach - it would only require some
./configure options to add --with-lz4 (and other compression
libraries), an initdb option to pick compression algorithm, and
probably noting the choice in cluster controldata.
Replacing pglz for all varlena values wasn't the goal of the patch, but
it's possible to do with it and I think that's good. And as Robert
mentioned pglz could appear as builtin undroppable compresssion method
so the others could be added too. And in the future it can open the
ways to specify compression for specific database or cluster.
Custom datatype-aware compression (e.g. the tsvector)
----------------------------------------------------------------------Exploiting knowledge of the internal data type structure is a
promising way to improve compression ratio and/or performance.The obvious question of course is why shouldn't this be done by the
data type code directly, which would also allow additional benefits
like operating directly on the compressed values.Another thing is that if the datatype representation changes in some
way, the compression method has to change too. So it's tightly coupled
to the datatype anyway.This does not really require any new infrastructure, all the pieces
are already there.In some cases that may not be quite possible - the datatype may not be
flexible enough to support alternative (compressed) representation,
e.g. because there are no bits available for "compressed" flag, etc.Conclusion: IMHO if we want to exploit the knowledge of the data type
internal structure, perhaps doing that in the datatype code directly
would be a better choice.
It could be, but let's imagine there will be internal compression for
tsvector. It means that tsvector has two formats now and minus one bit
somewhere in the header. After a while we found a better compression
but we can't add it because there is already one and it's not good to
have three different formats for one type. Or, the compression methods
were implemented and we decided to use dictionaries for tsvector (if
the user going to store limited number of words). But it will mean that
tsvector will go two compression stages (for its internal and for
dictionaries).
Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
----------------------------------------------------------------------Exploiting redundancy in multiple values in the same column (instead
of compressing them independently) is another attractive way to help
the compression. It is inherently datatype-aware, but currently can't
be implemented directly in datatype code as there's no concept of
column-specific storage (e.g. to store dictionary shared by all values
in a particular column).I believe any patch addressing this use case would have to introduce
such column-specific storage, and any solution doing that would
probably need to introduce the same catalogs, etc.The obvious disadvantage of course is that we need to decompress the
varlena value before doing pretty much anything with it, because the
datatype is not aware of the compression.So I wonder if the patch should instead provide infrastructure for
doing that in the datatype code directly.The other question is if the patch should introduce some
infrastructure for handling the column context (e.g. column
dictionary). Right now, whoever implements the compression has to
implement this bit too.
Column specific storage sounds optional to me. For example compressing
timestamp[] using some delta compression will not require it.
--
----
Regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
Hi!
Let me add my two cents too.
On Tue, Dec 12, 2017 at 2:41 PM, Ildus Kurbangaliev <
i.kurbangaliev@postgrespro.ru> wrote:
On Mon, 11 Dec 2017 20:53:29 +0100 Tomas Vondra <
tomas.vondra@2ndquadrant.com> wrote:Replacing the algorithm used to compress all varlena values (in a way
that makes it transparent for the data type code).
----------------------------------------------------------------------While pglz served us well over time, it was repeatedly mentioned that
in some cases it becomes the bottleneck. So supporting other state of
the art compression algorithms seems like a good idea, and this patch
is one way to do that.But perhaps we should simply make it an initdb option (in which case
the whole cluster would simply use e.g. lz4 instead of pglz)?That seems like a much simpler approach - it would only require some
./configure options to add --with-lz4 (and other compression
libraries), an initdb option to pick compression algorithm, and
probably noting the choice in cluster controldata.Replacing pglz for all varlena values wasn't the goal of the patch, but
it's possible to do with it and I think that's good. And as Robert
mentioned pglz could appear as builtin undroppable compresssion method
so the others could be added too. And in the future it can open the
ways to specify compression for specific database or cluster.
Yes, usage of custom compression methods to replace generic non
type-specific compression method is really not the primary goal of this
patch. However, I would consider that as useful side effect. However,
even in this case I see some advantages of custom compression methods over
initdb option.
1) In order to support alternative compression methods in initdb, we have
to provide builtin support for them. Then we immediately run into
dependencies/incompatible-licenses problem. Also, we tie appearance of new
compression methods to our release cycle. In real life, flexibility means
a lot. Giving users freedom to experiment with various compression
libraries without having to recompile PostgreSQL core is great.
2) It's not necessary that users would be satisfied with applying single
compression method to the whole database cluster. Various columns may have
different data distributions with different workloads. Optimal compression
type for one column is not necessary optimal for another column.
3) Possibility to change compression method on the fly without re-initdb is
very good too.
Custom datatype-aware compression (e.g. the tsvector)
----------------------------------------------------------------------
Exploiting knowledge of the internal data type structure is a
promising way to improve compression ratio and/or performance.The obvious question of course is why shouldn't this be done by the
data type code directly, which would also allow additional benefits
like operating directly on the compressed values.Another thing is that if the datatype representation changes in some
way, the compression method has to change too. So it's tightly coupled
to the datatype anyway.This does not really require any new infrastructure, all the pieces
are already there.In some cases that may not be quite possible - the datatype may not be
flexible enough to support alternative (compressed) representation,
e.g. because there are no bits available for "compressed" flag, etc.Conclusion: IMHO if we want to exploit the knowledge of the data type
internal structure, perhaps doing that in the datatype code directly
would be a better choice.It could be, but let's imagine there will be internal compression for
tsvector. It means that tsvector has two formats now and minus one bit
somewhere in the header. After a while we found a better compression
but we can't add it because there is already one and it's not good to
have three different formats for one type. Or, the compression methods
were implemented and we decided to use dictionaries for tsvector (if
the user going to store limited number of words). But it will mean that
tsvector will go two compression stages (for its internal and for
dictionaries).
I would like to add that even for single datatype various compression
methods may have different tradeoffs. For instance, one compression method
can have better compression ratio, but another one have faster
decompression. And it's OK for user to choose different compression
methods for different columns.
Depending extensions on datatype internal representation doesn't seem evil
for me. We already have bunch of extension depending on much more deeper
guts of PostgreSQL. On major release of PostgreSQL, extensions must adopt
the changes, that is the rule. And note, the datatype internal
representation alters relatively rare in comparison with other internals,
because it's related to on-disk format and ability to pg_upgrade.
Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
----------------------------------------------------------------------Exploiting redundancy in multiple values in the same column (instead
of compressing them independently) is another attractive way to help
the compression. It is inherently datatype-aware, but currently can't
be implemented directly in datatype code as there's no concept of
column-specific storage (e.g. to store dictionary shared by all values
in a particular column).I believe any patch addressing this use case would have to introduce
such column-specific storage, and any solution doing that would
probably need to introduce the same catalogs, etc.The obvious disadvantage of course is that we need to decompress the
varlena value before doing pretty much anything with it, because the
datatype is not aware of the compression.So I wonder if the patch should instead provide infrastructure for
doing that in the datatype code directly.The other question is if the patch should introduce some
infrastructure for handling the column context (e.g. column
dictionary). Right now, whoever implements the compression has to
implement this bit too.Column specific storage sounds optional to me. For example compressing
timestamp[] using some delta compression will not require it.
It's also could be useful to have custom compression method with fixed (not
dynamically complemented) dictionary. See [1] for example what other
databases do. We may specify fixed dictionary directly in the compression
method options, I see no problems. We may also compress that way not only
jsonb or other special data types, but also natural language texts. Using
fixed dictionaries for natural language we can effectively compress short
texts, when lz and other generic compression methods don't have enough of
information to effectively train per-value dictionary.
For sure, further work to improve infrastructure is required including
per-column storage for dictionary and tighter integration between
compression method and datatype. However, we are typically deal with such
complex tasks in step-by-step approach. And I'm not convinced that custom
compression methods are bad for the first step in this direction. For me
they look clear and already very useful in this shape.
1.
https://www.percona.com/doc/percona-server/LATEST/flexibility/compressed_columns.html
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Tue, Dec 12, 2017 at 6:07 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
Hi!
Let me add my two cents too.
On Tue, Dec 12, 2017 at 2:41 PM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:On Mon, 11 Dec 2017 20:53:29 +0100 Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Replacing the algorithm used to compress all varlena values (in a way
that makes it transparent for the data type code).
----------------------------------------------------------------------While pglz served us well over time, it was repeatedly mentioned that
in some cases it becomes the bottleneck. So supporting other state of
the art compression algorithms seems like a good idea, and this patch
is one way to do that.But perhaps we should simply make it an initdb option (in which case
the whole cluster would simply use e.g. lz4 instead of pglz)?That seems like a much simpler approach - it would only require some
./configure options to add --with-lz4 (and other compression
libraries), an initdb option to pick compression algorithm, and
probably noting the choice in cluster controldata.Replacing pglz for all varlena values wasn't the goal of the patch, but
it's possible to do with it and I think that's good. And as Robert
mentioned pglz could appear as builtin undroppable compresssion method
so the others could be added too. And in the future it can open the
ways to specify compression for specific database or cluster.Yes, usage of custom compression methods to replace generic non
type-specific compression method is really not the primary goal of this
patch. However, I would consider that as useful side effect. However, even
in this case I see some advantages of custom compression methods over initdb
option.1) In order to support alternative compression methods in initdb, we have to
provide builtin support for them. Then we immediately run into
dependencies/incompatible-licenses problem. Also, we tie appearance of new
compression methods to our release cycle. In real life, flexibility means a
lot. Giving users freedom to experiment with various compression libraries
without having to recompile PostgreSQL core is great.
2) It's not necessary that users would be satisfied with applying single
compression method to the whole database cluster. Various columns may have
different data distributions with different workloads. Optimal compression
type for one column is not necessary optimal for another column.
3) Possibility to change compression method on the fly without re-initdb is
very good too.
I consider custom compression as the way to custom TOAST. For example,
to optimal access
parts of very long document we need to compress slices of document.
Currently, long jsonb
document get compressed and then sliced and that killed all benefits
of binary jsonb. Also, we are
thinking about "lazy" access to the parts of jsonb from pl's, which is
currently awfully unefficient.
Custom datatype-aware compression (e.g. the tsvector)
----------------------------------------------------------------------Exploiting knowledge of the internal data type structure is a
promising way to improve compression ratio and/or performance.The obvious question of course is why shouldn't this be done by the
data type code directly, which would also allow additional benefits
like operating directly on the compressed values.Another thing is that if the datatype representation changes in some
way, the compression method has to change too. So it's tightly coupled
to the datatype anyway.This does not really require any new infrastructure, all the pieces
are already there.In some cases that may not be quite possible - the datatype may not be
flexible enough to support alternative (compressed) representation,
e.g. because there are no bits available for "compressed" flag, etc.Conclusion: IMHO if we want to exploit the knowledge of the data type
internal structure, perhaps doing that in the datatype code directly
would be a better choice.It could be, but let's imagine there will be internal compression for
tsvector. It means that tsvector has two formats now and minus one bit
somewhere in the header. After a while we found a better compression
but we can't add it because there is already one and it's not good to
have three different formats for one type. Or, the compression methods
were implemented and we decided to use dictionaries for tsvector (if
the user going to store limited number of words). But it will mean that
tsvector will go two compression stages (for its internal and for
dictionaries).I would like to add that even for single datatype various compression
methods may have different tradeoffs. For instance, one compression method
can have better compression ratio, but another one have faster
decompression. And it's OK for user to choose different compression methods
for different columns.Depending extensions on datatype internal representation doesn't seem evil
for me. We already have bunch of extension depending on much more deeper
guts of PostgreSQL. On major release of PostgreSQL, extensions must adopt
the changes, that is the rule. And note, the datatype internal
representation alters relatively rare in comparison with other internals,
because it's related to on-disk format and ability to pg_upgrade.Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
----------------------------------------------------------------------Exploiting redundancy in multiple values in the same column (instead
of compressing them independently) is another attractive way to help
the compression. It is inherently datatype-aware, but currently can't
be implemented directly in datatype code as there's no concept of
column-specific storage (e.g. to store dictionary shared by all values
in a particular column).I believe any patch addressing this use case would have to introduce
such column-specific storage, and any solution doing that would
probably need to introduce the same catalogs, etc.The obvious disadvantage of course is that we need to decompress the
varlena value before doing pretty much anything with it, because the
datatype is not aware of the compression.So I wonder if the patch should instead provide infrastructure for
doing that in the datatype code directly.The other question is if the patch should introduce some
infrastructure for handling the column context (e.g. column
dictionary). Right now, whoever implements the compression has to
implement this bit too.Column specific storage sounds optional to me. For example compressing
timestamp[] using some delta compression will not require it.It's also could be useful to have custom compression method with fixed (not
dynamically complemented) dictionary. See [1] for example what other
databases do. We may specify fixed dictionary directly in the compression
method options, I see no problems. We may also compress that way not only
jsonb or other special data types, but also natural language texts. Using
fixed dictionaries for natural language we can effectively compress short
texts, when lz and other generic compression methods don't have enough of
information to effectively train per-value dictionary.For sure, further work to improve infrastructure is required including
per-column storage for dictionary and tighter integration between
compression method and datatype. However, we are typically deal with such
complex tasks in step-by-step approach. And I'm not convinced that custom
compression methods are bad for the first step in this direction. For me
they look clear and already very useful in this shape.
+1
Show quoted text
1.
https://www.percona.com/doc/percona-server/LATEST/flexibility/compressed_columns.html------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, Dec 11, 2017 at 1:06 PM, Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
OK, but NOTICE that presumably unexpected table rewrite takes place could be
still useful.
I'm not going to complain too much about that, but I think that's
mostly a failure of expectation rather than a real problem. If the
documentation says what the user should expect, and they expect
something else, tough luck for them.
Also we probably should add some view that will expose compression methods
whose are currently preserved for columns. So that user can correctly
construct SET COMPRESSION query that doesn't rewrites table without digging
into internals (like directly querying pg_depend).
Yes. I wonder if \d or \d+ can show it somehow.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Mon, Dec 11, 2017 at 2:53 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
But let me play the devil's advocate for a while and question the
usefulness of this approach to compression. Some of the questions were
mentioned in the thread before, but I don't think they got the attention
they deserve.
Sure, thanks for chiming in. I think it is good to make sure we are
discussing this stuff.
But perhaps we should simply make it an initdb option (in which case the
whole cluster would simply use e.g. lz4 instead of pglz)?That seems like a much simpler approach - it would only require some
./configure options to add --with-lz4 (and other compression libraries),
an initdb option to pick compression algorithm, and probably noting the
choice in cluster controldata.No dependencies tracking, no ALTER TABLE issues, etc.
Of course, it would not allow using different compression algorithms for
different columns (although it might perhaps allow different compression
level, to some extent).Conclusion: If we want to offer a simple cluster-wide pglz alternative,
perhaps this patch is not the right way to do that.
I actually disagree with your conclusion here. I mean, if you do it
that way, then it has the same problem as checksums: changing
compression algorithms requires a full dump-and-reload of the
database, which makes it more or less a non-starter for large
databases. On the other hand, with the infrastructure provided by
this patch, we can have a default_compression_method GUC that will be
set to 'pglz' initially. If the user changes it to 'lz4', or we ship
a new release where the new default is 'lz4', then new tables created
will use that new setting, but the existing stuff keeps working. If
you want to upgrade your existing tables to use lz4 rather than pglz,
you can change the compression option for those columns to COMPRESS
lz4 PRESERVE pglz if you want to do it incrementally or just COMPRESS
lz4 to force a rewrite of an individual table. That's really
powerful, and I think users will like it a lot.
In short, your approach, while perhaps a little simpler to code, seems
like it is fraught with operational problems which this design avoids.
Custom datatype-aware compression (e.g. the tsvector)
----------------------------------------------------------------------Exploiting knowledge of the internal data type structure is a promising
way to improve compression ratio and/or performance.The obvious question of course is why shouldn't this be done by the data
type code directly, which would also allow additional benefits like
operating directly on the compressed values.Another thing is that if the datatype representation changes in some
way, the compression method has to change too. So it's tightly coupled
to the datatype anyway.This does not really require any new infrastructure, all the pieces are
already there.In some cases that may not be quite possible - the datatype may not be
flexible enough to support alternative (compressed) representation, e.g.
because there are no bits available for "compressed" flag, etc.Conclusion: IMHO if we want to exploit the knowledge of the data type
internal structure, perhaps doing that in the datatype code directly
would be a better choice.
I definitely think there's a place for compression built right into
the data type. I'm still happy about commit
145343534c153d1e6c3cff1fa1855787684d9a38 -- although really, more
needs to be done there. But that type of improvement and what is
proposed here are basically orthogonal. Having either one is good;
having both is better.
I think there may also be a place for declaring that a particular data
type has a "privileged" type of TOAST compression; if you use that
kind of compression for that data type, the data type will do smart
things, and if not, it will have to decompress in more cases. But I
think this infrastructure makes that kind of thing easier, not harder.
Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
----------------------------------------------------------------------Exploiting redundancy in multiple values in the same column (instead of
compressing them independently) is another attractive way to help the
compression. It is inherently datatype-aware, but currently can't be
implemented directly in datatype code as there's no concept of
column-specific storage (e.g. to store dictionary shared by all values
in a particular column).I believe any patch addressing this use case would have to introduce
such column-specific storage, and any solution doing that would probably
need to introduce the same catalogs, etc.The obvious disadvantage of course is that we need to decompress the
varlena value before doing pretty much anything with it, because the
datatype is not aware of the compression.So I wonder if the patch should instead provide infrastructure for doing
that in the datatype code directly.The other question is if the patch should introduce some infrastructure
for handling the column context (e.g. column dictionary). Right now,
whoever implements the compression has to implement this bit too.
I agree that having a place to store a per-column compression
dictionary would be awesome, but I think that could be added later on
top of this infrastructure. For example, suppose we stored each
per-column compression dictionary in a separate file and provided some
infrastructure for WAL-logging changes to the file on a logical basis
and checkpointing those updates. Then we wouldn't be tied to the
MVCC/transactional issues which storing the blobs in a table would
have, which seems like a big win. Of course, it also creates a lot of
little tiny files inside a directory that already tends to have too
many files, but maybe with some more work we can figure out a way
around that problem. Here again, it seems to me that the proposed
design is going more in the right direction than the wrong direction:
if some day we have per-column dictionaries, they will need to be tied
to specific compression methods on specific columns. If we already
have that concept, extending it to do something new is easier than if
we have to create it from scratch.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On 12/12/2017 10:33 PM, Robert Haas wrote:
On Mon, Dec 11, 2017 at 2:53 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:But let me play the devil's advocate for a while and question the
usefulness of this approach to compression. Some of the questions were
mentioned in the thread before, but I don't think they got the attention
they deserve.Sure, thanks for chiming in. I think it is good to make sure we are
discussing this stuff.But perhaps we should simply make it an initdb option (in which case the
whole cluster would simply use e.g. lz4 instead of pglz)?That seems like a much simpler approach - it would only require some
./configure options to add --with-lz4 (and other compression libraries),
an initdb option to pick compression algorithm, and probably noting the
choice in cluster controldata.No dependencies tracking, no ALTER TABLE issues, etc.
Of course, it would not allow using different compression algorithms for
different columns (although it might perhaps allow different compression
level, to some extent).Conclusion: If we want to offer a simple cluster-wide pglz alternative,
perhaps this patch is not the right way to do that.I actually disagree with your conclusion here. I mean, if you do it
that way, then it has the same problem as checksums: changing
compression algorithms requires a full dump-and-reload of the
database, which makes it more or less a non-starter for large
databases. On the other hand, with the infrastructure provided by
this patch, we can have a default_compression_method GUC that will be
set to 'pglz' initially. If the user changes it to 'lz4', or we ship
a new release where the new default is 'lz4', then new tables created
will use that new setting, but the existing stuff keeps working. If
you want to upgrade your existing tables to use lz4 rather than pglz,
you can change the compression option for those columns to COMPRESS
lz4 PRESERVE pglz if you want to do it incrementally or just COMPRESS
lz4 to force a rewrite of an individual table. That's really
powerful, and I think users will like it a lot.In short, your approach, while perhaps a little simpler to code, seems
like it is fraught with operational problems which this design avoids.
I agree the checksum-like limitations are annoying and make it
impossible to change the compression algorithm after the cluster is
initialized (although I recall a discussion about addressing that).
So yeah, if such flexibility is considered valuable/important, then the
patch is a better solution.
Custom datatype-aware compression (e.g. the tsvector)
----------------------------------------------------------------------Exploiting knowledge of the internal data type structure is a promising
way to improve compression ratio and/or performance.The obvious question of course is why shouldn't this be done by the data
type code directly, which would also allow additional benefits like
operating directly on the compressed values.Another thing is that if the datatype representation changes in some
way, the compression method has to change too. So it's tightly coupled
to the datatype anyway.This does not really require any new infrastructure, all the pieces are
already there.In some cases that may not be quite possible - the datatype may not be
flexible enough to support alternative (compressed) representation, e.g.
because there are no bits available for "compressed" flag, etc.Conclusion: IMHO if we want to exploit the knowledge of the data type
internal structure, perhaps doing that in the datatype code directly
would be a better choice.I definitely think there's a place for compression built right into
the data type. I'm still happy about commit
145343534c153d1e6c3cff1fa1855787684d9a38 -- although really, more
needs to be done there. But that type of improvement and what is
proposed here are basically orthogonal. Having either one is good;
having both is better.
Why orthogonal?
For example, why couldn't (or shouldn't) the tsvector compression be
done by tsvector code itself? Why should we be doing that at the varlena
level (so that the tsvector code does not even know about it)?
For example we could make the datatype EXTERNAL and do the compression
on our own, using a custom algorithm. Of course, that would require
datatype-specific implementation, but tsvector_compress does that too.
It seems to me the main reason is that tsvector actually does not allow
us to do that, as there's no good way to distinguish the different
internal format (e.g. by storing a flag or format version in some sort
of header, etc.).
I think there may also be a place for declaring that a particular data
type has a "privileged" type of TOAST compression; if you use that
kind of compression for that data type, the data type will do smart
things, and if not, it will have to decompress in more cases. But I
think this infrastructure makes that kind of thing easier, not harder.
I don't quite understand how that would be done. Isn't TOAST meant to be
entirely transparent for the datatypes? I can imagine custom TOAST
compression (which is pretty much what the patch does, after all), but I
don't see how the datatype could do anything smart about it, because it
has no idea which particular compression was used. And considering the
OIDs of the compression methods do change, I'm not sure that's fixable.
Custom datatype-aware compression with additional column-specific
metadata (e.g. the jsonb with external dictionary).
----------------------------------------------------------------------Exploiting redundancy in multiple values in the same column (instead of
compressing them independently) is another attractive way to help the
compression. It is inherently datatype-aware, but currently can't be
implemented directly in datatype code as there's no concept of
column-specific storage (e.g. to store dictionary shared by all values
in a particular column).I believe any patch addressing this use case would have to introduce
such column-specific storage, and any solution doing that would probably
need to introduce the same catalogs, etc.The obvious disadvantage of course is that we need to decompress the
varlena value before doing pretty much anything with it, because the
datatype is not aware of the compression.So I wonder if the patch should instead provide infrastructure for doing
that in the datatype code directly.The other question is if the patch should introduce some infrastructure
for handling the column context (e.g. column dictionary). Right now,
whoever implements the compression has to implement this bit too.I agree that having a place to store a per-column compression
dictionary would be awesome, but I think that could be added later on
top of this infrastructure. For example, suppose we stored each
per-column compression dictionary in a separate file and provided some
infrastructure for WAL-logging changes to the file on a logical basis
and checkpointing those updates. Then we wouldn't be tied to the
MVCC/transactional issues which storing the blobs in a table would
have, which seems like a big win. Of course, it also creates a lot of
little tiny files inside a directory that already tends to have too
many files, but maybe with some more work we can figure out a way
around that problem. Here again, it seems to me that the proposed
design is going more in the right direction than the wrong direction:
if some day we have per-column dictionaries, they will need to be tied
to specific compression methods on specific columns. If we already
have that concept, extending it to do something new is easier than if
we have to create it from scratch.
Well, it wasn't my goal to suddenly widen the scope of the patch and
require it adds all these pieces. My intent was more to point to pieces
that need to be filled in the future.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 12/12/2017 04:33 PM, Robert Haas wrote:
you want to upgrade your existing tables to use lz4 rather than pglz,
you can change the compression option for those columns to COMPRESS
lz4 PRESERVE pglz if you want to do it incrementally or just COMPRESS
This is a thread I've only been following peripherally, so forgive
a question that's probably covered somewhere upthread: how will this
be done? Surely not with compression-type bits in each tuple? By
remembering a txid where the compression was changed, and the former
algorithm for older txids?
-Chap
Re: [HACKERS] Custom compression methods
On Tue, Dec 12, 2017 at 5:07 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
I definitely think there's a place for compression built right into
the data type. I'm still happy about commit
145343534c153d1e6c3cff1fa1855787684d9a38 -- although really, more
needs to be done there. But that type of improvement and what is
proposed here are basically orthogonal. Having either one is good;
having both is better.Why orthogonal?
I mean, they are different things. Data types are already free to
invent more compact representations, and that does not preclude
applying pglz to the result.
For example, why couldn't (or shouldn't) the tsvector compression be
done by tsvector code itself? Why should we be doing that at the varlena
level (so that the tsvector code does not even know about it)?
We could do that, but then:
1. The compression algorithm would be hard-coded into the system
rather than changeable. Pluggability has some value.
2. If several data types can benefit from a similar approach, it has
to be separately implemented for each one.
3. Compression is only applied to large-ish values. If you are just
making the data type representation more compact, you probably want to
apply the new representation to all values. If you are compressing in
the sense that the original data gets smaller but harder to interpret,
then you probably only want to apply the technique where the value is
already pretty wide, and maybe respect the user's configured storage
attributes. TOAST knows about some of that kind of stuff.
It seems to me the main reason is that tsvector actually does not allow
us to do that, as there's no good way to distinguish the different
internal format (e.g. by storing a flag or format version in some sort
of header, etc.).
That is also a potential problem, although I suspect it is possible to
work around it somehow for most data types. It might be annoying,
though.
I think there may also be a place for declaring that a particular data
type has a "privileged" type of TOAST compression; if you use that
kind of compression for that data type, the data type will do smart
things, and if not, it will have to decompress in more cases. But I
think this infrastructure makes that kind of thing easier, not harder.I don't quite understand how that would be done. Isn't TOAST meant to be
entirely transparent for the datatypes? I can imagine custom TOAST
compression (which is pretty much what the patch does, after all), but I
don't see how the datatype could do anything smart about it, because it
has no idea which particular compression was used. And considering the
OIDs of the compression methods do change, I'm not sure that's fixable.
I don't think TOAST needs to be entirely transparent for the
datatypes. We've already dipped our toe in the water by allowing some
operations on "short" varlenas, and there's really nothing to prevent
a given datatype from going further. The OID problem you mentioned
would presumably be solved by hard-coding the OIDs for any built-in,
privileged compression methods.
Well, it wasn't my goal to suddenly widen the scope of the patch and
require it adds all these pieces. My intent was more to point to pieces
that need to be filled in the future.
Sure, that's fine. I'm not worked up about this, just explaining why
it seems reasonably well-designed to me.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On 12/13/2017 01:54 AM, Robert Haas wrote:
On Tue, Dec 12, 2017 at 5:07 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:I definitely think there's a place for compression built right into
the data type. I'm still happy about commit
145343534c153d1e6c3cff1fa1855787684d9a38 -- although really, more
needs to be done there. But that type of improvement and what is
proposed here are basically orthogonal. Having either one is good;
having both is better.Why orthogonal?
I mean, they are different things. Data types are already free to
invent more compact representations, and that does not preclude
applying pglz to the result.For example, why couldn't (or shouldn't) the tsvector compression be
done by tsvector code itself? Why should we be doing that at the varlena
level (so that the tsvector code does not even know about it)?We could do that, but then:
1. The compression algorithm would be hard-coded into the system
rather than changeable. Pluggability has some value.
Sure. I agree extensibility of pretty much all parts is a significant
asset of the project.
2. If several data types can benefit from a similar approach, it has
to be separately implemented for each one.
I don't think the current solution improves that, though. If you want to
exploit internal features of individual data types, it pretty much
requires code customized to every such data type.
For example you can't take the tsvector compression and just slap it on
tsquery, because it relies on knowledge of internal tsvector structure.
So you need separate implementations anyway.
3. Compression is only applied to large-ish values. If you are just
making the data type representation more compact, you probably want to
apply the new representation to all values. If you are compressing in
the sense that the original data gets smaller but harder to interpret,
then you probably only want to apply the technique where the value is
already pretty wide, and maybe respect the user's configured storage
attributes. TOAST knows about some of that kind of stuff.
Good point. One such parameter that I really miss is compression level.
I can imagine tuning it through CREATE COMPRESSION METHOD, but it does
not seem quite possible with compression happening in a datatype.
It seems to me the main reason is that tsvector actually does not allow
us to do that, as there's no good way to distinguish the different
internal format (e.g. by storing a flag or format version in some sort
of header, etc.).That is also a potential problem, although I suspect it is possible to
work around it somehow for most data types. It might be annoying,
though.I think there may also be a place for declaring that a particular data
type has a "privileged" type of TOAST compression; if you use that
kind of compression for that data type, the data type will do smart
things, and if not, it will have to decompress in more cases. But I
think this infrastructure makes that kind of thing easier, not harder.I don't quite understand how that would be done. Isn't TOAST meant to be
entirely transparent for the datatypes? I can imagine custom TOAST
compression (which is pretty much what the patch does, after all), but I
don't see how the datatype could do anything smart about it, because it
has no idea which particular compression was used. And considering the
OIDs of the compression methods do change, I'm not sure that's fixable.I don't think TOAST needs to be entirely transparent for the
datatypes. We've already dipped our toe in the water by allowing some
operations on "short" varlenas, and there's really nothing to prevent
a given datatype from going further. The OID problem you mentioned
would presumably be solved by hard-coding the OIDs for any built-in,
privileged compression methods.
Stupid question, but what do you mean by "short" varlenas?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Tue, 12 Dec 2017 15:52:01 -0500
Robert Haas <robertmhaas@gmail.com> wrote:
Yes. I wonder if \d or \d+ can show it somehow.
Yes, in current version of the patch, \d+ shows current compression.
It can be extended to show a list of current compression methods.
Since we agreed on ALTER syntax, i want to clear things about CREATE.
Should it be CREATE ACCESS METHOD .. TYPE СOMPRESSION or CREATE
COMPRESSION METHOD? I like the access method approach, and it
simplifies the code, but I'm just not sure a compression is an access
method or not.
Current implementation
----------------------
To avoid extra patches I also want to clear things about current
implementation. Right now there are two tables, "pg_compression" and
"pg_compression_opt". When compression method is linked to a column it
creates a record in pg_compression_opt. This record's Oid is stored in
the varlena. These Oids kept in first column so I can move them in
pg_upgrade but in all other aspects they behave like usual Oids. Also
it's easy to restore them.
Compression options linked to a specific column. When tuple is
moved between relations it will be decompressed.
Also in current implementation SET COMPRESSION contains WITH syntax
which is used to provide extra options to compression method.
What could be changed
---------------------
As Alvaro mentioned COMPRESSION METHOD is practically an access method,
so it could be created as CREATE ACCESS METHOD .. TYPE COMPRESSION.
This approach simplifies the patch and "pg_compression" table could be
removed. So compression method is created with something like:
CREATE ACCESS METHOD .. TYPE COMPRESSION HANDLER
awesome_compression_handler;
Syntax of SET COMPRESSION changes to SET COMPRESSION .. PRESERVE which
is useful to control rewrites and for pg_upgrade to make dependencies
between moved compression options and compression methods from pg_am
table.
Default compression is always pglz and if users want to change they run:
ALTER COLUMN <col> SET COMPRESSION awesome PRESERVE pglz;
Without PRESERVE it will rewrite the whole relation using new
compression. Also the rewrite removes all unlisted compression options
so their compresssion methods could be safely dropped.
"pg_compression_opt" table could be renamed to "pg_compression", and
compression options will be stored there.
I'd like to keep extra compression options, for example pglz can be
configured with them. Syntax would be slightly changed:
SET COMPRESSION pglz WITH (min_comp_rate=25) PRESERVE awesome;
Setting the same compression method with different options will create
new compression options record for future tuples but will not
rewrite table.
--
----
Regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
Tomas Vondra wrote:
On 12/13/2017 01:54 AM, Robert Haas wrote:
3. Compression is only applied to large-ish values. If you are just
making the data type representation more compact, you probably want to
apply the new representation to all values. If you are compressing in
the sense that the original data gets smaller but harder to interpret,
then you probably only want to apply the technique where the value is
already pretty wide, and maybe respect the user's configured storage
attributes. TOAST knows about some of that kind of stuff.Good point. One such parameter that I really miss is compression level.
I can imagine tuning it through CREATE COMPRESSION METHOD, but it does
not seem quite possible with compression happening in a datatype.
Hmm, actually isn't that the sort of thing that you would tweak using a
column-level option instead of a compression method?
ALTER TABLE ALTER COLUMN SET (compression_level=123)
The only thing we need for this is to make tuptoaster.c aware of the
need to check for a parameter.
I don't think TOAST needs to be entirely transparent for the
datatypes. We've already dipped our toe in the water by allowing some
operations on "short" varlenas, and there's really nothing to prevent
a given datatype from going further. The OID problem you mentioned
would presumably be solved by hard-coding the OIDs for any built-in,
privileged compression methods.Stupid question, but what do you mean by "short" varlenas?
Those are varlenas with 1-byte header rather than the standard 4-byte
header.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On 12/13/2017 05:55 PM, Alvaro Herrera wrote:
Tomas Vondra wrote:
On 12/13/2017 01:54 AM, Robert Haas wrote:
3. Compression is only applied to large-ish values. If you are just
making the data type representation more compact, you probably want to
apply the new representation to all values. If you are compressing in
the sense that the original data gets smaller but harder to interpret,
then you probably only want to apply the technique where the value is
already pretty wide, and maybe respect the user's configured storage
attributes. TOAST knows about some of that kind of stuff.Good point. One such parameter that I really miss is compression level.
I can imagine tuning it through CREATE COMPRESSION METHOD, but it does
not seem quite possible with compression happening in a datatype.Hmm, actually isn't that the sort of thing that you would tweak using a
column-level option instead of a compression method?
ALTER TABLE ALTER COLUMN SET (compression_level=123)
The only thing we need for this is to make tuptoaster.c aware of the
need to check for a parameter.
Wouldn't that require some universal compression level, shared by all
supported compression algorithms? I don't think there is such thing.
Defining it should not be extremely difficult, although I'm sure there
will be some cumbersome cases. For example what if an algorithm "a"
supports compression levels 0-10, and algorithm "b" only supports 0-3?
You may define 11 "universal" compression levels, and map the four
levels for "b" to that (how). But then everyone has to understand how
that "universal" mapping is defined.
Another issue is that there are algorithms without a compression level
(e.g. pglz does not have one, AFAICS), or with somewhat definition (lz4
does not have levels, and instead has "acceleration" which may be an
arbitrary positive integer, so not really compatible with "universal"
compression level).
So to me the
ALTER TABLE ALTER COLUMN SET (compression_level=123)
seems more like an unnecessary hurdle ...
I don't think TOAST needs to be entirely transparent for the
datatypes. We've already dipped our toe in the water by allowing some
operations on "short" varlenas, and there's really nothing to prevent
a given datatype from going further. The OID problem you mentioned
would presumably be solved by hard-coding the OIDs for any built-in,
privileged compression methods.Stupid question, but what do you mean by "short" varlenas?
Those are varlenas with 1-byte header rather than the standard 4-byte
header.
OK, that's what I thought. But that is still pretty transparent to the
data types, no?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Wed, Dec 13, 2017 at 5:10 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
2. If several data types can benefit from a similar approach, it has
to be separately implemented for each one.I don't think the current solution improves that, though. If you want to
exploit internal features of individual data types, it pretty much
requires code customized to every such data type.For example you can't take the tsvector compression and just slap it on
tsquery, because it relies on knowledge of internal tsvector structure.
So you need separate implementations anyway.
I don't think that's necessarily true. Certainly, it's true that *if*
tsvector compression depends on knowledge of internal tsvector
structure, *then* that you can't use the implementation for anything
else (this, by the way, means that there needs to be some way for a
compression method to reject being applied to a column of a data type
it doesn't like). However, it seems possible to imagine compression
algorithms that can work for a variety of data types, too. There
might be a compression algorithm that is theoretically a
general-purpose algorithm but has features which are particularly
well-suited to, say, JSON or XML data, because it looks for word
boundaries to decide on what strings to insert into the compression
dictionary.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Wed, Dec 13, 2017 at 1:34 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
Wouldn't that require some universal compression level, shared by all
supported compression algorithms? I don't think there is such thing.Defining it should not be extremely difficult, although I'm sure there
will be some cumbersome cases. For example what if an algorithm "a"
supports compression levels 0-10, and algorithm "b" only supports 0-3?You may define 11 "universal" compression levels, and map the four
levels for "b" to that (how). But then everyone has to understand how
that "universal" mapping is defined.
What we could do is use the "namespace" feature of reloptions to
distinguish options for the column itself from options for the
compression algorithm. Currently namespaces are used only to allow
you to configure toast.whatever = somevalue, but we could let you say
pglz.something = somevalue or lz4.whatever = somevalue. Or maybe, to
avoid confusion -- what happens if somebody invents a compression
method called toast? -- we should do it as compress.lz4.whatever =
somevalue. I think this takes us a bit far afield from the core
purpose of this patch and should be a separate patch at a later time,
but I think it would be cool.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Wed, Dec 13, 2017 at 7:18 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:
Since we agreed on ALTER syntax, i want to clear things about CREATE.
Should it be CREATE ACCESS METHOD .. TYPE СOMPRESSION or CREATE
COMPRESSION METHOD? I like the access method approach, and it
simplifies the code, but I'm just not sure a compression is an access
method or not.
+1 for ACCESS METHOD.
Current implementation
----------------------To avoid extra patches I also want to clear things about current
implementation. Right now there are two tables, "pg_compression" and
"pg_compression_opt". When compression method is linked to a column it
creates a record in pg_compression_opt. This record's Oid is stored in
the varlena. These Oids kept in first column so I can move them in
pg_upgrade but in all other aspects they behave like usual Oids. Also
it's easy to restore them.
pg_compression_opt -> pg_attr_compression, maybe.
Compression options linked to a specific column. When tuple is
moved between relations it will be decompressed.
Can we do this only if the compression method isn't OK for the new
column? For example, if the old column is COMPRESS foo PRESERVE bar
and the new column is COMPRESS bar PRESERVE foo, we don't need to
force decompression in any case.
Also in current implementation SET COMPRESSION contains WITH syntax
which is used to provide extra options to compression method.
Hmm, that's an alternative to use reloptions. Maybe that's fine.
What could be changed
---------------------As Alvaro mentioned COMPRESSION METHOD is practically an access method,
so it could be created as CREATE ACCESS METHOD .. TYPE COMPRESSION.
This approach simplifies the patch and "pg_compression" table could be
removed. So compression method is created with something like:CREATE ACCESS METHOD .. TYPE COMPRESSION HANDLER
awesome_compression_handler;Syntax of SET COMPRESSION changes to SET COMPRESSION .. PRESERVE which
is useful to control rewrites and for pg_upgrade to make dependencies
between moved compression options and compression methods from pg_am
table.Default compression is always pglz and if users want to change they run:
ALTER COLUMN <col> SET COMPRESSION awesome PRESERVE pglz;
Without PRESERVE it will rewrite the whole relation using new
compression. Also the rewrite removes all unlisted compression options
so their compresssion methods could be safely dropped.
That all sounds good.
"pg_compression_opt" table could be renamed to "pg_compression", and
compression options will be stored there.
See notes above.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On 12/14/2017 04:21 PM, Robert Haas wrote:
On Wed, Dec 13, 2017 at 5:10 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:2. If several data types can benefit from a similar approach, it has
to be separately implemented for each one.I don't think the current solution improves that, though. If you
want to exploit internal features of individual data types, it
pretty much requires code customized to every such data type.For example you can't take the tsvector compression and just slap
it on tsquery, because it relies on knowledge of internal tsvector
structure. So you need separate implementations anyway.I don't think that's necessarily true. Certainly, it's true that
*if* tsvector compression depends on knowledge of internal tsvector
structure, *then* that you can't use the implementation for anything
else (this, by the way, means that there needs to be some way for a
compression method to reject being applied to a column of a data
type it doesn't like).
I believe such dependency (on implementation details) is pretty much the
main benefit of datatype-aware compression methods. If you don't rely on
such assumption, then I'd say it's a general-purpose compression method.
However, it seems possible to imagine compression algorithms that can
work for a variety of data types, too. There might be a compression
algorithm that is theoretically a general-purpose algorithm but has
features which are particularly well-suited to, say, JSON or XML
data, because it looks for word boundaries to decide on what strings
to insert into the compression dictionary.
Can you give an example of such algorithm? Because I haven't seen such
example, and I find arguments based on hypothetical compression methods
somewhat suspicious.
FWIW I'm not against considering such compression methods, but OTOH it
may not be such a great primary use case to drive the overall design.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Thu, Dec 14, 2017 at 12:23 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
Can you give an example of such algorithm? Because I haven't seen such
example, and I find arguments based on hypothetical compression methods
somewhat suspicious.FWIW I'm not against considering such compression methods, but OTOH it
may not be such a great primary use case to drive the overall design.
Well it isn't, really. I am honestly not sure what we're arguing
about at this point. I think you've agreed that (1) opening avenues
for extensibility is useful, (2) substitution a general-purpose
compression algorithm could be useful, and (3) having datatype
compression that is enabled through TOAST rather than built into the
datatype might sometimes be desirable. That's more than adequate
justification for this proposal, whether half-general compression
methods exist or not. I am prepared to concede that there may be no
useful examples of such a thing.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Thu, 14 Dec 2017 10:29:10 -0500
Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Dec 13, 2017 at 7:18 AM, Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:Since we agreed on ALTER syntax, i want to clear things about
CREATE. Should it be CREATE ACCESS METHOD .. TYPE СOMPRESSION or
CREATE COMPRESSION METHOD? I like the access method approach, and it
simplifies the code, but I'm just not sure a compression is an
access method or not.+1 for ACCESS METHOD.
An access method then.
Current implementation
----------------------To avoid extra patches I also want to clear things about current
implementation. Right now there are two tables, "pg_compression" and
"pg_compression_opt". When compression method is linked to a column
it creates a record in pg_compression_opt. This record's Oid is
stored in the varlena. These Oids kept in first column so I can
move them in pg_upgrade but in all other aspects they behave like
usual Oids. Also it's easy to restore them.pg_compression_opt -> pg_attr_compression, maybe.
Compression options linked to a specific column. When tuple is
moved between relations it will be decompressed.Can we do this only if the compression method isn't OK for the new
column? For example, if the old column is COMPRESS foo PRESERVE bar
and the new column is COMPRESS bar PRESERVE foo, we don't need to
force decompression in any case.
Thanks, sounds right, i will add it to the patch.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 12/17/2017 04:32 AM, Robert Haas wrote:
On Thu, Dec 14, 2017 at 12:23 PM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Can you give an example of such algorithm? Because I haven't seen such
example, and I find arguments based on hypothetical compression methods
somewhat suspicious.FWIW I'm not against considering such compression methods, but OTOH it
may not be such a great primary use case to drive the overall design.Well it isn't, really. I am honestly not sure what we're arguing
about at this point. I think you've agreed that (1) opening avenues
for extensibility is useful, (2) substitution a general-purpose
compression algorithm could be useful, and (3) having datatype
compression that is enabled through TOAST rather than built into the
datatype might sometimes be desirable. That's more than adequate
justification for this proposal, whether half-general compression
methods exist or not. I am prepared to concede that there may be no
useful examples of such a thing.
I don't think we're arguing - we're discussing if a proposed patch is
the right design solving relevant use cases.
I personally am not quite convinced about that, for the reason I tried
to explain in my previous messages. I see it as a poor alternative to
compression built into the data type. I do like the idea of compression
with external dictionary, however.
But don't forget that it's not me in this thread - it's my evil twin,
moonlighting as Mr. Devil's lawyer ;-)
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Dec 18, 2017 at 10:43 AM, Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
I personally am not quite convinced about that, for the reason I tried
to explain in my previous messages. I see it as a poor alternative to
compression built into the data type. I do like the idea of compression
with external dictionary, however.
I think that compression built into the datatype and what is proposed
here are both useful and everybody's free to work on either one as the
prefer, so I don't see that as a reason not to accept this patch. And
I think this patch can be a stepping stone toward compression with an
external dictionary, so that seems like an affirmative reason to
accept this patch.
But don't forget that it's not me in this thread - it's my evil twin,
moonlighting as Mr. Devil's lawyer ;-)
Well, I don't mind you objecting to the patch under any persona, but
so far I'm not finding your reasons convincing...
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
Attached a new version of the patch. Main changes:
* compression as an access method
* pglz as default compression access method.
* PRESERVE syntax for tables rewrite control.
* pg_upgrade fixes
* support partitioned tables.
* more tests.
Regards,
Ildus Kurbangaliev
Attachments:
custom_compression_methods_v8.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3f02202caf..b3f60abfc7 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..4be8509903
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,63 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ An compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a72c50eadb..b357b09aaf 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -89,6 +89,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 041afdbd86..686e8db2b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&gist;
&spgist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 7bcf242846..8a6c89c480 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +321,23 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index a0c9a6d257..35147fb3f2 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..3e3f4f5c0e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 425bc5d06e..fa94f4fa00 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -945,11 +945,111 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_defelem(const void *a, const void *b)
+{
+ DefElem *da = *(DefElem **) a;
+ DefElem *db = *(DefElem **) b;
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ {
+ int i = 0;
+ DefElem **items = palloc(len * sizeof(DefElem *));
+
+ foreach(cell, options)
+ items[i++] = lfirst(cell);
+
+ qsort(items, len, sizeof(DefElem *), compare_defelem);
+ for (i = 0; i < len; i++)
+ resoptions = lappend(resoptions, items[i]);
+ pfree(items);
+ }
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..a051943149
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ PGLZ_strategy_default);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d94530766c..bdecedce29 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2333,13 +2334,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2426,7 +2428,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2595,7 +2597,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2656,8 +2658,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2696,7 +2702,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -3987,6 +3993,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4089,7 +4097,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..ad0e8514f6 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,57 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+/*
+ * Marks varlena as custom compressed. Notice that this macro should be called
+ * after TOAST_COMPRESS_SET_RAWSIZE because it will clean flags.
+ */
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +128,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +469,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +559,92 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,49 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1658,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1679,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1691,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2051,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2236,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2432,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2553,119 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/*
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ {
+ if (entry->amroutine)
+ {
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+ }
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+ }
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+ result->amroutine = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 80860128fb..8bf0a3f68a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index fac80612b8..e76567e8b8 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3398,6 +3398,8 @@ static const char *const no_priv_msg[MAX_ACL_KIND] =
gettext_noop("permission denied for publication %s"),
/* ACL_KIND_SUBSCRIPTION */
gettext_noop("permission denied for subscription %s"),
+ /* ACL_KIND_COMPRESSION_METHOD */
+ gettext_noop("permission denied for compression method %s"),
};
static const char *const not_owner_msg[MAX_ACL_KIND] =
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 269111b4c1..a9149e32ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1271,6 +1273,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2512,6 +2518,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 089b7965f2..93ada40910 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -31,6 +31,7 @@
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -628,6 +629,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1597,6 +1599,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 330488b96f..f91e5af181 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -393,6 +393,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -471,6 +472,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index bc999ca3c4..55743ac53f 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
ACL_KIND_STATISTICS,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2408,6 +2421,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3405,6 +3419,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3926,6 +3971,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4469,6 +4518,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 3995c5ef3d..59fea3395f 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 6bfca2a4af..e6876a7bdd 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2531,7 +2531,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
@@ -2763,8 +2763,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8455138ed3..87dc1dbfe7 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 44f3da9b51..c8b78bfab8 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f2a928b823..7134b7060e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -363,6 +369,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -459,6 +467,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static void ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -504,6 +514,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -678,6 +689,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -723,6 +736,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -770,6 +790,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1609,6 +1646,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -1943,6 +1981,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -1970,6 +2021,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2179,6 +2231,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3278,6 +3337,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3753,6 +3813,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4117,6 +4178,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_DetachPartition:
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4178,8 +4244,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4408,7 +4475,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4679,6 +4746,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5337,6 +5422,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5475,6 +5570,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5597,6 +5693,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5619,6 +5739,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6449,6 +6669,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8497,6 +8721,43 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 256, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9251,6 +9512,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9351,7 +9618,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9376,6 +9644,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9384,6 +9680,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12425,6 +12726,84 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static void
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index baf1e8fedf..5217b02757 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2808,6 +2808,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2826,6 +2827,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5490,6 +5503,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 80168de5d0..76558f74c2 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2554,6 +2554,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2572,6 +2573,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3628,6 +3639,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 5e72df137e..2704e6f688 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2801,6 +2801,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2817,6 +2818,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4108,6 +4119,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e42b7caff6..506e59f95f 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -310,6 +310,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -397,6 +398,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +584,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +620,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2183,6 +2189,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3347,11 +3362,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3361,8 +3377,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3409,6 +3425,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5279,12 +5332,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14941,6 +14999,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4220e24148..dd5654c1af 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1016,6 +1016,11 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/*
* Add to column list
*/
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1aa7c50c4..8b676f5c39 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2850,7 +2850,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index ec98a612ec..5d77f9ed85 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1013,6 +1013,7 @@ ProcessUtilitySlow(ParseState *pstate,
RELKIND_RELATION,
InvalidOid, NULL,
queryString);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 64e02ef434..a25b0ff0e5 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,13 @@
#include "postgres.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 00ba33bfb4..2351fa2ee4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -560,6 +561,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 041cd53a30..ae6801b869 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/common.c b/src/bin/pg_dump/common.c
index 7f5f351486..6a943b15fa 100644
--- a/src/bin/pg_dump/common.c
+++ b/src/bin/pg_dump/common.c
@@ -93,6 +93,7 @@ getSchemaData(Archive *fout, int *numTablesPtr)
NamespaceInfo *nspinfo;
ExtensionInfo *extinfo;
InhInfo *inhinfo;
+
int numAggregates;
int numInherits;
int numRules;
@@ -827,7 +828,6 @@ findExtensionByOid(Oid oid)
return (ExtensionInfo *) findObjectByOid(oid, extinfoindex, numExtensions);
}
-
/*
* setExtensionMembership
* accept and save data about which objects belong to extensions
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..0c5d97c455 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +173,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 41741aefbc..e54319c5c8 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 27628a397c..742fae5046 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -361,6 +363,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -581,6 +584,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -785,6 +791,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -3957,6 +3966,236 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -7895,6 +8134,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -7930,7 +8171,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -7949,7 +8229,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7975,7 +8257,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -7999,7 +8283,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8017,7 +8303,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8034,7 +8322,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8064,6 +8354,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8080,6 +8372,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8107,6 +8401,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9596,6 +9892,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12439,6 +12738,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15465,6 +15767,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15527,6 +15837,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17792,6 +18121,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 49a02b4fa8..0a48658ecf 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -83,7 +83,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -315,6 +316,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -611,6 +614,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -711,5 +736,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 6da1c35a42..2f7db02354 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -45,7 +45,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -80,7 +80,8 @@ static const int dbObjectTypePriority[] =
34, /* DO_POLICY */
35, /* DO_PUBLICATION */
36, /* DO_PUBLICATION_REL */
- 37 /* DO_SUBSCRIPTION */
+ 37, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1436,6 +1437,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 3dd2c3871e..be475c81f3 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -405,6 +407,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -628,6 +632,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 7cf9bdadb2..f1c33ca64d 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2611,6 +2667,39 @@ qr/^\QINSERT INTO test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUE
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4663,9 +4752,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4742,7 +4831,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4949,7 +5038,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4989,7 +5078,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5026,6 +5115,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index d2787ab41b..9da3aa87b9 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1718,6 +1718,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1832,6 +1848,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_MATVIEW ||
@@ -1927,6 +1948,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1934,7 +1961,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1945,7 +1972,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 0afe280586..051be21b65 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1986,11 +1986,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..982ac58684 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contain custom compressed
+ * varlenas */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 94739f7ac6..ce65e0f441 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 6f290d5c6f..8cdc241e47 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -165,10 +165,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..f25000b63b 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 3424, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 3424
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 3423, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 3423
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..f43d77267d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 3422 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 3422
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..de839faf47
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 3420
+
+CATALOG(pg_attr_compression,3420) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * we're not using system Oids.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 3422 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 3422
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index e7049438eb..3ae9c5fe56 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 298e0ae2f0..8f878c11c6 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -578,6 +578,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3809,6 +3813,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f f t f
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 3419 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3881,6 +3887,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..0b6693a715 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 5c4e492b89..b8b3379ad9 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -141,6 +141,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -150,8 +151,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 8e4142391d..3027f661fe 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 2eb3d6d371..2b1cc855a6 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -500,7 +501,8 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
- T_ForeignKeyCacheInfo /* in utils/rel.h */
+ T_ForeignKeyCacheInfo, /* in utils/rel.h */
+ T_CompressionAmRoutine, /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 86ac4fb9d9..fbaa4f862c 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +652,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1771,7 +1786,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 26af944e03..1f2fe78c55 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index a7f5e0caea..f18c68f98b 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,7 +22,7 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b69f88aa5b..1d65717305 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/acl.h b/src/include/utils/acl.h
index 67c7b2d4ac..f39135a56a 100644
--- a/src/include/utils/acl.h
+++ b/src/include/utils/acl.h
@@ -210,6 +210,7 @@ typedef enum AclObjectKind
ACL_KIND_EXTENSION, /* pg_extension */
ACL_KIND_PUBLICATION, /* pg_publication */
ACL_KIND_SUBSCRIPTION, /* pg_subscription */
+ ACL_KIND_COMPRESSION_METHOD, /* pg_compression */
MAX_ACL_KIND /* MUST BE LAST */
} AclObjectKind;
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 55d573c687..4f3314ec9f 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 65e9c626b3..112f0eda47 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..e511c7ccc3
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,326 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column f1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column f1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8e745402ae..394fe64d57 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -431,10 +431,10 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (((a + 1)))
Number of partitions: 0
@@ -592,10 +592,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -731,11 +731,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -744,11 +744,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])))
Partition key: RANGE (b)
@@ -758,11 +758,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -795,46 +795,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -854,11 +854,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..4ad0181e69 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index d2c184f2cf..481db55504 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index a79f891da7..9991897052 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 684f7f20a8..ab8661b73d 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1717,7 +1717,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index b101331d69..3b070f518c 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index b8dcf51a30..dbaa7e2f8c 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f1c1b44d6f..e358ff539c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index b69ceaa75e..1512d9919c 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -221,11 +221,11 @@ update range_parted set b = b + 1 where b = 10;
-- Creating default partition for range
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'a'::text) AND (b >= 10) AND (b < 20)) OR ((a = 'b'::text) AND (b >= 1) AND (b < 10)) OR ((a = 'b'::text) AND (b >= 10) AND (b < 20)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e224977791..6ff0b6f875 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 9fc5f1a268..6173133a0e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..ed8264205b
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,156 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e8fdf8454d..4d7d20980c 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1160,7 +1160,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index cc84217dd9..e1b5e5c1d5 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -341,6 +343,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -365,6 +368,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Hello Ildus,
15/01/2018 00:49, Ildus Kurbangaliev пишет:
Attached a new version of the patch. Main changes:
* compression as an access method
* pglz as default compression access method.
* PRESERVE syntax for tables rewrite control.
* pg_upgrade fixes
* support partitioned tables.
* more tests.
You need to rebase to the latest master, there are some conflicts. I've
applied it to the three days old master to try it.
As I can see the documentation is not yet complete. For example, there
is no section for ALTER COLUMN ... SET COMPRESSION in ddl.sgml; and
section "Compression Access Method Functions" in compression-am.sgml
hasn't been finished.
I've implemented an extension [1]https://github.com/zilder/pg_lz4 to understand the way developer would
go to work with new infrastructure. And for me it seems clear. (Except
that it took me some effort to wrap my mind around varlena macros but it
is probably a different topic).
I noticed that you haven't cover 'cmdrop' in the regression tests and I
saw the previous discussion about it. Have you considered using event
triggers to handle the drop of column compression instead of 'cmdrop'
function? This way you would kill two birds with one stone: it still
provides sufficient infrastructure to catch those events (and it
something postgres already has for different kinds of ddl commands) and
it would be easier to test.
Thanks!
[1]: https://github.com/zilder/pg_lz4
--
Ildar Musin
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, 22 Jan 2018 23:26:31 +0300
Ildar Musin <i.musin@postgrespro.ru> wrote:
Thanks for review! Attached new version of the patch. Fixed few bugs,
added more documentation and rebased to current master.
You need to rebase to the latest master, there are some conflicts.
I've applied it to the three days old master to try it.
Done.
As I can see the documentation is not yet complete. For example, there
is no section for ALTER COLUMN ... SET COMPRESSION in ddl.sgml; and
section "Compression Access Method Functions" in compression-am.sgml
hasn't been finished.
Not sure about ddl.sgml, it contains more common things, but since
postgres contains only pglz by default there is not much to show.
I've implemented an extension [1] to understand the way developer
would go to work with new infrastructure. And for me it seems clear.
(Except that it took me some effort to wrap my mind around varlena
macros but it is probably a different topic).I noticed that you haven't cover 'cmdrop' in the regression tests and
I saw the previous discussion about it. Have you considered using
event triggers to handle the drop of column compression instead of
'cmdrop' function? This way you would kill two birds with one stone:
it still provides sufficient infrastructure to catch those events
(and it something postgres already has for different kinds of ddl
commands) and it would be easier to test.
I have added support for event triggers for ALTER SET COMPRESSION in
current version. Event trigger on ALTER can be used to replace cmdrop
function but it will be far from trivial. There is not easy way to
understand that's attribute compression is really dropping in the
command.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v9.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..60d9d1d515 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..abafa21b82
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,146 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ An compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression method method. Could
+ be used to check compability with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains local-backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores result of
+ <structfield>cminitstate</structfield> function. Useful to store the parsed
+ view of the compression options.
+ </para>
+
+ <para>
+ Note that in any system cache invalidation related with
+ <structname>pg_attr_compression</structname>
+ relation the options will be cleaned, and recreated on next calls of
+ compression functions.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is creating. Can
+ return a pointer to memory that will be passed between compression
+ functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a72c50eadb..b357b09aaf 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -89,6 +89,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 041afdbd86..686e8db2b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&gist;
&spgist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 286c7a8589..7740d2f2c7 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +321,23 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index a0c9a6d257..35147fb3f2 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..3e3f4f5c0e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 274f7aa8e9..b95e185bca 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -945,11 +945,111 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_defelem(const void *a, const void *b)
+{
+ DefElem *da = *(DefElem **) a;
+ DefElem *db = *(DefElem **) b;
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ {
+ int i = 0;
+ DefElem **items = palloc(len * sizeof(DefElem *));
+
+ foreach(cell, options)
+ items[i++] = lfirst(cell);
+
+ qsort(items, len, sizeof(DefElem *), compare_defelem);
+ for (i = 0; i < len; i++)
+ resoptions = lappend(resoptions, items[i]);
+ pfree(items);
+ }
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 09b5794c52..42798eaaa9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2336,13 +2337,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2429,7 +2431,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2598,7 +2600,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2659,8 +2661,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2699,7 +2705,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -3990,6 +3996,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4092,7 +4100,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..1a4905f470 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,57 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+/*
+ * Marks varlena as custom compressed. Notice that this macro should be called
+ * after TOAST_COMPRESS_SET_RAWSIZE because it will clean flags.
+ */
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +128,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +469,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +559,92 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,49 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1658,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1679,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1691,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2051,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2236,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2432,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2553,135 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/*
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 80860128fb..8bf0a3f68a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index a053361163..3b52a96c15 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2524,6 +2530,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 774c07b03a..dd9ad4cf91 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -31,6 +31,7 @@
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -629,6 +630,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1598,6 +1600,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 93eba7f30a..7cad8698ac 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -410,6 +410,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -488,6 +489,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index d838666915..f32bc6a2b2 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2409,6 +2422,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3416,6 +3430,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3938,6 +3983,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4482,6 +4531,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 04a24c6082..7f1616514b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2518,7 +2518,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
@@ -2737,8 +2737,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c5af7e4dd1..ebe71dcb41 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ce5dd1bdfb..560560994e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -517,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -691,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -736,6 +749,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -783,6 +803,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1673,6 +1710,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2007,6 +2045,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2034,6 +2085,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2243,6 +2295,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3329,6 +3388,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3804,6 +3864,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4178,6 +4239,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4243,8 +4309,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4473,7 +4540,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4744,6 +4811,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5405,6 +5490,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5543,6 +5638,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5665,6 +5761,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5687,6 +5807,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6521,6 +6741,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8570,6 +8794,43 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 256, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9325,6 +9586,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9425,7 +9692,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9450,6 +9718,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9458,6 +9754,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12486,6 +12787,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 147853871a..6d647b4e44 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2810,6 +2810,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2828,6 +2829,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5493,6 +5506,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 91f4e1294c..2cfbbd7945 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2556,6 +2556,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2574,6 +2575,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3630,6 +3641,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e0f4befd9f..ec88a8d7d1 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2805,6 +2805,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2821,6 +2822,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4112,6 +4123,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 459a227e57..f0a51654de 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -310,6 +310,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -397,6 +398,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +584,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +620,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2208,6 +2214,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3372,11 +3387,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3386,8 +3402,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3434,6 +3450,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -5304,12 +5357,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14968,6 +15026,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 9e9472a2a0..c00019a707 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1013,6 +1013,11 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->constraints = NIL;
def->location = -1;
+ if (OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/*
* Add to column list
*/
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1aa7c50c4..8b676f5c39 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2850,7 +2850,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 3abe7d6155..cebbf0b0e7 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1016,6 +1016,7 @@ ProcessUtilitySlow(ParseState *pstate,
RELKIND_RELATION,
InvalidOid, NULL,
queryString);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 64e02ef434..a25b0ff0e5 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,13 @@
#include "postgres.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c081b88b73..762c1140af 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -565,6 +566,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 041cd53a30..ae6801b869 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..0c5d97c455 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +173,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index ab009e6fe3..4e4ec4b335 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 8d2818faae..27cba8c905 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -364,6 +366,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -584,6 +587,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -788,6 +794,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4157,6 +4166,236 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8140,6 +8379,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8175,7 +8416,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8194,7 +8474,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8220,7 +8502,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8244,7 +8528,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8262,7 +8548,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8279,7 +8567,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8309,6 +8599,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8325,6 +8617,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8352,6 +8646,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9844,6 +10140,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12687,6 +12986,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15717,6 +16019,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15779,6 +16089,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -18080,6 +18409,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index c39b2b81cd..b2be14d392 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -622,6 +625,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -722,5 +747,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 2fd5a025af..224029bdd4 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -71,6 +71,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -131,6 +132,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -398,6 +400,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -610,6 +614,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 74730bfc65..64167099f5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2614,6 +2670,39 @@ qr/^\QINSERT INTO test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUE
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4666,9 +4755,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4745,7 +4834,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4952,7 +5041,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4992,7 +5081,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5029,6 +5118,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 466a78004b..fe4e9da2c3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1719,6 +1719,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1835,6 +1851,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1932,6 +1953,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1940,7 +1967,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1951,7 +1978,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index e68ae8db94..74c7197b2e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2009,11 +2009,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..982ac58684 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contain custom compressed
+ * varlenas */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 94739f7ac6..ce65e0f441 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..f25000b63b 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 3424, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 3424
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 3423, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 3423
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..f43d77267d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 3422 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 3422
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..de839faf47
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 3420
+
+CATALOG(pg_attr_compression,3420) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * we're not using system Oids.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 3422 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 3422
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..1098811f9f 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index f01648c961..66d93a2719 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -578,6 +578,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3809,6 +3813,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f f t f
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 3419 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3881,6 +3887,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..0b6693a715 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 91fbaa0e49..5467de3927 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -142,6 +142,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -151,8 +152,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 74b094a9c3..65bdc02db9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -501,7 +502,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4dfe86ed55..0f0f66ada6 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +652,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1771,7 +1786,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 26af944e03..1f2fe78c55 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index 64aa8234e5..6e687e3734 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,7 +22,7 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b69f88aa5b..1d65717305 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 55d573c687..4f3314ec9f 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..9ad909e524
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,339 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column f1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column f1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8e745402ae..394fe64d57 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -431,10 +431,10 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (((a + 1)))
Number of partitions: 0
@@ -592,10 +592,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -731,11 +731,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -744,11 +744,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])))
Partition key: RANGE (b)
@@ -758,11 +758,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -795,46 +795,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -854,11 +854,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..4ad0181e69 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index d2c184f2cf..481db55504 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index a79f891da7..9991897052 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 684f7f20a8..ab8661b73d 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1717,7 +1717,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5433944c6a..9aece8074b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..b704b65e83 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..bbba808f1e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..914d54f3aa
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,169 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e8fdf8454d..4d7d20980c 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1160,7 +1160,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a42ff9794a..de6489688b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -341,6 +343,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -365,6 +368,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Hello Ildus,
On 23.01.2018 16:04, Ildus Kurbangaliev wrote:
On Mon, 22 Jan 2018 23:26:31 +0300
Ildar Musin <i.musin@postgrespro.ru> wrote:Thanks for review! Attached new version of the patch. Fixed few bugs,
added more documentation and rebased to current master.You need to rebase to the latest master, there are some conflicts.
I've applied it to the three days old master to try it.Done.
As I can see the documentation is not yet complete. For example, there
is no section for ALTER COLUMN ... SET COMPRESSION in ddl.sgml; and
section "Compression Access Method Functions" in compression-am.sgml
hasn't been finished.Not sure about ddl.sgml, it contains more common things, but since
postgres contains only pglz by default there is not much to show.I've implemented an extension [1] to understand the way developer
would go to work with new infrastructure. And for me it seems clear.
(Except that it took me some effort to wrap my mind around varlena
macros but it is probably a different topic).I noticed that you haven't cover 'cmdrop' in the regression tests and
I saw the previous discussion about it. Have you considered using
event triggers to handle the drop of column compression instead of
'cmdrop' function? This way you would kill two birds with one stone:
it still provides sufficient infrastructure to catch those events
(and it something postgres already has for different kinds of ddl
commands) and it would be easier to test.I have added support for event triggers for ALTER SET COMPRESSION in
current version. Event trigger on ALTER can be used to replace cmdrop
function but it will be far from trivial. There is not easy way to
understand that's attribute compression is really dropping in the
command.
I've encountered unexpected behavior in command 'CREATE TABLE ... (LIKE
...)'. It seems that it copies compression settings of the table
attributes no matter which INCLUDING options are specified. E.g.
create table xxx(id serial, msg text compression pg_lz4);
alter table xxx alter column msg set storage external;
\d+ xxx
Table "public.xxx"
Column | Type | ... | Storage | Compression |
--------+---------+ ... +----------+-------------+
id | integer | ... | plain | |
msg | text | ... | external | pg_lz4 |
Now copy the table structure with "INCLUDING ALL":
create table yyy (like xxx including all);
\d+ yyy
Table "public.yyy"
Column | Type | ... | Storage | Compression |
--------+---------+ ... +----------+-------------+
id | integer | ... | plain | |
msg | text | ... | external | pg_lz4 |
And now copy without "INCLUDING ALL":
create table zzz (like xxx);
\d+ zzz
Table "public.zzz"
Column | Type | ... | Storage | Compression |
--------+---------+ ... +----------+-------------+
id | integer | ... | plain | |
msg | text | ... | extended | pg_lz4 |
As you see, compression option is copied anyway. I suggest adding new
INCLUDING COMPRESSION option to enable user to explicitly specify
whether they want or not to copy compression settings.
I found a few phrases in documentation that can be improved. But the
documentation should be checked by a native speaker.
In compression-am.sgml:
"an compression access method" -> "a compression access method"
"compression method method" -> "compression method"
"compability" -> "compatibility"
Probably "local-backend cached state" would be better to replace with
"per backend cached state"?
"Useful to store the parsed view of the compression options" -> "It
could be useful for example to cache compression options"
"and stores result of" -> "and stores the result of"
"Called when CompressionAmOptions is creating." -> "Called when
<structname>CompressionAmOptions</structname> is being initialized"
"Note that in any system cache invalidation related with
pg_attr_compression relation the options will be cleaned" -> "Note that
any <literal>pg_attr_compression</literal> relation invalidation will
cause all the cached <literal>acstate</literal> options cleared."
"Function used to ..." -> "Function is used to ..."
I think it would be nice to mention custom compression methods in
storage.sgml. At this moment it only mentions built-in pglz compression.
--
Ildar Musin
i.musin@postgrespro.ru
Re: [HACKERS] Custom compression methods
On Thu, 25 Jan 2018 16:03:20 +0300
Ildar Musin <i.musin@postgrespro.ru> wrote:
Thanks for review!
As you see, compression option is copied anyway. I suggest adding new
INCLUDING COMPRESSION option to enable user to explicitly specify
whether they want or not to copy compression settings.
Good catch, i missed INCLUDE options for LIKE command. Added INCLUDING
COMPRESSION as you suggested.
I found a few phrases in documentation that can be improved. But the
documentation should be checked by a native speaker.In compression-am.sgml:
"an compression access method" -> "a compression access method"
"compression method method" -> "compression method"
"compability" -> "compatibility"
Probably "local-backend cached state" would be better to replace with
"per backend cached state"?
"Useful to store the parsed view of the compression options" -> "It
could be useful for example to cache compression options"
"and stores result of" -> "and stores the result of"
"Called when CompressionAmOptions is creating." -> "Called when
<structname>CompressionAmOptions</structname> is being initialized""Note that in any system cache invalidation related with
pg_attr_compression relation the options will be cleaned" -> "Note
that any <literal>pg_attr_compression</literal> relation invalidation
will cause all the cached <literal>acstate</literal> options cleared."
"Function used to ..." -> "Function is used to ..."I think it would be nice to mention custom compression methods in
storage.sgml. At this moment it only mentions built-in pglz
compression.
I agree, the documentation would require a native speaker. Fixed the
lines you mentioned.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v10.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..60d9d1d515 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..b415aa6e4b
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a72c50eadb..b357b09aaf 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -89,6 +89,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 041afdbd86..686e8db2b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&gist;
&spgist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 286c7a8589..7740d2f2c7 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +321,23 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index a0c9a6d257..35147fb3f2 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index c0e548fa5b..59c381c572 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..3e3f4f5c0e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 274f7aa8e9..b95e185bca 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -945,11 +945,111 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_defelem(const void *a, const void *b)
+{
+ DefElem *da = *(DefElem **) a;
+ DefElem *db = *(DefElem **) b;
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ {
+ int i = 0;
+ DefElem **items = palloc(len * sizeof(DefElem *));
+
+ foreach(cell, options)
+ items[i++] = lfirst(cell);
+
+ qsort(items, len, sizeof(DefElem *), compare_defelem);
+ for (i = 0; i < len; i++)
+ resoptions = lappend(resoptions, items[i]);
+ pfree(items);
+ }
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 09b5794c52..42798eaaa9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2336,13 +2337,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2429,7 +2431,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2598,7 +2600,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2659,8 +2661,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2699,7 +2705,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -3990,6 +3996,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4092,7 +4100,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..1a4905f470 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,57 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+/*
+ * Marks varlena as custom compressed. Notice that this macro should be called
+ * after TOAST_COMPRESS_SET_RAWSIZE because it will clean flags.
+ */
+#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
+do { \
+ (((toast_compress_header *) (ptr))->info |= (1 << 31)); \
+ (((toast_compress_header *) (ptr))->info &= ~(1 << 30)); \
+} while (0)
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +128,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +469,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +559,92 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,49 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CUSTOM(tmp);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1658,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1679,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1691,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2051,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2236,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2432,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2553,135 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/*
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 80860128fb..8bf0a3f68a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index a053361163..3b52a96c15 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2524,6 +2530,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 774c07b03a..dd9ad4cf91 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -31,6 +31,7 @@
#include "access/htup_details.h"
#include "access/multixact.h"
+#include "access/reloptions.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "access/xact.h"
@@ -629,6 +630,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1598,6 +1600,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 93eba7f30a..7cad8698ac 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -410,6 +410,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -488,6 +489,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index d838666915..f32bc6a2b2 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2409,6 +2422,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3416,6 +3430,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3938,6 +3983,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4482,6 +4531,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 04a24c6082..7f1616514b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2518,7 +2518,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
@@ -2737,8 +2737,8 @@ CopyFrom(CopyState cstate)
List *recheckIndexes = NIL;
/* OK, store the tuple and create index entries for it */
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);
if (resultRelInfo->ri_NumIndices > 0)
recheckIndexes = ExecInsertIndexTuples(slot,
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c5af7e4dd1..ebe71dcb41 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ce5dd1bdfb..560560994e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -517,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -691,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -736,6 +749,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -783,6 +803,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1673,6 +1710,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2007,6 +2045,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2034,6 +2085,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2243,6 +2295,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3329,6 +3388,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3804,6 +3864,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4178,6 +4239,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4243,8 +4309,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4473,7 +4540,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4744,6 +4811,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5405,6 +5490,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5543,6 +5638,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5665,6 +5761,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5687,6 +5807,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6521,6 +6741,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8570,6 +8794,43 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 256, &ctl, HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9325,6 +9586,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9425,7 +9692,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9450,6 +9718,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9458,6 +9754,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12486,6 +12787,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ if (atttableform->attstorage != 'x' && atttableform->attstorage != 'm')
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("storage for \"%s\" should be MAIN or EXTENDED", column)));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 147853871a..6d647b4e44 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2810,6 +2810,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2828,6 +2829,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5493,6 +5506,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 91f4e1294c..2cfbbd7945 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2556,6 +2556,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2574,6 +2575,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3630,6 +3641,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e0f4befd9f..ec88a8d7d1 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2805,6 +2805,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2821,6 +2822,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4112,6 +4123,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 459a227e57..e9f4b53e2d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -310,6 +310,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -397,6 +398,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +584,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +620,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2208,6 +2214,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3372,11 +3387,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3386,8 +3402,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3434,6 +3450,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3638,6 +3691,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
| COMMENTS { $$ = CREATE_TABLE_LIKE_COMMENTS; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5304,12 +5358,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14968,6 +15027,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 9e9472a2a0..f911c35be1 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1078,6 +1078,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1aa7c50c4..8b676f5c39 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2850,7 +2850,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 3abe7d6155..cebbf0b0e7 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -1016,6 +1016,7 @@ ProcessUtilitySlow(ParseState *pstate,
RELKIND_RELATION,
InvalidOid, NULL,
queryString);
+
EventTriggerCollectSimpleCommand(address,
secondaryObject,
stmt);
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/adt/tsvector.c b/src/backend/utils/adt/tsvector.c
index 64e02ef434..a25b0ff0e5 100644
--- a/src/backend/utils/adt/tsvector.c
+++ b/src/backend/utils/adt/tsvector.c
@@ -14,11 +14,13 @@
#include "postgres.h"
+#include "catalog/pg_type.h"
#include "libpq/pqformat.h"
#include "tsearch/ts_locale.h"
#include "tsearch/ts_utils.h"
#include "utils/builtins.h"
#include "utils/memutils.h"
+#include "common/pg_lzcompress.h"
typedef struct
{
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c081b88b73..762c1140af 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -77,6 +77,7 @@
#include "storage/smgr.h"
#include "utils/array.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -565,6 +566,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 041cd53a30..ae6801b869 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ce3100f09d..0c5d97c455 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -77,6 +77,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -121,6 +122,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -149,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -170,6 +173,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index f55aa36c49..4fb3d8d2fb 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 29223078ae..88731b249b 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -364,6 +366,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -584,6 +587,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -788,6 +794,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4157,6 +4166,236 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8140,6 +8379,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8175,7 +8416,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8194,7 +8474,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8220,7 +8502,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8244,7 +8528,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8262,7 +8548,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8279,7 +8567,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8309,6 +8599,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8325,6 +8617,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8352,6 +8646,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9844,6 +10140,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12687,6 +12986,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15717,6 +16019,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15779,6 +16089,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -18080,6 +18409,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index c39b2b81cd..b2be14d392 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -622,6 +625,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -722,5 +747,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 2fd5a025af..224029bdd4 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -71,6 +71,7 @@ static int use_setsessauth = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -131,6 +132,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -398,6 +400,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -610,6 +614,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 860a211a3c..810403ec9c 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -74,6 +74,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -122,6 +123,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -361,6 +363,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 74730bfc65..64167099f5 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2614,6 +2670,39 @@ qr/^\QINSERT INTO test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUE
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4666,9 +4755,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4745,7 +4834,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4952,7 +5041,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4992,7 +5081,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5029,6 +5118,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 466a78004b..fe4e9da2c3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1719,6 +1719,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1835,6 +1851,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1932,6 +1953,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1940,7 +1967,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1951,7 +1978,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index e68ae8db94..74c7197b2e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2009,11 +2009,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..982ac58684 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contain custom compressed
+ * varlenas */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 94739f7ac6..ce65e0f441 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..f25000b63b 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 3424, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 3424
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 3423, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 3423
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..f43d77267d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 3422 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 3422
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..de839faf47
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 3420
+
+CATALOG(pg_attr_compression,3420) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * we're not using system Oids.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 3422 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 3422
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..1098811f9f 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index f01648c961..66d93a2719 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -578,6 +578,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3809,6 +3813,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f f t f
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 3419 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3881,6 +3887,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..0b6693a715 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 91fbaa0e49..5467de3927 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -142,6 +142,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -151,8 +152,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 74b094a9c3..65bdc02db9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -501,7 +502,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4dfe86ed55..825224e8ff 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +652,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -672,6 +687,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 3,
CREATE_TABLE_LIKE_STORAGE = 1 << 4,
CREATE_TABLE_LIKE_COMMENTS = 1 << 5,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 6,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1771,7 +1787,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 26af944e03..1f2fe78c55 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/parser/parse_utilcmd.h b/src/include/parser/parse_utilcmd.h
index 64aa8234e5..6e687e3734 100644
--- a/src/include/parser/parse_utilcmd.h
+++ b/src/include/parser/parse_utilcmd.h
@@ -22,7 +22,7 @@ extern List *transformAlterTableStmt(Oid relid, AlterTableStmt *stmt,
const char *queryString);
extern IndexStmt *transformIndexStmt(Oid relid, IndexStmt *stmt,
const char *queryString);
-extern void transformRuleStmt(RuleStmt *stmt, const char *queryString,
+void transformRuleStmt(RuleStmt *stmt, const char *queryString,
List **actions, Node **whereClause);
extern List *transformCreateSchemaStmt(CreateSchemaStmt *stmt);
extern PartitionBoundSpec *transformPartitionBound(ParseState *pstate, Relation parent,
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b69f88aa5b..1d65717305 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 55d573c687..4f3314ec9f 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..b773adda48
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,339 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column f1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column f1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 8e745402ae..394fe64d57 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -431,10 +431,10 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (((a + 1)))
Number of partitions: 0
@@ -592,10 +592,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -731,11 +731,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -744,11 +744,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])))
Partition key: RANGE (b)
@@ -758,11 +758,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -795,46 +795,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -854,11 +854,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..4ad0181e69 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index d2c184f2cf..481db55504 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index a79f891da7..9991897052 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 684f7f20a8..ab8661b73d 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1717,7 +1717,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5433944c6a..9aece8074b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..b704b65e83 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..bbba808f1e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d79f2a0863
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,169 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e8fdf8454d..4d7d20980c 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1160,7 +1160,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a42ff9794a..de6489688b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -341,6 +343,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -365,6 +368,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Hello Ildus,
I continue reviewing your patch. Here are some thoughts.
1. When I set column storage to EXTERNAL then I cannot set compression.
Seems reasonable:
create table test(id serial, msg text);
alter table test alter column msg set storage external;
alter table test alter column msg set compression pg_lz4;
ERROR: storage for "msg" should be MAIN or EXTENDED
But if I reorder commands then it's ok:
create table test(id serial, msg text);
alter table test alter column msg set compression pg_lz4;
alter table test alter column msg set storage external;
\d+ test
Table "public.test"
Column | Type | ... | Storage | Compression
--------+---------+ ... +----------+-------------
id | integer | ... | plain |
msg | text | ... | external | pg_lz4
So we could either allow user to set compression settings even when
storage is EXTERNAL but with warning or prohibit user to set compression
and external storage at the same time. The same thing is with setting
storage PLAIN.
2. I think TOAST_COMPRESS_SET_RAWSIZE macro could be rewritten like
following to prevent overwriting of higher bits of 'info':
((toast_compress_header *) (ptr))->info = \
((toast_compress_header *) (ptr))->info & ~RAWSIZEMASK | (len);
It maybe does not matter at the moment since it is only used once, but
it could save some efforts for other developers in future.
In TOAST_COMPRESS_SET_CUSTOM() instead of changing individual bits you
may do something like this:
#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
do { \
((toast_compress_header *) (ptr))->info = \
((toast_compress_header *) (ptr))->info & RAWSIZEMASK | ((uint32) 0x02
<< 30) \
} while (0)
Also it would be nice if bit flags were explained and maybe replaced by
a macro.
3. In AlteredTableInfo, BulkInsertStateData and some functions (eg
toast_insert_or_update) there is a hash table used to keep preserved
compression methods list per attribute. I think a simple array of List*
would be sufficient in this case.
4. In optionListToArray() you can use list_qsort() to sort options list
instead of converting it manually into array and then back to a list.
5. Redundunt #includes:
In heap.c:
#include "access/reloptions.h"
In tsvector.c:
#include "catalog/pg_type.h"
#include "common/pg_lzcompress.h"
In relcache.c:
#include "utils/datum.h"
6. Just a minor thing: no reason to change formatting in copy.c
- heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid,
- hi_options, bistate);
+ heap_insert(resultRelInfo->ri_RelationDesc, tuple,
+ mycid, hi_options, bistate);7. Also in utility.c the extra new line was added which isn't relevant
for this patch.
8. In parse_utilcmd.h the 'extern' keyword was removed from
transformRuleStmt declaration which doesn't make sense in this patch.
9. Comments. Again, they should be read by a native speaker. So just a
few suggestions:
toast_prepare_varlena() - comment needed
invalidate_amoptions_cache() - comment format doesn't match other
functions in the file
In htup_details.h:
/* tuple contain custom compressed
* varlenas */
should be "contains"
--
Ildar Musin
i.musin@postgrespro.ru
Re: [HACKERS] Custom compression methods
On Fri, 26 Jan 2018 19:07:28 +0300
Ildar Musin <i.musin@postgrespro.ru> wrote:
Hello Ildus,
I continue reviewing your patch. Here are some thoughts.
Thanks! Attached new version of the patch.
1. When I set column storage to EXTERNAL then I cannot set
compression. Seems reasonable:
create table test(id serial, msg text);
alter table test alter column msg set storage external;
alter table test alter column msg set compression pg_lz4;
ERROR: storage for "msg" should be MAIN or EXTENDED
Changed the behaviour, now it's ok to change storages in any
directions for toastable types. Also added protection from untoastable
types.
2. I think TOAST_COMPRESS_SET_RAWSIZE macro could be rewritten like
following to prevent overwriting of higher bits of 'info':((toast_compress_header *) (ptr))->info = \
((toast_compress_header *) (ptr))->info & ~RAWSIZEMASK |
(len);It maybe does not matter at the moment since it is only used once, but
it could save some efforts for other developers in future.
In TOAST_COMPRESS_SET_CUSTOM() instead of changing individual bits you
may do something like this:#define TOAST_COMPRESS_SET_CUSTOM(ptr) \
do { \
((toast_compress_header *) (ptr))->info = \
((toast_compress_header *) (ptr))->info & RAWSIZEMASK
| ((uint32) 0x02 << 30) \
} while (0)Also it would be nice if bit flags were explained and maybe replaced
by a macro.
I noticed that there is no need of TOAST_COMPRESS_SET_CUSTOM at all,
so I just removed it, TOAST_COMPRESS_SET_RAWSIZE will set needed flags.
3. In AlteredTableInfo, BulkInsertStateData and some functions (eg
toast_insert_or_update) there is a hash table used to keep preserved
compression methods list per attribute. I think a simple array of
List* would be sufficient in this case.
Not sure about that, it will just complicate things without sufficient
improvements. Also it would require the passing the length of the array
and require more memory for tables with large number of attributes. But,
I've made default size of the hash table smaller, since unlikely the
user will change compression of many attributes at once.
4. In optionListToArray() you can use list_qsort() to sort options
list instead of converting it manually into array and then back to a
list.
Good, didn't know about this function.
5. Redundunt #includes:
In heap.c:
#include "access/reloptions.h"
In tsvector.c:
#include "catalog/pg_type.h"
#include "common/pg_lzcompress.h"
In relcache.c:
#include "utils/datum.h"6. Just a minor thing: no reason to change formatting in copy.c - heap_insert(resultRelInfo->ri_RelationDesc, tuple, mycid, - hi_options, bistate); + heap_insert(resultRelInfo->ri_RelationDesc, tuple, + mycid, hi_options, bistate);7. Also in utility.c the extra new line was added which isn't
relevant for this patch.8. In parse_utilcmd.h the 'extern' keyword was removed from
transformRuleStmt declaration which doesn't make sense in this patch.9. Comments. Again, they should be read by a native speaker. So just
a few suggestions:
toast_prepare_varlena() - comment needed
invalidate_amoptions_cache() - comment format doesn't match other
functions in the fileIn htup_details.h:
/* tuple contain custom compressed
* varlenas */
should be "contains"
5-9, all done. Thank you for noticing.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v11.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..60d9d1d515 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..b415aa6e4b
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a72c50eadb..b357b09aaf 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -89,6 +89,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 041afdbd86..686e8db2b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&gist;
&spgist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 286c7a8589..7740d2f2c7 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -320,6 +321,23 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index a0c9a6d257..35147fb3f2 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index c0e548fa5b..59c381c572 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..3e3f4f5c0e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 46276ceff1..4b14146c34 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -944,11 +944,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 09b5794c52..42798eaaa9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2336,13 +2337,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2429,7 +2431,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2598,7 +2600,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2659,8 +2661,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2699,7 +2705,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -3990,6 +3996,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4092,7 +4100,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..e845c360b5 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +122,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +463,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +553,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,48 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1657,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1678,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1690,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2050,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2235,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2431,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2552,142 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 80860128fb..8bf0a3f68a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index a053361163..3b52a96c15 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2524,6 +2530,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 774c07b03a..95d9db34ae 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -629,6 +629,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1598,6 +1599,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 93eba7f30a..7cad8698ac 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -410,6 +410,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -488,6 +489,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index d838666915..f32bc6a2b2 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2409,6 +2422,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3416,6 +3430,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3938,6 +3983,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4482,6 +4531,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 04a24c6082..2f00d51498 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2518,7 +2518,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c5af7e4dd1..ebe71dcb41 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 05123a6fd6..0ccfba4852 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -517,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -691,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -736,6 +749,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -783,6 +803,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1673,6 +1710,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2007,6 +2045,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2034,6 +2085,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2243,6 +2295,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3329,6 +3388,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3804,6 +3864,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4178,6 +4239,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4243,8 +4309,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4473,7 +4540,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4744,6 +4811,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5405,6 +5490,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5543,6 +5638,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5665,6 +5761,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5687,6 +5807,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6521,6 +6741,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8570,6 +8794,44 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9325,6 +9587,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9425,7 +9693,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9450,6 +9719,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9458,6 +9755,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12486,6 +12788,94 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba5095c22b..3322313ec2 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2810,6 +2810,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2828,6 +2829,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5492,6 +5505,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 652a2c5d0b..515c2765cc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2555,6 +2555,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2573,6 +2574,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3629,6 +3640,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e0f4befd9f..ec88a8d7d1 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2805,6 +2805,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2821,6 +2822,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4112,6 +4123,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 5329432f25..6ea6894509 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -310,6 +310,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -397,6 +398,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +584,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +620,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2208,6 +2214,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3372,11 +3387,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3386,8 +3402,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3434,6 +3450,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3638,6 +3691,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
| COMMENTS { $$ = CREATE_TABLE_LIKE_COMMENTS; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5304,12 +5358,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14968,6 +15027,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 9e9472a2a0..f1013a1505 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1078,6 +1078,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1aa7c50c4..8b676f5c39 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2850,7 +2850,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c081b88b73..7d462874aa 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -565,6 +565,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 041cd53a30..ae6801b869 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 520cd095d3..57a251d5a3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 7c5e8c018b..189c2d3dc6 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4478bdd41a..e6e0af80c6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -365,6 +367,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -585,6 +588,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -796,6 +802,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4180,6 +4189,236 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8140,6 +8379,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8175,7 +8416,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8194,7 +8474,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8220,7 +8502,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8244,7 +8528,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8262,7 +8548,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8279,7 +8567,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8309,6 +8599,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8325,6 +8617,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8352,6 +8646,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9852,6 +10148,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12705,6 +13004,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15735,6 +16037,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15797,6 +16107,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -18098,6 +18427,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index c39b2b81cd..b2be14d392 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -622,6 +625,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -722,5 +747,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 40ee5d1d8b..372d32b6b0 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -72,6 +72,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -133,6 +134,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -402,6 +404,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -615,6 +619,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index edc14f176f..33399676aa 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 3e9b4d94dc..6edbc876ef 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2615,6 +2671,39 @@ qr/^\QINSERT INTO test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUE
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4669,9 +4758,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4748,7 +4837,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4955,7 +5044,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4995,7 +5084,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5032,6 +5121,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 466a78004b..fe4e9da2c3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1719,6 +1719,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1835,6 +1851,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1932,6 +1953,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1940,7 +1967,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1951,7 +1978,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index e68ae8db94..74c7197b2e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2009,11 +2009,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..1a380585a7 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index b32c1e9efe..fb088f08e7 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..f25000b63b 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 3424, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 3424
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 3423, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 3423
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..f43d77267d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 3422 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 3422
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..de839faf47
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 3420
+
+CATALOG(pg_attr_compression,3420) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * we're not using system Oids.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 3422 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 3422
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..1098811f9f 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index f01648c961..66d93a2719 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -578,6 +578,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3809,6 +3813,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f f t f
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 3419 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3881,6 +3887,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..0b6693a715 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 91fbaa0e49..5467de3927 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -142,6 +142,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -151,8 +152,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 74b094a9c3..65bdc02db9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -501,7 +502,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index cf007004fa..849c908f77 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +652,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -672,6 +687,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 3,
CREATE_TABLE_LIKE_STORAGE = 1 << 4,
CREATE_TABLE_LIKE_COMMENTS = 1 << 5,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 6,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1771,7 +1787,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 26af944e03..1f2fe78c55 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 3dc62801aa..b5de4f9d12 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 55d573c687..4f3314ec9f 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..16186207c0
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,379 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column f1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column f1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ef0906776e..4743820015 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -436,10 +436,10 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (((a + 1)))
Number of partitions: 0
@@ -597,10 +597,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -736,11 +736,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['b'::text])))
Check constraints:
@@ -749,11 +749,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])))
Partition key: RANGE (b)
@@ -763,11 +763,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = ANY (ARRAY['c'::text])) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -800,46 +800,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -859,11 +859,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..4ad0181e69 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index d2c184f2cf..481db55504 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index a79f891da7..9991897052 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 684f7f20a8..ab8661b73d 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1717,7 +1717,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5433944c6a..9aece8074b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..b704b65e83 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..bbba808f1e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..1fb46b4dab
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,185 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(f1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e8fdf8454d..4d7d20980c 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1160,7 +1160,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index a42ff9794a..de6489688b 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -341,6 +343,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -365,6 +368,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Hello Ildus,
On 29.01.2018 14:44, Ildus Kurbangaliev wrote:
Thanks! Attached new version of the patch.
Patch applies cleanly, builds without any warnings, documentation builds
ok, all tests pass.
A remark for the committers. The patch is quite big, so I really wish
more reviewers looked into it for more comprehensive review. Also a
native english speaker should check the documentation and comments.
Another thing is that tests don't cover cmdrop method because the
built-in pglz compression doesn't use it (I know there is an jsonbd
extension [1]https://github.com/postgrespro/jsonbd based on this patch and which should benefit from cmdrop
method, but it doesn't test it either yet).
I think I did what I could and so passing this patch to committers for
the review. Changed status to "Ready for committer".
[1]: https://github.com/postgrespro/jsonbd
--
Ildar Musin
i.musin@postgrespro.ru
Re: [HACKERS] Custom compression methods
On Mon, 29 Jan 2018 17:29:29 +0300
Ildar Musin <i.musin@postgrespro.ru> wrote:
Patch applies cleanly, builds without any warnings, documentation
builds ok, all tests pass.A remark for the committers. The patch is quite big, so I really wish
more reviewers looked into it for more comprehensive review. Also a
native english speaker should check the documentation and comments.
Another thing is that tests don't cover cmdrop method because the
built-in pglz compression doesn't use it (I know there is an jsonbd
extension [1] based on this patch and which should benefit from
cmdrop method, but it doesn't test it either yet).I think I did what I could and so passing this patch to committers
for the review. Changed status to "Ready for committer".
Thank you!
About cmdrop, I checked that's is called manually, but going to check
it thoroughly in my extension.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, 29 Jan 2018 17:29:29 +0300
Ildar Musin <i.musin@postgrespro.ru> wrote:
Hello Ildus,
On 29.01.2018 14:44, Ildus Kurbangaliev wrote:
Thanks! Attached new version of the patch.
Patch applies cleanly, builds without any warnings, documentation
builds ok, all tests pass.A remark for the committers. The patch is quite big, so I really wish
more reviewers looked into it for more comprehensive review. Also a
native english speaker should check the documentation and comments.
Another thing is that tests don't cover cmdrop method because the
built-in pglz compression doesn't use it (I know there is an jsonbd
extension [1] based on this patch and which should benefit from
cmdrop method, but it doesn't test it either yet).I think I did what I could and so passing this patch to committers
for the review. Changed status to "Ready for committer".
Attached rebased version of the patch so it can be applied to current
master.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v12.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..60d9d1d515 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..b415aa6e4b
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a72c50eadb..b357b09aaf 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -89,6 +89,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 041afdbd86..686e8db2b2 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&gist;
&spgist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 2b514b7606..2684ff5360 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index d2df40d543..790b3707e7 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -817,6 +818,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index c0e548fa5b..59c381c572 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0a13251067..3e3f4f5c0e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 46276ceff1..16e17444ce 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -944,11 +944,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 29d594c6a4..aa4b2e00b9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2356,13 +2357,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2449,7 +2451,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2618,7 +2620,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2679,8 +2681,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2719,7 +2725,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4010,6 +4016,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4112,7 +4120,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..e845c360b5 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +122,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +463,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +553,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,48 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1657,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1678,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1690,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2050,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2235,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2431,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2552,142 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 28ff2f0979..87ac1a2720 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index a053361163..3b52a96c15 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2524,6 +2530,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 0f34f5381a..d383769c6c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -629,6 +629,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1598,6 +1599,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index f8d856cc7a..d109dc99bd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -411,6 +411,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -489,6 +490,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index d838666915..f32bc6a2b2 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2409,6 +2422,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3416,6 +3430,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3938,6 +3983,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4482,6 +4531,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index b3933df9af..92505ed6d8 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2517,7 +2517,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c5af7e4dd1..ebe71dcb41 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7a2da31fbf..c10c49a216 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -517,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -691,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -736,6 +749,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -783,6 +803,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1673,6 +1710,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2007,6 +2045,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2034,6 +2085,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2243,6 +2295,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3329,6 +3388,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3804,6 +3864,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4178,6 +4239,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4243,8 +4309,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4473,7 +4540,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4744,6 +4811,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5405,6 +5490,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5557,6 +5652,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5679,6 +5775,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5701,6 +5821,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6535,6 +6755,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8584,6 +8808,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9339,6 +9602,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9439,7 +9708,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9464,6 +9734,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9472,6 +9770,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12500,6 +12803,94 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6b9e79957e..fd37f7b64d 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2810,6 +2810,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2829,6 +2830,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5493,6 +5506,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 2b34ed53aa..28ea876b47 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2555,6 +2555,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2574,6 +2575,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3630,6 +3641,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e6ba096257..a5231bf963 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2805,6 +2805,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2822,6 +2823,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4113,6 +4124,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 5329432f25..6ea6894509 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -310,6 +310,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -397,6 +398,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -582,6 +584,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -614,9 +620,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2208,6 +2214,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3372,11 +3387,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3386,8 +3402,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3434,6 +3450,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3638,6 +3691,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
| COMMENTS { $$ = CREATE_TABLE_LIKE_COMMENTS; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5304,12 +5358,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14968,6 +15027,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 2e7a473b09..72826fbc73 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,6 +1086,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1aa7c50c4..8b676f5c39 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2850,7 +2850,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d5cc246156..490255ec9a 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -565,6 +565,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 520cd095d3..57a251d5a3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 7c5e8c018b..189c2d3dc6 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4478bdd41a..e6e0af80c6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -365,6 +367,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -585,6 +588,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -796,6 +802,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4180,6 +4189,236 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8140,6 +8379,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8175,7 +8416,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8194,7 +8474,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8220,7 +8502,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8244,7 +8528,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8262,7 +8548,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8279,7 +8567,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8309,6 +8599,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8325,6 +8617,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8352,6 +8646,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9852,6 +10148,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12705,6 +13004,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15735,6 +16037,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15797,6 +16107,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -18098,6 +18427,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index c39b2b81cd..b2be14d392 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -622,6 +625,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -722,5 +747,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 40ee5d1d8b..372d32b6b0 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -72,6 +72,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -133,6 +134,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -402,6 +404,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -615,6 +619,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index edc14f176f..33399676aa 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 3e9b4d94dc..6edbc876ef 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 3420, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2615,6 +2671,39 @@ qr/^\QINSERT INTO test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUE
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4669,9 +4758,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4748,7 +4837,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4955,7 +5044,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4995,7 +5084,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5032,6 +5121,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 466a78004b..fe4e9da2c3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1719,6 +1719,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1835,6 +1851,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1932,6 +1953,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1940,7 +1967,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1951,7 +1978,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index e68ae8db94..74c7197b2e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2009,11 +2009,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..1a380585a7 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index b32c1e9efe..fb088f08e7 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..f25000b63b 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 3424, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 3424
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 3423, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 3423
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..f43d77267d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 3422 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 3422
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..de839faf47
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 3420
+
+CATALOG(pg_attr_compression,3420) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * we're not using system Oids.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 3422 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 3422
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..1098811f9f 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index f01648c961..66d93a2719 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -578,6 +578,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3421 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3809,6 +3813,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f f t f
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 3419 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3881,6 +3887,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 3425 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 3421 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 3426 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3421" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..0b6693a715 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 3421 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 3421
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 91fbaa0e49..5467de3927 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -142,6 +142,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -151,8 +152,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 74b094a9c3..65bdc02db9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -501,7 +502,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3413b478cf..d27c87f707 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -615,6 +615,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -638,6 +652,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -674,6 +689,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 3,
CREATE_TABLE_LIKE_STORAGE = 1 << 4,
CREATE_TABLE_LIKE_COMMENTS = 1 << 5,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 6,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1773,7 +1789,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 26af944e03..1f2fe78c55 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 3dc62801aa..b5de4f9d12 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -56,7 +56,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -68,7 +68,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -146,9 +147,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -281,8 +291,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -311,6 +327,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..1d2be6a3d0
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,379 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column d1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column d1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 3420 | 0 | n
+ 3420 | 0 | 1259 | 1 | i
+ 3420 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index f5e56365f5..4fdf111b32 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -437,11 +437,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -450,11 +450,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -612,10 +612,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -751,11 +751,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -764,11 +764,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -778,11 +778,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -815,46 +815,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -874,11 +874,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -887,10 +887,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..4ad0181e69 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 6a1b278e5a..7e9dea2396 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index a79f891da7..9991897052 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 684f7f20a8..ab8661b73d 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1717,7 +1717,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5433944c6a..9aece8074b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..b704b65e83 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..bbba808f1e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..573e51b701
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,185 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 3420 OR classid = 3420) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e8fdf8454d..4d7d20980c 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1160,7 +1160,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index d4765ce3b0..6d8cfebef3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -343,6 +345,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -367,6 +370,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Hi,
Attached new version of the patch, rebased to current master, and fixed
conflicting catalog Oids.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v13.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 71e20f2740..60d9d1d515 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..b415aa6e4b
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 732b8ab7d0..59716f31aa 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -90,6 +90,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 054347b17d..229990941d 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..9a0b2902cd 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 338dddd7cc..4d110bc7a9 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -831,6 +832,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index c0e548fa5b..59c381c572 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a2f67f2332..4edd8112d8 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 46276ceff1..16e17444ce 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -944,11 +944,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 29d594c6a4..aa4b2e00b9 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2356,13 +2357,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2449,7 +2451,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2618,7 +2620,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2679,8 +2681,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2719,7 +2725,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4010,6 +4016,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4112,7 +4120,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..e845c360b5 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +122,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +463,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +553,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,48 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1657,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1678,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1690,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2050,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2235,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2431,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2552,142 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 28ff2f0979..87ac1a2720 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 4433afde9c..81c0431504 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2540,6 +2546,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..c7cf40ef3b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -629,6 +629,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1598,6 +1599,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c2b0137707..d2e5d042b8 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -411,6 +411,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -489,6 +490,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 9c3055ac0c..3a28c564eb 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2411,6 +2424,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3419,6 +3433,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3941,6 +3986,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4485,6 +4534,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 4562a5121d..4ab4dd2001 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2516,7 +2516,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c5af7e4dd1..ebe71dcb41 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8dfbb5bc69..c842aedde1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -517,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -691,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -736,6 +749,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -783,6 +803,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1676,6 +1713,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2010,6 +2048,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2037,6 +2088,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2246,6 +2298,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3332,6 +3391,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3807,6 +3867,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4181,6 +4242,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4246,8 +4312,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4476,7 +4543,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4747,6 +4814,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5408,6 +5493,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5560,6 +5655,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5682,6 +5778,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5704,6 +5824,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6539,6 +6759,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8599,6 +8823,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9354,6 +9617,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9454,7 +9723,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9479,6 +9749,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9487,6 +9785,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12515,6 +12818,94 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 04b9456ec3..e582bbff2a 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2820,6 +2820,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2839,6 +2840,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5504,6 +5517,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 7a9e6efb05..0b344257a5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2556,6 +2556,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2575,6 +2576,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3636,6 +3647,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 011d2a3fa9..0966314ecc 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2810,6 +2810,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2827,6 +2828,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4123,6 +4134,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d99f2be2c9..49aef6606b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -310,6 +310,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -397,6 +398,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -583,6 +585,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -615,9 +621,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2212,6 +2218,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3376,11 +3391,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3390,8 +3406,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3438,6 +3454,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3642,6 +3695,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
| COMMENTS { $$ = CREATE_TABLE_LIKE_COMMENTS; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5308,12 +5362,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14974,6 +15033,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 36cca60302..896b50dd10 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1062,6 +1062,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1aa7c50c4..8b676f5c39 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2850,7 +2850,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 7ca1bcb9b3..1ca34a014d 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -795,6 +795,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 1ebf9c4ed2..8ec594afe9 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -565,6 +565,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 520cd095d3..57a251d5a3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index a4deb53e3a..66493e971d 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -179,6 +179,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f4e75a69fa..265b7e6477 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -365,6 +367,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -585,6 +588,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -796,6 +802,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4180,6 +4189,236 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
destroyPQExpBuffer(query);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ /* Make sure we are in proper schema */
+ selectSourceSchema(fout, "pg_catalog");
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8113,6 +8352,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8148,7 +8389,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8167,7 +8447,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8193,7 +8475,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8217,7 +8501,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8235,7 +8521,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8252,7 +8540,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8282,6 +8572,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8298,6 +8590,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8325,6 +8619,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9825,6 +10121,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12678,6 +12977,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15708,6 +16010,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15770,6 +16080,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
}
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -18118,6 +18447,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 017dbf58eb..7b69eb0de9 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -621,6 +624,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -721,5 +746,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 40ee5d1d8b..372d32b6b0 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -72,6 +72,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -133,6 +134,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -402,6 +404,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -615,6 +619,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index edc14f176f..33399676aa 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ac9cfa04c1..8da0024011 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2615,6 +2671,39 @@ qr/^\QINSERT INTO test_table_identity (col1, col2) OVERRIDING SYSTEM VALUE VALUE
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4669,9 +4758,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4748,7 +4837,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4955,7 +5044,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4995,7 +5084,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5032,6 +5121,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 466a78004b..fe4e9da2c3 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1719,6 +1719,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1835,6 +1851,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1932,6 +1953,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1940,7 +1967,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1951,7 +1978,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index e68ae8db94..74c7197b2e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2009,11 +2009,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..1a380585a7 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index b32c1e9efe..fb088f08e7 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..fd5c76a44f 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..5cb2852f41 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 4002 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 4002
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..47e739622c
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 4001
+
+CATALOG(pg_attr_compression,4001) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * these are just numbers and they are not using system Oid column.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 4002 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 4002
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 26b1866c69..1098811f9f 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 29 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index c00d055940..ef4e6642e3 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -578,6 +578,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 4005 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3843,6 +3847,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f f t f
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 4008 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3915,6 +3921,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 4006 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f f i s 1 0 4005 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 4007 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f f t f i s 1 0 2275 "4005" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..1b0283122b 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 4005 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 4005
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 91930d7e61..85d82e1afb 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -144,6 +144,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -153,8 +154,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 74b094a9c3..65bdc02db9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -501,7 +502,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 73cd3e85e3..be159bf510 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -621,6 +621,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -644,6 +658,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -680,6 +695,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 3,
CREATE_TABLE_LIKE_STORAGE = 1 << 4,
CREATE_TABLE_LIKE_COMMENTS = 1 << 5,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 6,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1787,7 +1803,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index cf32197bc3..773d896b9b 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index bbcb50e41f..abba43cb94 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..af23cdb5fd
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,379 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column d1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column d1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 39a963888d..ed4e844d6c 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -425,11 +425,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -438,11 +438,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -600,10 +600,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -739,11 +739,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -752,11 +752,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -766,11 +766,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -803,46 +803,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -862,11 +862,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -875,10 +875,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 3f405c94ce..4ad0181e69 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -156,32 +156,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -195,12 +195,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -210,12 +210,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -229,11 +229,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 6a1b278e5a..7e9dea2396 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index a79f891da7..9991897052 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 6616cc1bf0..2d93b304d9 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1723,7 +1723,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5433944c6a..9aece8074b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2799,11 +2799,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2819,11 +2819,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..b704b65e83 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..bbba808f1e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..0c359c4aa9
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,185 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e8fdf8454d..4d7d20980c 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1160,7 +1160,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index d4765ce3b0..6d8cfebef3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -343,6 +345,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -367,6 +370,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
On Mon, 26 Feb 2018 15:25:56 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
Hi,
Attached new version of the patch, rebased to current master, and
fixed conflicting catalog Oids.
Attached rebased version of the patch, fixed conficts in pg_proc, and
tap tests.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v14.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 1e22c1eefc..053db72c10 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -416,12 +416,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -430,12 +430,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -443,12 +443,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -461,13 +461,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index a0e6d7062b..f0b9c0911f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..b415aa6e4b
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 732b8ab7d0..59716f31aa 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -90,6 +90,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 054347b17d..229990941d 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..9a0b2902cd 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 14a43b45e9..0c0a50b6aa 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -834,6 +835,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index c0e548fa5b..59c381c572 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a2f67f2332..4edd8112d8 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1456,6 +1475,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 46276ceff1..16e17444ce 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -944,11 +944,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 579ca8ff35..4ac6a7ef9b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2356,13 +2357,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2449,7 +2451,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2618,7 +2620,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2679,8 +2681,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2719,7 +2725,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4010,6 +4016,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4112,7 +4120,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..e845c360b5 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +122,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +463,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +553,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,48 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1657,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1678,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1690,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2050,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2235,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2431,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2552,142 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 28ff2f0979..87ac1a2720 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 4433afde9c..81c0431504 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2540,6 +2546,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index cf36ce4add..c7cf40ef3b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -629,6 +629,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1598,6 +1599,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5112429a97..53dd150b3c 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -411,6 +411,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -489,6 +490,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index f6775f4048..ecf432025b 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2411,6 +2424,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3419,6 +3433,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3941,6 +3986,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4485,6 +4534,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 4562a5121d..4ab4dd2001 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2516,7 +2516,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index c5af7e4dd1..ebe71dcb41 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index ab6a889b12..e19683a017 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -478,7 +478,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 037d2d2152..b4db785dba 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -517,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -691,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -736,6 +749,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -783,6 +803,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1676,6 +1713,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2010,6 +2048,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2037,6 +2088,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2246,6 +2298,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3332,6 +3391,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3807,6 +3867,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4181,6 +4242,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4246,8 +4312,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4476,7 +4543,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4747,6 +4814,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5408,6 +5493,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5560,6 +5655,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5682,6 +5778,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5704,6 +5824,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6539,6 +6759,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8599,6 +8823,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9354,6 +9617,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9454,7 +9723,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9479,6 +9749,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9487,6 +9785,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12515,6 +12818,94 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 795bdeb5b8..58812eed8b 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2820,6 +2820,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2839,6 +2840,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5505,6 +5518,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 0db328113c..da01c72d2b 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2557,6 +2557,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2576,6 +2577,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3637,6 +3648,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index fd80891954..df4068a666 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2811,6 +2811,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2828,6 +2829,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4124,6 +4135,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 06c03dff3c..eed970abc4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -398,6 +399,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -584,6 +586,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -616,9 +622,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2213,6 +2219,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3377,11 +3392,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3391,8 +3407,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3439,6 +3455,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3644,6 +3697,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5312,12 +5366,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14994,6 +15053,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 52b234cd9c..cad90b8dd3 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1073,6 +1073,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2280d73701..6d83b5dc2d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2876,7 +2876,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 7ca1bcb9b3..1ca34a014d 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -795,6 +795,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6ab4db26bd..42178ecfb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -565,6 +565,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ceedd481fb..b33fcbc168 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index fc233a608f..e058d72bcb 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -180,6 +180,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bef6b0f88f..677a3c09d1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -377,6 +379,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -597,6 +600,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -808,6 +814,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4195,6 +4204,233 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
free(qsubname);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8068,6 +8304,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8097,7 +8335,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8116,7 +8393,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8142,7 +8421,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8166,7 +8447,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8184,7 +8467,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8201,7 +8486,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8231,6 +8518,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8247,6 +8536,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8274,6 +8565,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9764,6 +10057,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12522,6 +12818,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15447,6 +15746,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15505,6 +15812,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17765,6 +18091,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 017dbf58eb..7b69eb0de9 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -621,6 +624,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -721,5 +746,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 75e4a539bf..575449b22b 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -135,6 +136,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -404,6 +406,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -617,6 +621,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index edc14f176f..33399676aa 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1bea6ae81d..3cf5e4d836 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2615,6 +2671,39 @@ qr/^\QINSERT INTO dump_test.test_table_identity (col1, col2) OVERRIDING SYSTEM V
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4669,9 +4758,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4748,7 +4837,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4955,7 +5044,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4995,7 +5084,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5032,6 +5121,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 0c3be1f504..7bd0caf2f6 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1755,6 +1755,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1871,6 +1887,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1968,6 +1989,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1976,7 +2003,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1987,7 +2014,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8bee858042..c201467c3f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2159,11 +2159,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 2ab1815390..1a380585a7 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -265,7 +265,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -679,6 +681,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -794,7 +799,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index b32c1e9efe..fb088f08e7 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 0bb875441e..fd5c76a44f 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..5cb2852f41 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 4002 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 4002
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..47e739622c
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 4001
+
+CATALOG(pg_attr_compression,4001) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * these are just numbers and they are not using system Oid column.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 4002 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 4002
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 01cf59e7a3..c06900d474 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 28 0 t f f f f f f t n f 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index 0fdb42f639..8a60380e53 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -576,6 +576,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 4005 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3841,6 +3845,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f t f s
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 4008 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3913,6 +3919,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f i s 1
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 4006 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f i s 1 0 4005 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 4007 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f t f i s 1 0 2275 "4005" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..1b0283122b 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 4005 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 4005
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 91930d7e61..85d82e1afb 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -144,6 +144,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -153,8 +154,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 74b094a9c3..65bdc02db9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -470,6 +470,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -501,7 +502,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index a09b1d4f21..12d8cddb88 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -621,6 +621,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -644,6 +658,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -681,6 +696,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1788,7 +1804,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index cf32197bc3..773d896b9b 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index bbcb50e41f..abba43cb94 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..af23cdb5fd
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,379 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column d1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column d1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 39a963888d..ed4e844d6c 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -425,11 +425,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -438,11 +438,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -600,10 +600,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -739,11 +739,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -752,11 +752,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -766,11 +766,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -803,46 +803,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -862,11 +862,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -875,10 +875,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 6a1b278e5a..7e9dea2396 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: pt1
DROP FOREIGN TABLE ft2;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT pt1;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE pt1 ADD COLUMN c6 integer;
ALTER TABLE pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE pt1 ADD COLUMN c8 integer;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE pt1 DROP COLUMN c6;
ALTER TABLE pt1 DROP COLUMN c7;
ALTER TABLE pt1 DROP COLUMN c8;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT pt1;
-- child does not inherit NO INHERIT constraints
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk1" CHECK (c1 > 0) NO INHERIT
"pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE pt1 DROP CONSTRAINT pt1chk2 CASCADE;
INSERT INTO pt1 VALUES (1, 'pt1'::text, '1994-01-01'::date);
ALTER TABLE pt1 ADD CONSTRAINT pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE pt1 VALIDATE CONSTRAINT pt1chk3;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: pt1
-- OID system column
ALTER TABLE pt1 SET WITH OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE pt1 SET WITHOUT OIDS;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE pt1 RENAME CONSTRAINT pt1chk3 TO f2_check;
\d+ pt1
- Table "public.pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE pt2 (
CREATE FOREIGN TABLE pt2_1 PARTITION OF pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "pt2_1" contains column "c4" not found in parent "pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE pt2_1;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ALTER c2 SET NOT NULL;
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE pt2 ATTACH PARTITION pt2_1 FOR VALUES IN (1);
ALTER TABLE pt2 DETACH PARTITION pt2_1;
ALTER TABLE pt2 ADD CONSTRAINT pt2chk1 CHECK (c1 > 0);
\d+ pt2
- Table "public.pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index d768dc0215..29133afa66 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index dcbaad8e2f..0cd28a6e5d 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -759,11 +759,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -775,74 +775,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 01608d2c04..bd85b93d3e 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1724,7 +1724,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index d7eff6c0a7..d63a5c6763 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2801,11 +2801,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2821,11 +2821,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ad9434fb87..b704b65e83 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 27cd49845e..bbba808f1e 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..0c359c4aa9
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,185 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index a593d37643..9c2d8fe89d 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1161,7 +1161,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index d4765ce3b0..6d8cfebef3 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -343,6 +345,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -367,6 +370,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
Attached rebased version of the patch. Fixed conflicts in pg_class.h.
--
----
Regards,
Ildus Kurbangaliev
Attachments:
custom_compression_methods_v14.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 95a5b113b9..8118b92e0a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..8950c5d1ec
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 732b8ab7d0..59716f31aa 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -90,6 +90,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 054347b17d..229990941d 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index afe213910c..9a0b2902cd 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 14a43b45e9..0c0a50b6aa 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -834,6 +835,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index c0e548fa5b..59c381c572 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c45a48812b..c697a68b18 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -145,7 +145,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -169,6 +169,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -195,6 +196,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
*bitP |= bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on
* an offset. This is a bit of a hack.
@@ -213,6 +217,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
Pointer val = DatumGetPointer(values[i]);
*infomask |= HEAP_HASVARWIDTH;
+
if (VARATT_IS_EXTERNAL(val))
{
if (VARATT_IS_EXTERNAL_EXPANDED(val))
@@ -230,10 +235,20 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
else
{
+
*infomask |= HEAP_HASEXTERNAL;
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -257,6 +272,9 @@ heap_fill_tuple(TupleDesc tupleDesc,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -774,6 +792,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1482,6 +1501,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 46276ceff1..16e17444ce 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -944,11 +944,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index f1f44230cd..e43818a5e4 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "miscadmin.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -135,6 +146,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -197,6 +209,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -280,6 +293,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->attnotnull = false;
dstAtt->atthasdef = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -384,6 +398,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -434,6 +450,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -496,6 +514,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -601,6 +620,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -713,7 +733,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..76463b84ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,168 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cmdrop = NULL; /* no drop behavior */
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..0ebf6e46a8
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetCompressionAmRoutine
+ *
+ * Return CompressionAmRoutine by attribute compression Oid
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutine(Oid acoid)
+{
+ Oid amoid;
+ regproc amhandler;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* extract access method name and get handler for it */
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct. */
+ return InvokeCompressionAmHandler(amhandler);
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index c08ab14c02..75eaf0ae08 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -93,7 +93,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2356,13 +2357,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2449,7 +2451,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2618,7 +2620,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2679,8 +2681,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2719,7 +2725,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4010,6 +4016,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4112,7 +4120,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..e845c360b5 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,24 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,6 +122,9 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
+static void init_amoptions_cache(void);
+static CompressionAmOptions *lookup_amoptions(Oid acoid);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
@@ -421,7 +463,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +553,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +656,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +668,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +803,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +887,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +902,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +920,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1065,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1198,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1427,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,7 +1507,6 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
-
/* ----------
* toast_compress_datum -
*
@@ -1368,54 +1521,48 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_amoptions(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_CMID(tmp, cmoptions->acoid);
/* successful compression */
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1657,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1678,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1690,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2050,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2235,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2431,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_amoptions(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2552,142 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ if (entry->amroutine)
+ pfree(entry->amroutine);
+ if (entry->acoptions)
+ list_free_deep(entry->acoptions);
+ if (entry->acstate)
+ pfree(entry->acstate);
+
+ if (hash_search(amoptions_cache,
+ (void *) &entry->acoid,
+ HASH_REMOVE,
+ NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_amoptions
+ *
+ * Return or generate cache record for attribute compression.
+ */
+static CompressionAmOptions *
+lookup_amoptions(Oid acoid)
+{
+ bool found;
+ CompressionAmOptions *result;
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt;
+
+ result->amroutine = NULL;
+ result->acoptions = NULL;
+ result->acstate = NULL;
+
+ /* fill access method options in cache */
+ oldcxt = MemoryContextSwitchTo(amoptions_cache_mcxt);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = GetAttrCompressionAmOid(acoid);
+ result->amroutine = GetCompressionAmRoutine(acoid);
+ result->acoptions = GetAttrCompressionOptions(acoid);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_amoptions(acoid1);
+ b = lookup_amoptions(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 28ff2f0979..87ac1a2720 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b7e39af7a2..7490abcc24 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2540,6 +2546,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b69bb1e2a4..193e5da312 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -629,6 +629,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1601,6 +1602,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bfac37f9d1..26c33febe5 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -411,6 +411,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -489,6 +490,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 119297b33a..1b395e6b5a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2411,6 +2424,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3419,6 +3433,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3941,6 +3986,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4485,6 +4534,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 0d63866fb0..4dda5c1690 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3a051a215e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,659 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ CompressionAmRoutine *routine;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ CompressionAmRoutine *routine;
+
+ /* call drop callback */
+ routine = GetCompressionAmRoutine(acform->acoid);
+ if (routine->cmdrop)
+ routine->cmdrop(acoid);
+ pfree(routine);
+
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index a42861da0d..2148b7064b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2516,7 +2516,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 549c7ea51d..f4948465ea 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 23892b1b81..2802e33402 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e74fb1f469..69ad7be718 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -518,6 +528,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -692,6 +703,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -737,6 +750,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -785,6 +805,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1677,6 +1714,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2011,6 +2049,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2038,6 +2089,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2247,6 +2299,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3333,6 +3392,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3808,6 +3868,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4182,6 +4243,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4247,8 +4313,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4477,7 +4544,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4748,6 +4815,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5409,6 +5494,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5561,6 +5656,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5683,6 +5779,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5705,6 +5825,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6540,6 +6760,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8601,6 +8825,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9356,6 +9619,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9456,7 +9725,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9481,6 +9751,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9489,6 +9787,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12517,6 +12820,94 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index c7293a60d7..e331f19448 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2821,6 +2821,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2840,6 +2841,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5507,6 +5520,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 765b1be74b..25f2ce9ded 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2558,6 +2558,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2577,6 +2578,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3638,6 +3649,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f61ae03ac5..de061ecbea 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2812,6 +2812,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2829,6 +2830,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4125,6 +4136,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index cd5ba2d4d8..99f3d1a6a7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -398,6 +399,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -584,6 +586,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -616,9 +622,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2213,6 +2219,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3377,11 +3392,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3391,8 +3407,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3439,6 +3455,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3644,6 +3697,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5312,12 +5366,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14989,6 +15048,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0fd14f43c6..cfcfde5683 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1073,6 +1073,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 5ffe638b19..dce56f7bcd 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2883,7 +2883,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 0bf5fe8cc7..3668310c2a 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -795,6 +795,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6ab4db26bd..42178ecfb4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -565,6 +565,11 @@ RelationBuildTupleDesc(Relation relation)
attrdef[ndef].adbin = NULL;
ndef++;
}
+
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
need--;
if (need == 0)
break;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ceedd481fb..b33fcbc168 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 83c976eaf7..319d78b548 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -180,6 +180,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b8d65a9ee3..23db7eac21 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -377,6 +379,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -597,6 +600,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -808,6 +814,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4195,6 +4204,233 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
free(qsubname);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8068,6 +8304,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8097,7 +8335,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8116,7 +8393,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8142,7 +8421,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8166,7 +8447,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8184,7 +8467,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8201,7 +8486,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8231,6 +8518,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8247,6 +8536,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8274,6 +8565,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9764,6 +10057,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12526,6 +12822,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15451,6 +15750,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15509,6 +15816,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17769,6 +18095,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a4d6d926a8..9585040ed6 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -621,6 +624,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -721,5 +746,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 75e4a539bf..575449b22b 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -135,6 +136,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -404,6 +406,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -617,6 +621,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index edc14f176f..33399676aa 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1bea6ae81d..3cf5e4d836 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2615,6 +2671,39 @@ qr/^\QINSERT INTO dump_test.test_table_identity (col1, col2) OVERRIDING SYSTEM V
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4669,9 +4758,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4748,7 +4837,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4955,7 +5044,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4995,7 +5084,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5032,6 +5121,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 0c3be1f504..7bd0caf2f6 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1755,6 +1755,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1871,6 +1887,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1968,6 +1989,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1976,7 +2003,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1987,7 +2014,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 08d8ef09a4..46af83344f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2159,11 +2159,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..4d21d4303e
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,86 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef void (*cmdrop_function) (Oid acoid);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cmdrop' - called before drop of attribute compression. Could be used
+ * by an extension to cleanup some data related with attribute compression
+ * (like dictionaries etc).
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern CompressionAmRoutine *GetCompressionAmRoutine(Oid acoid);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 3616a17b6f..0d59a20b76 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -269,7 +269,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -683,6 +685,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -798,7 +803,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index b32c1e9efe..fb088f08e7 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -258,7 +258,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -269,6 +268,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 415efbab97..2747a9bc55 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -43,6 +45,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..d5561406cc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 7dd9d108d6..1eb3e319c5 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..5cb2852f41 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 4002 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 4002
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..47e739622c
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 4001
+
+CATALOG(pg_attr_compression,4001) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * these are just numbers and they are not using system Oid column.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 4002 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 4002
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 8159383834..e63eaa52f2 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -174,10 +177,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -191,29 +194,30 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 22
-#define Anum_pg_attribute_attrelid 1
-#define Anum_pg_attribute_attname 2
-#define Anum_pg_attribute_atttypid 3
-#define Anum_pg_attribute_attstattarget 4
-#define Anum_pg_attribute_attlen 5
-#define Anum_pg_attribute_attnum 6
-#define Anum_pg_attribute_attndims 7
-#define Anum_pg_attribute_attcacheoff 8
-#define Anum_pg_attribute_atttypmod 9
-#define Anum_pg_attribute_attbyval 10
-#define Anum_pg_attribute_attstorage 11
-#define Anum_pg_attribute_attalign 12
-#define Anum_pg_attribute_attnotnull 13
-#define Anum_pg_attribute_atthasdef 14
-#define Anum_pg_attribute_attidentity 15
-#define Anum_pg_attribute_attisdropped 16
-#define Anum_pg_attribute_attislocal 17
-#define Anum_pg_attribute_attinhcount 18
-#define Anum_pg_attribute_attcollation 19
-#define Anum_pg_attribute_attacl 20
-#define Anum_pg_attribute_attoptions 21
-#define Anum_pg_attribute_attfdwoptions 22
+#define Natts_pg_attribute 23
+#define Anum_pg_attribute_attrelid 1
+#define Anum_pg_attribute_attname 2
+#define Anum_pg_attribute_atttypid 3
+#define Anum_pg_attribute_attstattarget 4
+#define Anum_pg_attribute_attlen 5
+#define Anum_pg_attribute_attnum 6
+#define Anum_pg_attribute_attndims 7
+#define Anum_pg_attribute_attcacheoff 8
+#define Anum_pg_attribute_atttypmod 9
+#define Anum_pg_attribute_attbyval 10
+#define Anum_pg_attribute_attstorage 11
+#define Anum_pg_attribute_attalign 12
+#define Anum_pg_attribute_attnotnull 13
+#define Anum_pg_attribute_atthasdef 14
+#define Anum_pg_attribute_attidentity 15
+#define Anum_pg_attribute_attisdropped 16
+#define Anum_pg_attribute_attislocal 17
+#define Anum_pg_attribute_attinhcount 18
+#define Anum_pg_attribute_attcollation 19
+#define Anum_pg_attribute_attcompression 20
+#define Anum_pg_attribute_attacl 21
+#define Anum_pg_attribute_attoptions 22
+#define Anum_pg_attribute_attfdwoptions 23
/* ----------------
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 85cdb99f1f..a25daf18c3 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 22 0 f f f f f f t n f 0 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 23 0 f f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 28 0 t f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index bfc90098f8..96df396aac 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -576,6 +576,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3453 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 4005 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3843,6 +3847,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f t f s
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 4008 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3915,6 +3921,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f i s 1
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 4006 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f i s 1 0 4005 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 4007 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f t f i s 1 0 2275 "4005" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..1b0283122b 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 4005 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 4005
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8fc9e424cf..9acbebb0bc 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -154,8 +155,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 443de22704..3edf3bba81 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -471,6 +471,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -502,7 +503,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 92082b3a7a..72ac1ca3f2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -621,6 +621,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -644,6 +658,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -681,6 +696,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1788,7 +1804,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index cf32197bc3..773d896b9b 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index bbcb50e41f..abba43cb94 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..af23cdb5fd
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,379 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column d1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column d1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ f1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ f2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 39a963888d..ed4e844d6c 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -425,11 +425,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -438,11 +438,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -600,10 +600,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -739,11 +739,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -752,11 +752,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -766,11 +766,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -803,46 +803,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -862,11 +862,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -875,10 +875,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 7b54e96d30..9faf540236 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: fd_pt1
-- OID system column
ALTER TABLE fd_pt1 SET WITH OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE fd_pt1 SET WITHOUT OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index f56151fc1e..d4215af768 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 97419a744f..b33f8f93e2 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -785,11 +785,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -801,74 +801,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 01608d2c04..bd85b93d3e 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1724,7 +1724,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5149b72fe9..8b0baeb0c3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2801,11 +2801,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2821,11 +2821,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index d308a05117..230ad1faa9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 45147e9328..91a53699f0 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..0c359c4aa9
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,185 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(f1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE INT USING f1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'f1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(f1 TEXT, f2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN f2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN f1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN f1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index a593d37643..9c2d8fe89d 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1161,7 +1161,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 17bf55c1f5..8f85146a62 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -343,6 +345,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -368,6 +371,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
On Mon, 26 Mar 2018 20:38:25 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
Attached rebased version of the patch. Fixed conflicts in pg_class.h.
New rebased version due to conflicts in master. Also fixed few errors
and removed cmdrop method since it couldnt be tested.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
custom_compression_methods_v15.patchtext/x-patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index d6a9d8c580..c2504acff9 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..8950c5d1ec
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 56b8da0448..4575a3d4b7 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index a7f6c8dc6a..586c4e30c8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 0070603fc3..d957f9b08b 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -252,6 +252,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 69f3355ede..9c3b7cfa0c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 14a43b45e9..0c0a50b6aa 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -834,6 +835,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 70a822e059..dd4cc8352b 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 960bbe4203..ae693569fb 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -216,6 +216,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -246,6 +247,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -284,6 +288,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -307,6 +320,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -343,7 +359,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -367,6 +383,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -377,6 +394,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -795,6 +813,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -919,6 +938,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -937,6 +957,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -989,6 +1010,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].ammissing,
false);
}
@@ -999,6 +1021,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1154,6 +1177,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1853,6 +1877,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f7103e53bc..d1417445ab 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -47,6 +47,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -74,13 +75,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -92,7 +110,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -142,6 +160,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 35c09987ad..f5d7ac3a77 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -953,11 +953,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2658399484..72eeb72084 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -74,6 +75,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -96,8 +98,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -138,6 +149,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -302,6 +315,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -468,6 +484,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -553,6 +571,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -659,6 +678,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
@@ -772,7 +792,6 @@ TupleDescInitEntryCollation(TupleDesc desc,
TupleDescAttr(desc, attributeNumber - 1)->attcollation = collationid;
}
-
/*
* BuildDescForRelation
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..a0c0ca0811
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,167 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz_cm.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d7279248e7..2d47307662 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -96,7 +96,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2360,13 +2361,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2453,7 +2455,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2622,7 +2624,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2683,8 +2685,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2723,7 +2729,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4016,6 +4022,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -4118,7 +4126,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7d466c2588..4f3c99a70a 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -654,7 +654,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 546f80f05c..9a9805f6b6 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +61,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -421,7 +462,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +552,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +655,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +667,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +802,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +886,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +901,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +919,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1064,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1046,6 +1197,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1274,6 +1426,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1353,6 +1506,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1368,54 +1533,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1668,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1689,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1701,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2061,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2246,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2442,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2563,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate caches so we need to call it before
+ * we're putting something to cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreateExtended(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 28ff2f0979..87ac1a2720 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -731,6 +731,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 30ca509534..eeb0438fb3 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -33,7 +33,7 @@ POSTGRES_BKI_SRCS = $(addprefix $(top_srcdir)/src/include/catalog/,\
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b7e39af7a2..7490abcc24 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -173,7 +174,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1283,6 +1285,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2540,6 +2546,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index a1def77944..82e182895d 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -634,6 +634,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1607,6 +1608,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bc99a60d34..3f4a0505c7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -413,6 +413,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
to->attcollation = collationObjectId[i];
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -491,6 +492,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 119297b33a..1b395e6b5a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -490,6 +491,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -2411,6 +2424,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
(errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
errmsg("must be superuser")));
break;
+
case OBJECT_STATISTIC_EXT:
if (!pg_statistics_object_ownercheck(address.objectId, roleid))
aclcheck_error_type(ACLCHECK_NOT_OWNER, address.objectId);
@@ -3419,6 +3433,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -3941,6 +3986,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4485,6 +4534,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 0d63866fb0..4dda5c1690 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..4e3d912f7a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,654 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression and
+ * call drop callback from access method routine.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index a42861da0d..2148b7064b 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2516,7 +2516,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 549c7ea51d..f4948465ea 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1187,6 +1187,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 23892b1b81..2802e33402 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 83a881eff3..5fb84470c8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -21,6 +21,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -32,6 +33,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_constraint_fn.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits_fn.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -90,6 +93,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -370,6 +376,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -518,6 +528,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -692,6 +703,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -738,6 +751,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -786,6 +806,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -1678,6 +1715,7 @@ storage_name(char c)
}
}
+
/*----------
* MergeAttributes
* Returns new schema given initial schema and superclasses.
@@ -2012,6 +2050,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2039,6 +2090,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2248,6 +2300,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3334,6 +3393,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3809,6 +3869,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4183,6 +4244,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4248,8 +4314,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4478,7 +4545,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4749,6 +4816,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -5411,6 +5496,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5585,6 +5680,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5707,6 +5803,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5729,6 +5849,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6565,6 +6785,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -8626,6 +8850,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9381,6 +9644,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9481,7 +9750,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9506,6 +9776,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9514,6 +9812,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12542,6 +12845,94 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index c7293a60d7..e331f19448 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2821,6 +2821,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2840,6 +2841,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5507,6 +5520,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 765b1be74b..25f2ce9ded 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2558,6 +2558,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2577,6 +2578,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3638,6 +3649,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6c76c41ebe..92b53e873e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3623,6 +3623,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3630,6 +3632,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f61ae03ac5..de061ecbea 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2812,6 +2812,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2829,6 +2830,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4125,6 +4136,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index cd5ba2d4d8..99f3d1a6a7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -398,6 +399,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -584,6 +586,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -616,9 +622,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2213,6 +2219,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3377,11 +3392,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3391,8 +3407,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3439,6 +3455,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3644,6 +3697,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5312,12 +5366,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14989,6 +15048,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0fd14f43c6..cfcfde5683 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1073,6 +1073,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index b4016ed52b..480a1a2fd7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2880,7 +2880,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 0bf5fe8cc7..3668310c2a 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -795,6 +795,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 69a2114a10..e72aa2e540 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -575,6 +575,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ceedd481fb..b33fcbc168 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 83c976eaf7..319d78b548 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -180,6 +180,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b8d65a9ee3..23db7eac21 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate.h"
#include "catalog/pg_am.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_class.h"
#include "catalog/pg_default_acl.h"
@@ -377,6 +379,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -597,6 +600,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -808,6 +814,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4195,6 +4204,233 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
free(qsubname);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8068,6 +8304,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8097,7 +8335,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8116,7 +8393,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8142,7 +8421,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8166,7 +8447,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8184,7 +8467,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8201,7 +8486,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8231,6 +8518,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8247,6 +8536,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8274,6 +8565,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9764,6 +10057,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12526,6 +12822,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15451,6 +15750,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15509,6 +15816,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17769,6 +18095,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a4d6d926a8..9585040ed6 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -621,6 +624,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -721,5 +746,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index 5ce3c5d485..463afaa3c3 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 75e4a539bf..575449b22b 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -135,6 +136,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -404,6 +406,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -617,6 +621,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index edc14f176f..33399676aa 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 1bea6ae81d..3cf5e4d836 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -904,6 +904,62 @@ my %tests = (
section_pre_data => 1,
section_data => 1, }, },
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
all_runs => 1,
catch_all => 'ALTER TABLE ... commands',
@@ -2615,6 +2671,39 @@ qr/^\QINSERT INTO dump_test.test_table_identity (col1, col2) OVERRIDING SYSTEM V
section_post_data => 1,
test_schema_plus_blobs => 1, }, },
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
@@ -4669,9 +4758,9 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -4748,7 +4837,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like => {
@@ -4955,7 +5044,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -4995,7 +5084,7 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -5032,6 +5121,49 @@ qr/CREATE TRANSFORM FOR integer LANGUAGE sql \(FROM SQL WITH FUNCTION pg_catalog
role => 1,
section_post_data => 1, }, },
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
all_runs => 1,
catch_all => 'CREATE ... commands',
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 0c3be1f504..7bd0caf2f6 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1755,6 +1755,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1871,6 +1887,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1968,6 +1989,12 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1976,7 +2003,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1987,7 +2014,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 6926ca132e..2092e4b431 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2159,11 +2159,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..6bb40a2fea
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,81 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "access/transam.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 4c0256b18a..9b091ff583 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -47,6 +47,9 @@ typedef enum LockTupleMode
LockTupleExclusive
} LockTupleMode;
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
#define MaxLockTupleMode LockTupleExclusive
/*
@@ -72,7 +75,6 @@ typedef struct HeapUpdateFailureData
CommandId cmax;
} HeapUpdateFailureData;
-
/* ----------------
* function prototypes for heap access method
*
@@ -146,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index cebaea097d..ff4f9e3d99 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -269,7 +269,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -683,6 +685,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -798,7 +803,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index ef09611e0d..2709a85b3c 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -259,7 +259,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -270,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 708160f645..5bac03168a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -46,6 +48,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -83,6 +89,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..538fd61fcb 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -101,6 +101,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +118,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +135,19 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/* Hash table control struct is an opaque type known only within dynahash.c */
+typedef struct HTAB HTAB;
+
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +156,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +232,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +258,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 7dd9d108d6..1eb3e319c5 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -88,6 +88,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 2e785c4cec..5cb2852f41 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ typedef FormData_pg_am *Form_pg_am;
* ----------------
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
/* ----------------
* initial contents of pg_am
@@ -83,5 +84,8 @@ DESCR("SP-GiST index access method");
DATA(insert OID = 3580 ( brin brinhandler i ));
DESCR("block range index (BRIN) access method");
#define BRIN_AM_OID 3580
+DATA(insert OID = 4002 ( pglz pglzhandler c ));
+DESCR("PGLZ compression access method");
+#define PGLZ_COMPRESSION_AM_OID 4002
#endif /* PG_AM_H */
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..47e739622c
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,72 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * the genbki.pl script reads this file and generates .bki
+ * information from the DATA() statements.
+ *
+ * XXX do NOT break up DATA() statements into multiple lines!
+ * the scripts are not as smart as you might think...
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+#define AttrCompressionRelationId 4001
+
+CATALOG(pg_attr_compression,4001) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid; /* attribute relation */
+ int16 acattnum; /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1]; /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression * Form_pg_attr_compression;
+
+/* ----------------
+ * compiler constants for pg_attr_compression
+ * ----------------
+ */
+#define Natts_pg_attr_compression 5
+#define Anum_pg_attr_compression_acoid 1
+#define Anum_pg_attr_compression_acname 2
+#define Anum_pg_attr_compression_acrelid 3
+#define Anum_pg_attr_compression_acattnum 4
+#define Anum_pg_attr_compression_acoptions 5
+
+/*
+ * Predefined compression options for builtin compression access methods
+ * It is safe to use Oids that equal to Oids of access methods, since
+ * these are just numbers and they are not using system Oid column.
+ *
+ * Note that predefined options should have 0 in acrelid and acattnum.
+ */
+
+DATA(insert ( 4002 pglz 0 0 _null_ ));
+#define PGLZ_AC_OID 4002
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 5bb64f7c31..e17a7be545 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -159,6 +159,9 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,10 +186,10 @@ CATALOG(pg_attribute,1249) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BKI_ROWTYPE_OID(75) BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
@@ -200,7 +203,7 @@ typedef FormData_pg_attribute *Form_pg_attribute;
* ----------------
*/
-#define Natts_pg_attribute 24
+#define Natts_pg_attribute 25
#define Anum_pg_attribute_attrelid 1
#define Anum_pg_attribute_attname 2
#define Anum_pg_attribute_atttypid 3
@@ -221,10 +224,11 @@ typedef FormData_pg_attribute *Form_pg_attribute;
#define Anum_pg_attribute_attislocal 18
#define Anum_pg_attribute_attinhcount 19
#define Anum_pg_attribute_attcollation 20
-#define Anum_pg_attribute_attacl 21
-#define Anum_pg_attribute_attoptions 22
-#define Anum_pg_attribute_attfdwoptions 23
-#define Anum_pg_attribute_attmissingval 24
+#define Anum_pg_attribute_attcompression 21
+#define Anum_pg_attribute_attacl 22
+#define Anum_pg_attribute_attoptions 23
+#define Anum_pg_attribute_attfdwoptions 24
+#define Anum_pg_attribute_attmissingval 25
/* ----------------
* initial contents of pg_attribute
diff --git a/src/include/catalog/pg_class.h b/src/include/catalog/pg_class.h
index 135f33d0f3..a96a73ea49 100644
--- a/src/include/catalog/pg_class.h
+++ b/src/include/catalog/pg_class.h
@@ -149,7 +149,7 @@ typedef FormData_pg_class *Form_pg_class;
*/
DATA(insert OID = 1247 ( pg_type PGNSP 71 0 PGUID 0 0 0 0 0 0 0 f f p r 30 0 t f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
-DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 24 0 f f f f f f t n f 0 3 1 _null_ _null_ _null_));
+DATA(insert OID = 1249 ( pg_attribute PGNSP 75 0 PGUID 0 0 0 0 0 0 0 f f p r 25 0 f f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
DATA(insert OID = 1255 ( pg_proc PGNSP 81 0 PGUID 0 0 0 0 0 0 0 f f p r 28 0 t f f f f f t n f 0 3 1 _null_ _null_ _null_));
DESCR("");
diff --git a/src/include/catalog/pg_proc.h b/src/include/catalog/pg_proc.h
index ec50afcdf0..9d20c1a4b2 100644
--- a/src/include/catalog/pg_proc.h
+++ b/src/include/catalog/pg_proc.h
@@ -576,6 +576,10 @@ DESCR("brin: standalone scan new table pages");
DATA(insert OID = 4014 ( brin_desummarize_range PGNSP PGUID 12 1 0 0 0 f f f t f v s 2 0 2278 "2205 20" _null_ _null_ _null_ _null_ _null_ brin_desummarize_range _null_ _null_ _null_ ));
DESCR("brin: desummarize page range");
+/* Compression access method handlers */
+DATA(insert OID = 3438 ( pglzhandler PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 4005 "2281" _null_ _null_ _null_ _null_ _null_ pglzhandler _null_ _null_ _null_ ));
+DESCR("pglz compression access method handler");
+
DATA(insert OID = 338 ( amvalidate PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 16 "26" _null_ _null_ _null_ _null_ _null_ amvalidate _null_ _null_ _null_ ));
DESCR("validate an operator class");
@@ -3859,6 +3863,8 @@ DATA(insert OID = 3454 ( pg_filenode_relation PGNSP PGUID 12 1 0 0 0 f f f t f s
DESCR("relation OID for filenode and tablespace");
DATA(insert OID = 3034 ( pg_relation_filepath PGNSP PGUID 12 1 0 0 0 f f f t f s s 1 0 25 "2205" _null_ _null_ _null_ _null_ _null_ pg_relation_filepath _null_ _null_ _null_ ));
DESCR("file path of relation");
+DATA(insert OID = 4008 ( pg_column_compression PGNSP PGUID 12 1 0 0 0 f f f t f s s 2 0 25 "2205 25" _null_ _null_ _null_ _null_ _null_ pg_column_compression _null_ _null_ _null_ ));
+DESCR("list of compression access methods used by the column");
DATA(insert OID = 2316 ( postgresql_fdw_validator PGNSP PGUID 12 1 0 0 0 f f f t f i s 2 0 16 "1009 26" _null_ _null_ _null_ _null_ _null_ postgresql_fdw_validator _null_ _null_ _null_));
DESCR("(internal)");
@@ -3931,6 +3937,10 @@ DATA(insert OID = 3311 ( tsm_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f i s 1
DESCR("I/O");
DATA(insert OID = 3312 ( tsm_handler_out PGNSP PGUID 12 1 0 0 0 f f f t f i s 1 0 2275 "3310" _null_ _null_ _null_ _null_ _null_ tsm_handler_out _null_ _null_ _null_ ));
DESCR("I/O");
+DATA(insert OID = 4006 ( compression_am_handler_in PGNSP PGUID 12 1 0 0 0 f f f f f i s 1 0 4005 "2275" _null_ _null_ _null_ _null_ _null_ compression_am_handler_in _null_ _null_ _null_ ));
+DESCR("I/O");
+DATA(insert OID = 4007 ( compression_am_handler_out PGNSP PGUID 12 1 0 0 0 f f f t f i s 1 0 2275 "4005" _null_ _null_ _null_ _null_ _null_ compression_am_handler_out _null_ _null_ _null_ ));
+DESCR("I/O");
/* tablesample method handlers */
DATA(insert OID = 3313 ( bernoulli PGNSP PGUID 12 1 0 0 0 f f f t f v s 1 0 3310 "2281" _null_ _null_ _null_ _null_ _null_ tsm_bernoulli_handler _null_ _null_ _null_ ));
diff --git a/src/include/catalog/pg_type.h b/src/include/catalog/pg_type.h
index 5b5b1218de..1b0283122b 100644
--- a/src/include/catalog/pg_type.h
+++ b/src/include/catalog/pg_type.h
@@ -712,6 +712,8 @@ DATA(insert OID = 3310 ( tsm_handler PGNSP PGUID 4 t p P f t \054 0 0 0 tsm_han
#define TSM_HANDLEROID 3310
DATA(insert OID = 3831 ( anyrange PGNSP PGUID -1 f p P f t \054 0 0 0 anyrange_in anyrange_out - - - - - d x f 0 -1 0 0 _null_ _null_ _null_ ));
#define ANYRANGEOID 3831
+DATA(insert OID = 4005 ( compression_am_handler PGNSP PGUID 4 t p P f t \054 0 0 0 compression_am_handler_in compression_am_handler_out - - - - - i p f 0 -1 0 0 _null_ _null_ _null_ ));
+#define COMPRESSION_AM_HANDLEROID 4005
/*
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8fc9e424cf..9acbebb0bc 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -154,8 +155,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 443de22704..3edf3bba81 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -471,6 +471,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -502,7 +503,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 92082b3a7a..72ac1ca3f2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -621,6 +621,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -644,6 +658,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -681,6 +696,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1788,7 +1804,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index cf32197bc3..773d896b9b 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index bbcb50e41f..abba43cb94 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..abd0674b02
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,379 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: table droptest column d1 depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to table droptest column d1
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 39a963888d..ed4e844d6c 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -425,11 +425,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -438,11 +438,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -600,10 +600,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -739,11 +739,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -752,11 +752,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -766,11 +766,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -803,46 +803,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -862,11 +862,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -875,10 +875,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index f4eebb75cf..11924864e9 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 7b54e96d30..9faf540236 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1330,12 +1330,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1351,12 +1351,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1375,12 +1375,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1418,12 +1418,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1443,17 +1443,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1475,17 +1475,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1517,17 +1517,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1555,12 +1555,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1592,12 +1592,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1636,12 +1636,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1667,12 +1667,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1694,12 +1694,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1721,12 +1721,12 @@ Inherits: fd_pt1
-- OID system column
ALTER TABLE fd_pt1 SET WITH OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1751,12 +1751,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE fd_pt1 SET WITHOUT OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1782,12 +1782,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1846,12 +1846,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1918,12 +1918,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1946,12 +1946,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1976,12 +1976,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2004,12 +2004,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index f56151fc1e..d4215af768 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 97419a744f..b33f8f93e2 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -432,11 +432,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -456,10 +456,10 @@ drop function dummy_hashint4(a int4, seed int8);
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -785,11 +785,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -801,74 +801,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 01608d2c04..bd85b93d3e 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1724,7 +1724,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0c86c647bc..a7744bb6f2 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 5149b72fe9..8b0baeb0c3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2801,11 +2801,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2821,11 +2821,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index ac0fb539e9..b23386ddb8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index d858a0e7db..2aa7aee62d 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -30,7 +30,7 @@ test: point lseg line box path polygon circle date time timetz timestamp timesta
# geometry depends on point, lseg, box, path, polygon and circle
# horology depends on interval, timetz, timestamp, timestamptz, reltime and abstime
# ----------
-test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions
+test: geometry horology regex oidjoins type_sanity opr_sanity misc_sanity comments expressions create_cm
# ----------
# These four each depend on the previous one
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 99f8ca37ba..6de740c0ee 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..15aa5cceac
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,185 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index a593d37643..9c2d8fe89d 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1161,7 +1161,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 17bf55c1f5..8f85146a62 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -138,6 +138,8 @@ AttoptCacheKey
AttrDefInfo
AttrDefault
AttrNumber
+AttributeCompressionInfo
+AttributeCompressionItem
AttributeOpts
AuthRequest
AutoPrewarmSharedState
@@ -343,6 +345,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -368,6 +371,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
On 30.03.2018 19:50, Ildus Kurbangaliev wrote:
On Mon, 26 Mar 2018 20:38:25 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:Attached rebased version of the patch. Fixed conflicts in pg_class.h.
New rebased version due to conflicts in master. Also fixed few errors
and removed cmdrop method since it couldnt be tested.
�I seems to be useful (and not so difficult) to use custom compression
methods also for WAL compression: replace direct calls of pglz_compress
in xloginsert.c
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:
On 30.03.2018 19:50, Ildus Kurbangaliev wrote:
On Mon, 26 Mar 2018 20:38:25 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:Attached rebased version of the patch. Fixed conflicts in pg_class.h.
New rebased version due to conflicts in master. Also fixed few errors
and removed cmdrop method since it couldnt be tested.
I seems to be useful (and not so difficult) to use custom compression
methods also for WAL compression: replace direct calls of pglz_compress in
xloginsert.c
I'm going to object this at point, and I've following arguments for that:
1) WAL compression is much more critical for durability than datatype
compression. Imagine, compression algorithm contains a bug which
cause decompress method to issue a segfault. In the case of datatype
compression, that would cause crash on access to some value which
causes segfault; but in the rest database will be working giving you
a chance to localize the issue and investigate that. In the case of
WAL compression, recovery would cause a server crash. That seems
to be much more serious disaster. You wouldn't be able to make
your database up and running and the same happens on the standby.
2) Idea of custom compression method is that some columns may
have specific data distribution, which could be handled better with
particular compression method and particular parameters. In the
WAL compression you're dealing with the whole WAL stream containing
all the values from database cluster. Moreover, if custom compression
method are defined for columns, then in WAL stream you've values
already compressed in the most efficient way. However, it might
appear that some compression method is better for WAL in general
case (there are benchmarks showing our pglz is not very good in
comparison to the alternatives). But in this case I would prefer to just
switch our WAL to different compression method one day. Thankfully
we don't preserve WAL compatibility between major releases.
3) This patch provides custom compression methods recorded in
the catalog. During recovery you don't have access to the system
catalog, because it's not recovered yet, and can't fetch compression
method metadata from there. The possible thing is to have GUC,
which stores shared module and function names for WAL compression.
But that seems like quite different mechanism from the one present
in this patch.
Taking into account all of above, I think we would give up with custom
WAL compression method. Or, at least, consider it unrelated to this
patch.
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Sun, 22 Apr 2018 16:21:31 +0300
Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:On 30.03.2018 19:50, Ildus Kurbangaliev wrote:
On Mon, 26 Mar 2018 20:38:25 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:Attached rebased version of the patch. Fixed conflicts in
pg_class.h.New rebased version due to conflicts in master. Also fixed few
errorsand removed cmdrop method since it couldnt be tested.
I seems to be useful (and not so difficult) to use custom
compressionmethods also for WAL compression: replace direct calls of
pglz_compress in xloginsert.cI'm going to object this at point, and I've following arguments for
that:1) WAL compression is much more critical for durability than datatype
compression. Imagine, compression algorithm contains a bug which
cause decompress method to issue a segfault. In the case of datatype
compression, that would cause crash on access to some value which
causes segfault; but in the rest database will be working giving you
a chance to localize the issue and investigate that. In the case of
WAL compression, recovery would cause a server crash. That seems
to be much more serious disaster. You wouldn't be able to make
your database up and running and the same happens on the standby.2) Idea of custom compression method is that some columns may
have specific data distribution, which could be handled better with
particular compression method and particular parameters. In the
WAL compression you're dealing with the whole WAL stream containing
all the values from database cluster. Moreover, if custom compression
method are defined for columns, then in WAL stream you've values
already compressed in the most efficient way. However, it might
appear that some compression method is better for WAL in general
case (there are benchmarks showing our pglz is not very good in
comparison to the alternatives). But in this case I would prefer to
just switch our WAL to different compression method one day.
Thankfully we don't preserve WAL compatibility between major releases.3) This patch provides custom compression methods recorded in
the catalog. During recovery you don't have access to the system
catalog, because it's not recovered yet, and can't fetch compression
method metadata from there. The possible thing is to have GUC,
which stores shared module and function names for WAL compression.
But that seems like quite different mechanism from the one present
in this patch.Taking into account all of above, I think we would give up with custom
WAL compression method. Or, at least, consider it unrelated to this
patch.
I agree with these points. I also think this should be done in another
patch. It's not so hard to implement but would make sense if there will
be few more builtin compression methods suitable for wal compression.
Some static array could contain function pointers for direct calls.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 22.04.2018 16:21, Alexander Korotkov wrote:
On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik
<k.knizhnik@postgrespro.ru <mailto:k.knizhnik@postgrespro.ru>> wrote:On 30.03.2018 19:50, Ildus Kurbangaliev wrote:
On Mon, 26 Mar 2018 20:38:25 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru
<mailto:i.kurbangaliev@postgrespro.ru>> wrote:Attached rebased version of the patch. Fixed conflicts in
pg_class.h.New rebased version due to conflicts in master. Also fixed few
errors
and removed cmdrop method since it couldnt be tested.I seems to be useful (and not so difficult) to use custom
compression methods also for WAL compression: replace direct calls
of pglz_compress in xloginsert.cI'm going to object this at point, and I've following arguments for that:
1) WAL compression is much more critical for durability than datatype
compression. Imagine, compression algorithm contains a bug which
cause decompress method to issue a segfault. In the case of datatype
compression, that would cause crash on access to some value which
causes segfault; but in the rest database will be working giving you
a chance to localize the issue and investigate that. In the case of
WAL compression, recovery would cause a server crash. That seems
to be much more serious disaster. You wouldn't be able to make
your database up and running and the same happens on the standby.
Well, I do not think that somebody will try to implement its own
compression algorithm...
From my point of view the main value of this patch is that it allows to
replace pglz algorithm with more efficient one, for example zstd.
At some data sets zstd provides more than 10 times better compression
ratio and at the same time is faster then pglz.
I do not think that risk of data corruption caused by WAL compression
with some alternative compression algorithm (zlib, zstd,...) is higher
than in case of using builtin Postgres compression.
2) Idea of custom compression method is that some columns may
have specific data distribution, which could be handled better with
particular compression method and particular parameters. In the
WAL compression you're dealing with the whole WAL stream containing
all the values from database cluster. Moreover, if custom compression
method are defined for columns, then in WAL stream you've values
already compressed in the most efficient way. However, it might
appear that some compression method is better for WAL in general
case (there are benchmarks showing our pglz is not very good in
comparison to the alternatives). But in this case I would prefer to just
switch our WAL to different compression method one day. Thankfully
we don't preserve WAL compatibility between major releases.
Frankly speaking I do not believe that somebody will use custom
compression in this way: implement its own compression methods for the
specific data type.
May be just for json/jsonb, but also only in the case when custom
compression API allows to separately store compression dictionary (which
as far as I understand is not currently supported).
When I worked for SciDB (database for scientists which has to deal
mostly with multidimensional arrays of data) our first intention was to
implement custom compression methods for the particular data types and
data distributions. For example, there are very fast, simple and
efficient algorithms for encoding sequence of monotonically increased
integers, ....
But after several experiments we rejected this idea and switch to using
generic compression methods. Mostly because we do not want compressor to
know much about page layout, data type representation,... In Postgres,
from my point of view, we have similar situation. Assume that we have
column of serial type. So it is good candidate of compression, isn't it?
But this approach deals only with particular attribute values. It can
not take any advantages from the fact that this particular column is
monotonically increased. It can be done only with page level
compression, but it is a different story.
So current approach works only for blob-like types: text, json,... But
them usually have quite complex internal structure and for them
universal compression algorithms used to be more efficient than any
hand-written specific implementation. Also algorithms like zstd, are
able to efficiently recognize and compress many common data
distributions, line monotonic sequences, duplicates, repeated series,...
3) This patch provides custom compression methods recorded in
the catalog. During recovery you don't have access to the system
catalog, because it's not recovered yet, and can't fetch compression
method metadata from there. The possible thing is to have GUC,
which stores shared module and function names for WAL compression.
But that seems like quite different mechanism from the one present
in this patch.
I do not think that assignment default compression method through GUC is
so bad idea.
Taking into account all of above, I think we would give up with custom
WAL compression method. Or, at least, consider it unrelated to this
patch.
Sorry for repeating the same thing, but from my point of view the main
advantage of this patch is that it allows to replace pglz with more
efficient compression algorithms.
I do not see much sense in specifying custom compression method for some
particular columns.
It will be more useful from my point of view to include in this patch
implementation of compression API not only or pglz, but also for zlib,
zstd and may be for some other popular compressing libraries which
proved their efficiency.
Postgres already has zlib dependency (unless explicitly excluded with
--without-zlib), so zlib implementation can be included in Postgres build.
Other implementations can be left as module which customer can build
himself. It is certainly less convenient, than using preexistred stuff,
but much more convenient then making users to write this code themselves.
There is yet another aspect which is not covered by this patch:
streaming compression.
Streaming compression is needed if we want to compress libpq traffic. It
can be very efficient for COPY command and for replication. Also libpq
compression can improve speed of queries returning large results (for
example containing JSON columns) throw slow network.
I have proposed such patch for libpq, which is using either zlib,
either zstd streaming API. Postgres built-in compression implementation
doesn't have streaming API at all, so it can not be used here. Certainly
support of streaming may significantly complicates compression API, so
I am not sure that it actually needed to be included in this patch.
But I will be pleased if Ildus can consider this idea.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, Apr 23, 2018 at 12:40 PM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:
On 22.04.2018 16:21, Alexander Korotkov wrote:
On Fri, Apr 20, 2018 at 7:45 PM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:On 30.03.2018 19:50, Ildus Kurbangaliev wrote:
On Mon, 26 Mar 2018 20:38:25 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:Attached rebased version of the patch. Fixed conflicts in pg_class.h.
New rebased version due to conflicts in master. Also fixed few errors
and removed cmdrop method since it couldnt be tested.
I seems to be useful (and not so difficult) to use custom compression
methods also for WAL compression: replace direct calls of pglz_compress in
xloginsert.cI'm going to object this at point, and I've following arguments for that:
1) WAL compression is much more critical for durability than datatype
compression. Imagine, compression algorithm contains a bug which
cause decompress method to issue a segfault. In the case of datatype
compression, that would cause crash on access to some value which
causes segfault; but in the rest database will be working giving you
a chance to localize the issue and investigate that. In the case of
WAL compression, recovery would cause a server crash. That seems
to be much more serious disaster. You wouldn't be able to make
your database up and running and the same happens on the standby.Well, I do not think that somebody will try to implement its own
compression algorithm...
But that the main goal of this patch: let somebody implement own compression
algorithm which best fit for particular dataset.
From my point of view the main value of this patch is that it allows to
replace pglz algorithm with more efficient one, for example zstd.
At some data sets zstd provides more than 10 times better compression
ratio and at the same time is faster then pglz.
Not exactly. If we want to replace pglz with more efficient one, then we
should
just replace pglz with better algorithm. Pluggable compression methods are
definitely don't worth it for just replacing pglz with zstd.
I do not think that risk of data corruption caused by WAL compression with
some alternative compression algorithm (zlib, zstd,...) is higher than in
case of using builtin Postgres compression.
It speaking about zlib or zstd, then yes risk of corruption is very low.
But again,
switching to zlib or zstd don't justify this patch.
2) Idea of custom compression method is that some columns may
have specific data distribution, which could be handled better with
particular compression method and particular parameters. In the
WAL compression you're dealing with the whole WAL stream containing
all the values from database cluster. Moreover, if custom compression
method are defined for columns, then in WAL stream you've values
already compressed in the most efficient way. However, it might
appear that some compression method is better for WAL in general
case (there are benchmarks showing our pglz is not very good in
comparison to the alternatives). But in this case I would prefer to just
switch our WAL to different compression method one day. Thankfully
we don't preserve WAL compatibility between major releases.Frankly speaking I do not believe that somebody will use custom
compression in this way: implement its own compression methods for the
specific data type.
May be just for json/jsonb, but also only in the case when custom
compression API allows to separately store compression dictionary (which as
far as I understand is not currently supported).When I worked for SciDB (database for scientists which has to deal mostly
with multidimensional arrays of data) our first intention was to implement
custom compression methods for the particular data types and data
distributions. For example, there are very fast, simple and efficient
algorithms for encoding sequence of monotonically increased integers, ....
But after several experiments we rejected this idea and switch to using
generic compression methods. Mostly because we do not want compressor to
know much about page layout, data type representation,... In Postgres, from
my point of view, we have similar situation. Assume that we have column of
serial type. So it is good candidate of compression, isn't it?
No, it's not. Exactly because compressor shouldn't deal with page layout
etc.
But it's absolutely OK for datatype compressor to deal with particular type
representation.
But this approach deals only with particular attribute values. It can not
take any advantages from the fact that this particular column is
monotonically increased. It can be done only with page level compression,
but it is a different story.
Yes, compression of data series spear across multiple rows is different
story.
So current approach works only for blob-like types: text, json,... But
them usually have quite complex internal structure and for them universal
compression algorithms used to be more efficient than any hand-written
specific implementation. Also algorithms like zstd, are able to efficiently
recognize and compress many common data distributions, line monotonic
sequences, duplicates, repeated series,...
Some types blob-like datatypes might be not long enough to let generic
compression algorithms like zlib or zstd train a dictionary. For example,
MySQL successfully utilize column-level dictionaries for JSON [1]. Also
JSON(B) might utilize some compression which let user extract
particular attributes without decompression of the whole document.
3) This patch provides custom compression methods recorded in
the catalog. During recovery you don't have access to the system
catalog, because it's not recovered yet, and can't fetch compression
method metadata from there. The possible thing is to have GUC,
which stores shared module and function names for WAL compression.
But that seems like quite different mechanism from the one present
in this patch.I do not think that assignment default compression method through GUC is
so bad idea.
It's probably not so bad, but it's a different story. Unrelated to this
patch, I think.
Taking into account all of above, I think we would give up with custom
WAL compression method. Or, at least, consider it unrelated to this
patch.Sorry for repeating the same thing, but from my point of view the main
advantage of this patch is that it allows to replace pglz with more
efficient compression algorithms.
I do not see much sense in specifying custom compression method for some
particular columns.
This patch is about giving user an ability to select particular compression
method and its parameters for particular column.
It will be more useful from my point of view to include in this patch
implementation of compression API not only or pglz, but also for zlib, zstd
and may be for some other popular compressing libraries which proved their
efficiency.Postgres already has zlib dependency (unless explicitly excluded with
--without-zlib), so zlib implementation can be included in Postgres build.
Other implementations can be left as module which customer can build
himself. It is certainly less convenient, than using preexistred stuff, but
much more convenient then making users to write this code themselves.There is yet another aspect which is not covered by this patch: streaming
compression.
Streaming compression is needed if we want to compress libpq traffic. It
can be very efficient for COPY command and for replication. Also libpq
compression can improve speed of queries returning large results (for
example containing JSON columns) throw slow network.
I have proposed such patch for libpq, which is using either zlib, either
zstd streaming API. Postgres built-in compression implementation doesn't
have streaming API at all, so it can not be used here. Certainly support of
streaming may significantly complicates compression API, so I am not sure
that it actually needed to be included in this patch.
But I will be pleased if Ildus can consider this idea.
I think streaming compression seems like a completely different story.
client-server traffic compression is not just server feature. It must
be also supported at client side. And I really doubt it should be
pluggable.
In my opinion, you propose good things like compression of WAL
with better algorithm and compression of client-server traffic.
But I think those features are unrelated to this patch and should
be considered separately. It's not features, which should be
added to this patch. Regarding this patch the points you provided
more seems like criticism of the general idea.
I think the problem of this patch is that it lacks of good example.
It would be nice if Ildus implement simple compression with
column-defined dictionary (like [1] does), and show its efficiency
of real-life examples, which can't be achieved with generic
compression methods (like zlib or zstd). That would be a good
answer to the criticism you provide.
*Links*
1.
https://www.percona.com/doc/percona-server/LATEST/flexibility/compressed_columns.html
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 23.04.2018 18:32, Alexander Korotkov wrote:
But that the main goal of this patch: let somebody implement own
compression
algorithm which best fit for particular dataset.
Hmmm...Frankly speaking I don't believe in this "somebody".
From my point of view the main value of this patch is that it
allows to replace pglz algorithm with more efficient one, for
example zstd.
At some data sets zstd provides more than 10 times better
compression ratio and at the same time is faster then pglz.Not exactly. If we want to replace pglz with more efficient one, then
we should
just replace pglz with better algorithm. Pluggable compression
methods are
definitely don't worth it for just replacing pglz with zstd.
As far as I understand it is not possible for many reasons (portability,
patents,...) to replace pglz with zstd.
I think that even replacing pglz with zlib (which is much worser than
zstd) will not be accepted by community.
So from my point of view the main advantage of custom compression method
is to replace builting pglz compression with more advanced one.
Some types blob-like datatypes might be not long enough to let generic
compression algorithms like zlib or zstd train a dictionary. For example,
MySQL successfully utilize column-level dictionaries for JSON [1]. Also
JSON(B) might utilize some compression which let user extract
particular attributes without decompression of the whole document.
Well, I am not an expert in compression.
But I will be very surprised if somebody will show me some real example
with large enough compressed data buffer (>2kb) where some specialized
algorithm will provide significantly
better compression ratio than advanced universal compression algorithm.
Also may be I missed something, but current compression API doesn't
support partial extraction (extra some particular attribute or range).
If we really need it, then it should be expressed in custom compressor
API. But I am not sure how frequently it will needed.
Large values are splitted into 2kb TOAST chunks. With compression it can
be about 4-8k of raw data. IMHO storing larger JSON objects is database
design flaw.
And taken in account that in JSONB we need also extract header (so at
least two chunks), it makes more obscure advantages of partial JSONB
decompression.
I do not think that assignment default compression method through
GUC is so bad idea.It's probably not so bad, but it's a different story. Unrelated to
this patch, I think.
May be. But in any cases, there are several direction where compression
can be used:
- custom compression algorithms
- libpq compression
- page level compression
...
and them should be somehow finally "married" with each other.
I think streaming compression seems like a completely different story.
client-server traffic compression is not just server feature. It must
be also supported at client side. And I really doubt it should be
pluggable.In my opinion, you propose good things like compression of WAL
with better algorithm and compression of client-server traffic.
But I think those features are unrelated to this patch and should
be considered separately. It's not features, which should be
added to this patch. Regarding this patch the points you provided
more seems like criticism of the general idea.I think the problem of this patch is that it lacks of good example.
It would be nice if Ildus implement simple compression with
column-defined dictionary (like [1] does), and show its efficiency
of real-life examples, which can't be achieved with generic
compression methods (like zlib or zstd). That would be a good
answer to the criticism you provide.*Links*
1.
https://www.percona.com/doc/percona-server/LATEST/flexibility/compressed_columns.html------
Alexander Korotkov
Postgres Professional:http://www.postgrespro.com
<http://www.postgrespro.com/>
The Russian Postgres Company
Sorry, I really looking at this patch under the different angle.
And this is why I have some doubts about general idea.
Postgres allows to defined custom types, access methods,...
But do you know any production system using some special data types or
custom indexes which are not included in standard Postgres distribution
or popular extensions (like postgis)?
IMHO end-user do not have skills and time to create their own
compression algorithms. And without knowledge of specific of particular
data set,
it is very hard to implement something more efficient than universal
compression library.
But if you think that it is not a right place and time to discuss it, I
do not insist.
But in any case, I think that it will be useful to provide some more
examples of custom compression API usage.
From my point of view the most useful will be integration with zstd.
But if it is possible to find some example of data-specific compression
algorithms which show better results than universal compression,
it will be even more impressive.
--
Konstantin Knizhnik
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, 23 Apr 2018 19:34:38 +0300
Konstantin Knizhnik <k.knizhnik@postgrespro.ru> wrote:
Sorry, I really looking at this patch under the different angle.
And this is why I have some doubts about general idea.
Postgres allows to defined custom types, access methods,...
But do you know any production system using some special data types
or custom indexes which are not included in standard Postgres
distribution or popular extensions (like postgis)?IMHO end-user do not have skills and time to create their own
compression algorithms. And without knowledge of specific of
particular data set,
it is very hard to implement something more efficient than universal
compression library.
But if you think that it is not a right place and time to discuss it,
I do not insist.But in any case, I think that it will be useful to provide some more
examples of custom compression API usage.
From my point of view the most useful will be integration with zstd.
But if it is possible to find some example of data-specific
compression algorithms which show better results than universal
compression, it will be even more impressive.
Ok, let me clear up the purpose of this patch. I understand that you
want to compress everything by it but now the idea is just to bring
basic functionality to compress toasting values with external
compression algorithms. It's unlikely that compression algorithms like
zstd, snappy and others will be in postgres core but with this patch
it's really easy to make an extension and start to compress values
using it right away. And the end-user should not be expert in
compression algorithms to make such extension. One of these algorithms
could end up in core if its license will allow it.
I'm not trying to cover all the places in postgres which will benefit
from compression, and this patch only is the first step. It's quite big
already and with every new feature that will increase its size, chances
of its reviewing and commiting will decrease.
The API is very simple now and contains what an every compression
method can do - get some block of data and return a compressed form
of the data. And it can be extended with streaming and other features in
the future.
Maybe the reason of your confusion is that there is no GUC that changes
pglz to some custom compression so all new attributes will use it. I
will think about adding it. Also there was a discussion about
specifying the compression for the type and it was decided that's
better to do it later by a separate patch.
As an example of specialized compression could be time series
compression described in [1]http://www.vldb.org/pvldb/vol8/p1816-teller.pdf. [2]https://github.com/zilder/pg_lz4 contains an example of an extension
that adds lz4 compression using this patch.
[1]: http://www.vldb.org/pvldb/vol8/p1816-teller.pdf
[2]: https://github.com/zilder/pg_lz4
--
----
Regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
On Mon, Apr 23, 2018 at 7:34 PM, Konstantin Knizhnik <
k.knizhnik@postgrespro.ru> wrote:
IMHO end-user do not have skills and time to create their own compression
algorithms. And without knowledge of specific of particular data set,
it is very hard to implement something more efficient than universal
compression library.
But if you think that it is not a right place and time to discuss it, I do
not insist.
For sure, end-users wouldn't implement own compression algorithms.
In the same way as end-users wouldn't implement custom datatypes,
operator classes, procedural language handlers etc. But those are
useful extension mechanisms which pass test of time. And extension
developers use them.
But in any case, I think that it will be useful to provide some more
examples of custom compression API usage.
From my point of view the most useful will be integration with zstd.
But if it is possible to find some example of data-specific compression
algorithms which show better results than universal compression,
it will be even more impressive.
Yes, this patch definitely lacks of good usage example. That may
lead to some misunderstanding of its purpose. Good use-cases
should be shown before we can consider committing this. I think
Ildus should try to implement at least custom dictionary compression
method where dictionary is specified by user in parameters.
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Tue, 24 Apr 2018 14:05:20 +0300
Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
Yes, this patch definitely lacks of good usage example. That may
lead to some misunderstanding of its purpose. Good use-cases
should be shown before we can consider committing this. I think
Ildus should try to implement at least custom dictionary compression
method where dictionary is specified by user in parameters.
Hi,
attached v16 of the patch. I have splitted the patch to 8 parts so now
it should be easier to make a review. The main improvement is zlib
compression method with dictionary support like you mentioned. My
synthetic tests showed that zlib gives more compression but usually
slower than pglz.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
0002-Add-compression-catalog-tables-and-the-basic-inf-v16.patchtext/x-patchDownload
From 1546ece3a7b5c735eb046f011b012a76974233f3 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 3 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 4 +-
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 84 +++
src/backend/commands/compressioncmds.c | 653 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 3 +-
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 3 +-
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
46 files changed, 2129 insertions(+), 195 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index aa52a96259..9a1cebbe2f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index e0c9c3431c..7880b0ad9e 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -962,11 +962,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 72395a50b8..fdac4bbb67 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2684,8 +2684,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -4129,7 +4130,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 8d3c861a33..fcaa128d8c 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -655,7 +655,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index cd42c50b09..413b0b2b9c 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +61,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -421,7 +462,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +552,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +655,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +667,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +802,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +886,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +901,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +919,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1064,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1353,6 +1504,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1368,54 +1531,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1666,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1687,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1699,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2059,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2244,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2440,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2561,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate caches so we need to call it before
+ * we're putting something to cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreateExtended(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 7e34bee63e..d78ef2600c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,6 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 0865240f11..5560fc05b8 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 4f1d365357..61ebd316ba 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -170,7 +171,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1280,6 +1282,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2537,6 +2543,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d59bd5bb00..e13531b0f1 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -633,6 +633,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1606,6 +1607,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8b276bc430..e55a951f78 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -413,8 +413,9 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
+ to->attcompression = InvalidOid;
to->attcollation = (i < numkeyatts) ?
- collationObjectId[i] : InvalidOid;
+ collationObjectId[i] : InvalidOid;
ReleaseSysCache(tuple);
@@ -501,6 +502,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ad682673e6..d3787e71c4 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -489,6 +490,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3546,6 +3559,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4070,6 +4114,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4614,6 +4662,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3768a38459
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,653 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index eecc85d14e..ee67d07b2f 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1209,6 +1209,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8b848f91a7..2c3f1187f5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -22,6 +22,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -33,6 +34,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -40,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -89,6 +92,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -367,6 +371,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -465,6 +471,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -519,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -693,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -739,6 +750,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -787,6 +805,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2104,6 +2139,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2131,6 +2179,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2340,6 +2389,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3427,6 +3483,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3902,6 +3959,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4276,6 +4334,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4341,8 +4404,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5504,6 +5568,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5678,6 +5752,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5800,6 +5875,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5822,6 +5921,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6658,6 +6857,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9559,6 +9762,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9659,7 +9868,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9684,6 +9894,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9692,6 +9930,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12720,6 +12963,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1c12075b01..3009d29cce 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2859,6 +2859,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2878,6 +2879,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5553,6 +5566,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 6a971d0141..06b34a032c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2549,6 +2549,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2568,6 +2569,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3627,6 +3638,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index a10014f755..07ec871980 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3648,6 +3648,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3655,6 +3657,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 979d523e00..06097965c7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2859,6 +2859,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2876,6 +2877,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4181,6 +4192,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 17b54b20cc..2b909fc949 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1061,6 +1061,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index e2f59bf580..c4cc4286a2 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -2915,7 +2915,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 4022c14a83..c9302347e8 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -259,7 +259,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -270,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..793367ff58 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 24915824ca..dc41e50bd8 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -87,6 +87,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..563aa65a2a
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index dc36753ede..08e074ea4d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index 9fffdef379..49643b934e 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -36,7 +36,7 @@
reloftype => '0', relowner => 'PGUID', relam => '0', relfilenode => '0',
reltablespace => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasoids => 'f', relhasrules => 'f', relhastriggers => 'f',
relhassubclass => 'f', relrowsecurity => 'f', relforcerowsecurity => 'f',
relispopulated => 't', relreplident => 'n', relispartition => 'f',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 66c6c224a8..50682e07d1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6915,6 +6915,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7084,6 +7087,13 @@
{ oid => '3312', descr => 'I/O',
proname => 'tsm_handler_out', prorettype => 'cstring',
proargtypes => 'tsm_handler', prosrc => 'tsm_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 48e01cd694..722b11674f 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -929,6 +929,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
typcategory => 'P', typinput => 'tsm_handler_in',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8fc9e424cf..9acbebb0bc 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -154,8 +155,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 03cee8ec88..c54dfc7462 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -505,7 +505,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b596fcb513..916c75c7d8 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 54850ee4d6..5f672889d6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -350,6 +350,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -375,6 +376,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.17.1
0003-Add-rewrite-rules-and-tupdesc-flags-v16.patchtext/x-patchDownload
From d50b44dba3e6acc04f789991a35a02e06d3f7345 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 68 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 156 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 104172184f..1c76cbe85b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -215,6 +215,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -245,6 +246,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -283,6 +287,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -306,6 +319,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -342,7 +358,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -366,6 +382,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -376,6 +393,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -794,6 +812,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -918,6 +937,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -936,6 +956,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -988,6 +1009,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].ammissing,
false);
}
@@ -998,6 +1020,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1153,6 +1176,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1856,6 +1880,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 9a1cebbe2f..38fbdf7205 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 2658399484..b270f5e903 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -74,6 +75,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -96,8 +98,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -138,6 +149,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -302,6 +315,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -468,6 +484,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -553,6 +571,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -659,6 +678,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index fdac4bbb67..0917019f72 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -96,7 +96,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2361,13 +2362,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2454,7 +2456,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2623,7 +2625,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2685,8 +2687,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2725,7 +2730,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4028,6 +4033,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 413b0b2b9c..f75ca6d9b2 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1197,6 +1197,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1425,6 +1426,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 3a66cb5025..253a454fbf 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2555,7 +2555,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index e1eb7c374b..969c8160c2 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2c3f1187f5..11947942cf 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -166,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -4635,7 +4637,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4906,6 +4908,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -9007,6 +9027,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9902,7 +9961,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -9910,7 +9969,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -13022,7 +13081,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index b1b6883c19..b0e6dfc996 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -814,6 +814,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index d85dc92505..b1405e6165 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -577,6 +577,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ca5cad7497..755682a4f8 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -22,6 +22,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -146,7 +147,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 1867a70f6f..9e255154ad 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -273,7 +273,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -686,6 +688,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -801,7 +806,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 708160f645..5bac03168a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -46,6 +48,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -83,6 +89,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 793367ff58..50f20611ab 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.17.1
0001-Make-syntax-changes-for-custom-compression-metho-v16.patchtext/x-patchDownload
From 339ee4a258de2b693fdea5ecd9d3fc5b5204d945 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
---
src/backend/parser/gram.y | 76 ++++++++++++++++++++++++++++++----
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 ++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 89 insertions(+), 9 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 90dfac2cb1..ab8fa0b7dd 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -399,6 +400,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -585,6 +587,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -617,9 +623,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2214,6 +2220,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3380,11 +3395,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3394,8 +3410,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3442,6 +3458,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3647,6 +3700,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5324,12 +5378,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -15031,6 +15090,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 26cb234987..a3b75c5256 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -51,6 +51,7 @@ typedef FormData_pg_am *Form_pg_am;
* Allowed values for amtype
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 43f1552241..03cee8ec88 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -473,6 +473,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 6390f7e8c1..69a03d4388 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,6 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -683,6 +698,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1790,7 +1806,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..76c33cbc5f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.17.1
0004-Add-pglz-compression-method-v16.patchtext/x-patchDownload
From 13327b233f5e8f507ad0c3722be012fbfa81fe40 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index bef53a319a..6f7ad79613 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -30,5 +30,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 50682e07d1..7ae891d569 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -956,6 +956,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.17.1
0005-Add-zlib-compression-method-v16.patchtext/x-patchDownload
From d7a6f846fe33a4f13e98448f51dbc98b3327434b Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 25af514fba..855be8203c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6f7ad79613..b5754b9759 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 7ae891d569..5b4e0c82d3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -961,6 +961,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.17.1
0006-Add-psql-pg_dump-and-pg_upgrade-support-v16.patchtext/x-patchDownload
From 94c17da56fde56e232e70284e72814cf07c48f09 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
---
src/bin/pg_dump/pg_backup.h | 5 +
src/bin/pg_dump/pg_backup_archiver.c | 1 +
src/bin/pg_dump/pg_dump.c | 339 ++++++++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 28 ++-
src/bin/pg_dump/pg_dump_sort.c | 10 +-
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 144 +++++++++++-
src/bin/psql/describe.c | 47 +++-
src/bin/psql/tab-complete.c | 5 +-
10 files changed, 569 insertions(+), 18 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ceedd481fb..b33fcbc168 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 83c976eaf7..319d78b548 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -180,6 +180,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ea2f022eee..1ba564c781 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -377,6 +379,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -597,6 +600,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -807,6 +813,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4212,6 +4221,233 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
free(qsubname);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8103,6 +8339,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attoptions;
int i_attcollation;
int i_attfdwoptions;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8132,7 +8370,46 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 100000)
+ if (fout->remoteVersion >= 110000)
+ {
+ /*
+ * attcompression is new in version 11
+ */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw options */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions, "
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname "
+ /* FROM */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 100000)
{
/*
* attidentity is new in version 10.
@@ -8151,7 +8428,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8177,7 +8456,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"' ' || pg_catalog.quote_literal(option_value) "
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
- "), E',\n ') AS attfdwoptions "
+ "), E',\n ') AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8201,7 +8482,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"array_to_string(a.attoptions, ', ') AS attoptions, "
"CASE WHEN a.attcollation <> t.typcollation "
"THEN a.attcollation ELSE 0 END AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8219,7 +8502,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"array_to_string(a.attoptions, ', ') AS attoptions, "
"0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8236,7 +8521,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attlen, a.attalign, a.attislocal, "
"pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
"'' AS attoptions, 0 AS attcollation, "
- "NULL AS attfdwoptions "
+ "NULL AS attfdwoptions, "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8266,6 +8553,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attoptions = PQfnumber(res, "attoptions");
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8282,6 +8571,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8309,6 +8600,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attoptions[j] = pg_strdup(PQgetvalue(res, j, i_attoptions));
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9799,6 +10092,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12561,6 +12857,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15486,6 +15785,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15544,6 +15851,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17827,6 +18153,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e96c662b1e..86f0d47ce3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -316,6 +317,8 @@ typedef struct _tableInfo
char **attoptions; /* per-attribute options */
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -624,6 +627,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -724,5 +749,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index d2b0949d6b..6ef978a50c 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 54db0cd174..41a05bb383 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -135,6 +136,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -404,6 +406,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -618,6 +622,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 2342a63571..b78c6e8ed0 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 7eee870259..ed41cc9b20 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -631,6 +631,62 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1381,6 +1437,39 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2214,9 +2303,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2256,7 +2345,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like =>
@@ -2366,7 +2455,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2384,7 +2473,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -2401,6 +2490,49 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index e5b3c1ebf9..d0892e1e23 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1755,6 +1755,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1871,6 +1887,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1968,6 +1989,28 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, firstvcol + 1);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1976,7 +2019,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1987,7 +2030,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 7bb47eadc6..86b6240ebc 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2161,11 +2161,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
--
2.17.1
0007-Add-tests-for-compression-methods-v16.patchtext/x-patchDownload
From 3badeb595533f5cffaa99bd3c74d42d4a418e7c9 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 398 ++++++++++++++++++
src/test/regress/expected/create_table.out | 132 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 282 ++++++-------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 +++---
src/test/regress/expected/opr_sanity.out | 4 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 198 +++++++++
src/test/regress/sql/opr_sanity.sql | 4 +-
20 files changed, 1066 insertions(+), 461 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..357fbd85d8
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,398 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+(1 row)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 470fca0cab..a41d8378c3 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -415,11 +415,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -428,11 +428,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -590,10 +590,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -729,11 +729,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -742,11 +742,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -756,11 +756,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -793,46 +793,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -864,11 +864,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -877,10 +877,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
@@ -890,10 +890,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 0b5a9041b0..36151d0b06 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 339a43ff9e..668828e21f 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1337,12 +1337,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1358,12 +1358,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1382,12 +1382,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1425,12 +1425,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1450,17 +1450,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1482,17 +1482,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1524,17 +1524,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1562,12 +1562,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1599,12 +1599,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1643,12 +1643,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1674,12 +1674,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1701,12 +1701,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1728,12 +1728,12 @@ Inherits: fd_pt1
-- OID system column
ALTER TABLE fd_pt1 SET WITH OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1758,12 +1758,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE fd_pt1 SET WITHOUT OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1789,12 +1789,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1853,12 +1853,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1898,12 +1898,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1925,12 +1925,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1953,12 +1953,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1983,12 +1983,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2011,12 +2011,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index b2b912ed5c..41ad519632 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 5edf269367..ef1479f395 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index a1e18a6ceb..19f476a787 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1725,7 +1725,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index ae0cd253d5..9f9adc3dcc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2803,11 +2803,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2823,11 +2823,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 0aa5357917..4395152a94 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -105,6 +107,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 16f979c8d9..b8473c0568 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 42632be675..a4e3a771e2 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..0e500e6bf8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,198 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index a593d37643..9c2d8fe89d 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1161,7 +1161,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
--
2.17.1
0008-Add-documentation-for-custom-compression-methods-v16.patchtext/x-patchDownload
From 1fef4881228af515f01b097463390c874a54120c Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 145 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 203 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3ed9021c2f..216f8dd7dc 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..8950c5d1ec
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,145 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Method Interface Definition</title>
+ <para>
+ This chapter defines the interface between the core
+ <productname>PostgreSQL</productname> system and <firstterm>compression access
+ methods</firstterm>, which manage individual compression types.
+ </para>
+
+ <sect1 id="compression-api">
+ <title>Basic API Structure for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cmdrop_function cmdrop; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+<programlisting>
+void
+cmdrop (Oid acoid);
+</programlisting>
+ Called when the attribute compression is going to be removed.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any <structname>pg_attr_compression</structname> relation invalidation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index f010cd4c3b..78df1cf2be 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 24c3405f91..94ea769cfc 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 0070603fc3..d957f9b08b 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -252,6 +252,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 1cce00eaf9..5ecd2d4b69 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 2a1eac9592..fecaf7c5fa 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -860,6 +861,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 8ef2ac8010..4524eb65fc 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.17.1
Re: [HACKERS] Custom compression methods
On Mon, Apr 23, 2018 at 12:34 PM, Konstantin Knizhnik
<k.knizhnik@postgrespro.ru> wrote:
May be. But in any cases, there are several direction where compression can
be used:
- custom compression algorithms
- libpq compression
- page level compression
...and them should be somehow finally "married" with each other.
I agree that we should try to avoid multiplying the number of
compression-related APIs. Ideally there should be one API for
registering a compression algorithms, and then there can be different
methods of selecting that compression algorithm depending on the
purpose for which it will be used. For instance, you could select a
column compression format using some variant of ALTER TABLE ... ALTER
COLUMN, but you would obviously use some other method to select the
WAL compression format. However, it's a little unclear to me how we
would actually make the idea of a single API work. For column
compression, we need everything to be accessible through the catalogs.
For something like WAL compression, we need it to be completely
independent of the catalogs. Those things are opposites, so a single
API can't have both properties. Maybe there can be some pieces
shared, but as much as I'd like it to be otherwise, it doesn't seem
possible to share it completely.
I also agree with Ildus and Alexander that we cannot and should not
try to solve every problem in one patch. Rather, we should just think
ahead, so that we make as much of what goes into this patch reusable
in the future as we can.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Mon, 18 Jun 2018 17:30:45 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
On Tue, 24 Apr 2018 14:05:20 +0300
Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:Yes, this patch definitely lacks of good usage example. That may
lead to some misunderstanding of its purpose. Good use-cases
should be shown before we can consider committing this. I think
Ildus should try to implement at least custom dictionary compression
method where dictionary is specified by user in parameters.Hi,
attached v16 of the patch. I have splitted the patch to 8 parts so now
it should be easier to make a review. The main improvement is zlib
compression method with dictionary support like you mentioned. My
synthetic tests showed that zlib gives more compression but usually
slower than pglz.
Hi,
I have noticed that my patch is failing to apply on cputube. Attached a
rebased version of the patch. Nothing have really changed, just added
and fixed some tests for zlib and improved documentation.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
0001-Make-syntax-changes-for-custom-compression-metho-v17.patchtext/x-patchDownload
From 4b4a4b77c7a0c16cafa9d7745d9f02065bb80ea2 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
---
src/backend/parser/gram.y | 76 ++++++++++++++++++++++++++++++----
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 ++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 89 insertions(+), 9 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 90dfac2cb1..ab8fa0b7dd 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -399,6 +400,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -585,6 +587,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -617,9 +623,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2214,6 +2220,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3380,11 +3395,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3394,8 +3410,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3442,6 +3458,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3647,6 +3700,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5324,12 +5378,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -15031,6 +15090,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 26cb234987..a3b75c5256 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -51,6 +51,7 @@ typedef FormData_pg_am *Form_pg_am;
* Allowed values for amtype
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 43f1552241..03cee8ec88 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -473,6 +473,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 6390f7e8c1..69a03d4388 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,6 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -683,6 +698,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1790,7 +1806,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..76c33cbc5f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.18.0
0002-Add-compression-catalog-tables-and-the-basic-inf-v17.patchtext/x-patchDownload
From 99b58beec7e034121cc9814f64341ba72fefb6ea Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 3 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 4 +-
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 84 +++
src/backend/commands/compressioncmds.c | 653 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 3 +-
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 3 +-
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
46 files changed, 2129 insertions(+), 195 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index aa52a96259..9a1cebbe2f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index db84da0678..bf836305a9 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -962,11 +962,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 72395a50b8..fdac4bbb67 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2684,8 +2684,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -4129,7 +4130,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index ed7ba181c7..802497a4d9 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -655,7 +655,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index cd42c50b09..413b0b2b9c 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +61,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -421,7 +462,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +552,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +655,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +667,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +802,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +886,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +901,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +919,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1064,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1353,6 +1504,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1368,54 +1531,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1666,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1687,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1699,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2059,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2244,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2440,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2561,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate caches so we need to call it before
+ * we're putting something to cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreateExtended(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 7e34bee63e..d78ef2600c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,6 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 0865240f11..5560fc05b8 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 4f1d365357..61ebd316ba 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -170,7 +171,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1280,6 +1282,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2537,6 +2543,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d223ba8537..a67755e40a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -633,6 +633,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1606,6 +1607,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8b276bc430..e55a951f78 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -413,8 +413,9 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
+ to->attcompression = InvalidOid;
to->attcollation = (i < numkeyatts) ?
- collationObjectId[i] : InvalidOid;
+ collationObjectId[i] : InvalidOid;
ReleaseSysCache(tuple);
@@ -501,6 +502,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 7db942dcba..828240040b 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -489,6 +490,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3546,6 +3559,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4070,6 +4114,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4614,6 +4662,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3768a38459
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,653 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+builtin_removal:
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index eecc85d14e..ee67d07b2f 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1209,6 +1209,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 5c53aeeaeb..6db6e38a07 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7c0cf0d7ee..0c7c201952 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -22,6 +22,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -33,6 +34,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -40,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -89,6 +92,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -367,6 +371,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -465,6 +471,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -519,6 +527,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -693,6 +702,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -739,6 +750,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -787,6 +805,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2117,6 +2152,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2144,6 +2192,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2353,6 +2402,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3440,6 +3496,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3915,6 +3972,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4289,6 +4347,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4354,8 +4417,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5517,6 +5581,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5691,6 +5765,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5813,6 +5888,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5835,6 +5934,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6671,6 +6870,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9572,6 +9775,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9672,7 +9881,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9697,6 +9907,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9705,6 +9943,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12733,6 +12976,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1c12075b01..3009d29cce 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2859,6 +2859,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2878,6 +2879,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5553,6 +5566,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 6a971d0141..06b34a032c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2549,6 +2549,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2568,6 +2569,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3627,6 +3638,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index a10014f755..07ec871980 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3648,6 +3648,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3655,6 +3657,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 979d523e00..06097965c7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2859,6 +2859,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2876,6 +2877,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4181,6 +4192,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 17b54b20cc..2b909fc949 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1061,6 +1061,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 5792cd14a0..300eb6fd4c 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3038,7 +3038,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 4022c14a83..c9302347e8 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -259,7 +259,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -270,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..793367ff58 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 24915824ca..dc41e50bd8 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -87,6 +87,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..563aa65a2a
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index dc36753ede..08e074ea4d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index 9fffdef379..49643b934e 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -36,7 +36,7 @@
reloftype => '0', relowner => 'PGUID', relam => '0', relfilenode => '0',
reltablespace => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasoids => 'f', relhasrules => 'f', relhastriggers => 'f',
relhassubclass => 'f', relrowsecurity => 'f', relforcerowsecurity => 'f',
relispopulated => 't', relreplident => 'n', relispartition => 'f',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 40d54ed030..3242179a8a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6915,6 +6915,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7084,6 +7087,13 @@
{ oid => '3312', descr => 'I/O',
proname => 'tsm_handler_out', prorettype => 'cstring',
proargtypes => 'tsm_handler', prosrc => 'tsm_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 48e01cd694..722b11674f 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -929,6 +929,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
typcategory => 'P', typinput => 'tsm_handler_in',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8fc9e424cf..9acbebb0bc 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -154,8 +155,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 03cee8ec88..c54dfc7462 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -505,7 +505,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b596fcb513..916c75c7d8 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 03867cbce5..2310208331 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -350,6 +350,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -375,6 +376,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.18.0
0003-Add-rewrite-rules-and-tupdesc-flags-v17.patchtext/x-patchDownload
From c8340037fac1cc83e399c14d3130f61408ceafae Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 68 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 156 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 2ec7e6a439..680fc9df4b 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -214,6 +214,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -244,6 +245,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -282,6 +286,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -305,6 +318,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -341,7 +357,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -365,6 +381,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -375,6 +392,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -793,6 +811,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -917,6 +936,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -935,6 +955,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -987,6 +1008,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -997,6 +1019,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1152,6 +1175,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1855,6 +1879,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 9a1cebbe2f..38fbdf7205 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b0434b4672..0628416f0d 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -74,6 +75,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -96,8 +98,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -138,6 +149,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -302,6 +315,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -468,6 +484,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -553,6 +571,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -659,6 +678,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index fdac4bbb67..0917019f72 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -96,7 +96,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2361,13 +2362,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2454,7 +2456,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2623,7 +2625,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2685,8 +2687,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2725,7 +2730,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* Allocate some memory to use for constructing the WAL record. Using
@@ -4028,6 +4033,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 413b0b2b9c..f75ca6d9b2 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1197,6 +1197,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1425,6 +1426,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 3a66cb5025..253a454fbf 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2555,7 +2555,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index e1eb7c374b..969c8160c2 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 0c7c201952..e17a94a778 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -166,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -4648,7 +4650,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4919,6 +4921,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -9020,6 +9040,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9915,7 +9974,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -9923,7 +9982,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -13035,7 +13094,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index b1b6883c19..b0e6dfc996 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -814,6 +814,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 6125421d39..44be85af82 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -577,6 +577,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ca5cad7497..755682a4f8 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -22,6 +22,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -146,7 +147,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 1867a70f6f..9e255154ad 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -273,7 +273,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -686,6 +688,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -801,7 +806,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 708160f645..5bac03168a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -46,6 +48,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -83,6 +89,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 793367ff58..50f20611ab 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.18.0
0004-Add-pglz-compression-method-v17.patchtext/x-patchDownload
From 7c58263e843c36fcaa5e55dbbbf2e5484b2a0ad2 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index bef53a319a..6f7ad79613 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -30,5 +30,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 3242179a8a..eef66eae5e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -956,6 +956,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.18.0
0005-Add-zlib-compression-method-v17.patchtext/x-patchDownload
From b278e58fb7b3c578fd24fa7d18d5d2b5b642bbaa Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 25af514fba..855be8203c 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6f7ad79613..b5754b9759 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index eef66eae5e..64c105a462 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -961,6 +961,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.18.0
0006-Add-psql-pg_dump-and-pg_upgrade-support-v17.patchtext/x-patchDownload
From 5f31ab2ce177d5366a8a1b91723fa245fedd606a Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
---
src/bin/pg_dump/pg_backup.h | 5 +
src/bin/pg_dump/pg_backup_archiver.c | 1 +
src/bin/pg_dump/pg_dump.c | 330 ++++++++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 28 ++-
src/bin/pg_dump/pg_dump_sort.c | 10 +-
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 144 +++++++++++-
src/bin/psql/describe.c | 47 +++-
src/bin/psql/tab-complete.c | 5 +-
10 files changed, 565 insertions(+), 13 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ceedd481fb..b33fcbc168 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -122,6 +123,7 @@ typedef struct _restoreOptions
bool *idWanted; /* array showing which dump IDs to emit */
int enable_row_security;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump attribute compression table */
int binary_upgrade;
} RestoreOptions;
@@ -151,6 +153,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
@@ -172,6 +175,8 @@ typedef struct _dumpOptions
char *outputSuperuser;
int sequence_data; /* dump sequence data even in schema-only mode */
+ int compression_data; /* dump compression options even in
+ * schema-only mode */
} DumpOptions;
/*
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 83c976eaf7..319d78b548 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -180,6 +180,7 @@ dumpOptionsFromRestoreOptions(RestoreOptions *ropt)
dopt->include_everything = ropt->include_everything;
dopt->enable_row_security = ropt->enable_row_security;
dopt->sequence_data = ropt->sequence_data;
+ dopt->compression_data = ropt->compression_data;
return dopt;
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 463639208d..923d173598 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -377,6 +379,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{NULL, 0, NULL, 0}
@@ -597,6 +600,9 @@ main(int argc, char **argv)
if (dopt.binary_upgrade)
dopt.sequence_data = 1;
+ if (dopt.binary_upgrade)
+ dopt.compression_data = 1;
+
if (dopt.dataOnly && dopt.schemaOnly)
{
write_msg(NULL, "options -s/--schema-only and -a/--data-only cannot be used together\n");
@@ -807,6 +813,9 @@ main(int argc, char **argv)
if (dopt.schemaOnly && dopt.sequence_data)
getTableData(&dopt, tblinfo, numTables, dopt.oids, RELKIND_SEQUENCE);
+ if (dopt.schemaOnly && dopt.compression_data)
+ getAttributeCompressionData(fout);
+
/*
* In binary-upgrade mode, we do not have to worry about the actual blob
* data or the associated metadata that resides in the pg_largeobject and
@@ -4212,6 +4221,233 @@ dumpSubscription(Archive *fout, SubscriptionInfo *subinfo)
free(qsubname);
}
+/*
+ * getAttributeCompressionData
+ * get information from pg_attr_compression
+ * this information should be migrated at binary upgrade
+ */
+void
+getAttributeCompressionData(Archive *fout)
+{
+ PQExpBuffer query;
+ PGresult *res;
+ AttributeCompressionInfo *cminfo;
+ int i_tableoid;
+ int i_oid;
+ int i_relid;
+ int i_attnum;
+ int i_name;
+ int i_options;
+ int i_iscurrent;
+ int i_attname;
+ int i_parsedoptions;
+ int i_preserve;
+ int i_cnt;
+ int i,
+ j,
+ k,
+ attcnt,
+ ntups;
+
+ if (fout->remoteVersion < 110000)
+ return;
+
+ res = ExecuteSqlQuery(fout,
+ "SELECT tableoid, acrelid::pg_catalog.regclass as acrelid, acattnum, COUNT(*) AS cnt "
+ "FROM pg_catalog.pg_attr_compression "
+ "WHERE acrelid > 0 AND acattnum > 0 "
+ "GROUP BY tableoid, acrelid, acattnum;",
+ PGRES_TUPLES_OK);
+
+ attcnt = PQntuples(res);
+ i_tableoid = PQfnumber(res, "tableoid");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_cnt = PQfnumber(res, "cnt");
+
+ cminfo = pg_malloc(attcnt * sizeof(AttributeCompressionInfo));
+ for (i = 0; i < attcnt; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+ int cnt = atoi(PQgetvalue(res, i, i_cnt));
+
+ cminfo[i].dobj.catId.tableoid = atooid(PQgetvalue(res, i, i_tableoid));
+ cminfo[i].dobj.dump = DUMP_COMPONENT_DEFINITION;
+ cminfo[i].dobj.objType = DO_ATTRIBUTE_COMPRESSION;
+ AssignDumpId(&cminfo[i].dobj);
+
+ cminfo[i].acrelid = acrelid;
+ cminfo[i].acattnum = attnum;
+ cminfo[i].preserve = NULL;
+ cminfo[i].attname = NULL;
+ cminfo[i].nitems = cnt;
+ cminfo[i].items = pg_malloc0(sizeof(AttributeCompressionItem) * cnt);
+ cminfo[i].dobj.name = psprintf("%s.%d", acrelid, attnum);
+ }
+ PQclear(res);
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+ appendPQExpBuffer(query,
+ "SELECT c.acoid, c.acname, c.acattnum,"
+ " c.acrelid::pg_catalog.regclass as acrelid, c.acoptions,"
+ " a.attcompression = c.acoid as iscurrent, a.attname,"
+ " pg_catalog.pg_column_compression(a.attrelid, a.attname) as preserve,"
+ " pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ') AS parsedoptions "
+ "FROM pg_catalog.pg_attr_compression c "
+ "JOIN pg_attribute a ON (c.acrelid = a.attrelid "
+ " AND c.acattnum = a.attnum) "
+ "WHERE acrelid > 0 AND attnum > 0 "
+ "ORDER BY iscurrent");
+
+ res = ExecuteSqlQuery(fout, query->data, PGRES_TUPLES_OK);
+
+ ntups = PQntuples(res);
+
+ i_oid = PQfnumber(res, "acoid");
+ i_name = PQfnumber(res, "acname");
+ i_relid = PQfnumber(res, "acrelid");
+ i_attnum = PQfnumber(res, "acattnum");
+ i_options = PQfnumber(res, "acoptions");
+ i_iscurrent = PQfnumber(res, "iscurrent");
+ i_attname = PQfnumber(res, "attname");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_preserve = PQfnumber(res, "preserve");
+
+ for (i = 0; i < ntups; i++)
+ {
+ char *acrelid = pg_strdup(PQgetvalue(res, i, i_relid));
+ uint16 attnum = atoi(PQgetvalue(res, i, i_attnum));
+
+ for (j = 0; j < attcnt; j++)
+ {
+ if (strcmp(cminfo[j].acrelid, acrelid) == 0 &&
+ cminfo[j].acattnum == attnum)
+ {
+ /* in the end it will be current */
+ cminfo[j].dobj.catId.oid = atooid(PQgetvalue(res, i, i_oid));
+
+ if (cminfo[j].preserve == NULL)
+ cminfo[j].preserve = pg_strdup(PQgetvalue(res, i, i_preserve));
+ if (cminfo[j].attname == NULL)
+ cminfo[j].attname = pg_strdup(PQgetvalue(res, i, i_attname));
+
+ for (k = 0; k < cminfo[j].nitems; k++)
+ {
+ AttributeCompressionItem *item = &cminfo[j].items[k];
+
+ if (item->acname == NULL)
+ {
+ item->acoid = atooid(PQgetvalue(res, i, i_oid));
+ item->acname = pg_strdup(PQgetvalue(res, i, i_name));
+ item->acoptions = pg_strdup(PQgetvalue(res, i, i_options));
+ item->iscurrent = PQgetvalue(res, i, i_iscurrent)[0] == 't';
+ item->parsedoptions = pg_strdup(PQgetvalue(res, i, i_parsedoptions));
+
+ /* to next tuple */
+ break;
+ }
+ }
+
+ /* to next tuple */
+ break;
+ }
+ }
+ }
+
+ PQclear(res);
+ destroyPQExpBuffer(query);
+}
+
+/*
+ * dumpAttributeCompression
+ * dump the given compression options
+ */
+static void
+dumpAttributeCompression(Archive *fout, AttributeCompressionInfo *cminfo)
+{
+ PQExpBuffer query;
+ AttributeCompressionItem *item;
+ int i;
+
+ if (!(cminfo->dobj.dump & DUMP_COMPONENT_DEFINITION))
+ return;
+
+ query = createPQExpBuffer();
+ resetPQExpBuffer(query);
+
+ for (i = 0; i < cminfo->nitems; i++)
+ {
+ item = &cminfo->items[i];
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_attr_compression\n\t"
+ "VALUES (%d, '%s', '%s'::pg_catalog.regclass, %d, ",
+ item->acoid,
+ item->acname,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ if (strlen(item->acoptions) > 0)
+ appendPQExpBuffer(query, "'%s');\n", item->acoptions);
+ else
+ appendPQExpBuffer(query, "NULL);\n");
+
+ /*
+ * Restore dependency with access method. pglz is pinned by the
+ * system, no need to create dependency.
+ */
+ if (strcmp(item->acname, "pglz") != 0)
+ appendPQExpBuffer(query,
+ "WITH t AS (SELECT oid FROM pg_am WHERE amname = '%s')\n\t"
+ "INSERT INTO pg_catalog.pg_depend "
+ "SELECT %d, %d, 0, %d, t.oid, 0, 'n' FROM t;\n",
+ item->acname,
+ AttrCompressionRelationId,
+ item->acoid,
+ AccessMethodRelationId);
+
+ if (item->iscurrent)
+ {
+ appendPQExpBuffer(query, "ALTER TABLE %s ALTER COLUMN %s\n\tSET COMPRESSION %s",
+ cminfo->acrelid,
+ cminfo->attname,
+ item->acname);
+ if (strlen(item->parsedoptions) > 0)
+ appendPQExpBuffer(query, " WITH (%s)", item->parsedoptions);
+ appendPQExpBuffer(query, " PRESERVE (%s);\n", cminfo->preserve);
+ }
+ else if (!IsBuiltinCompression(item->acoid))
+ {
+ /* restore dependency */
+ appendPQExpBuffer(query, "INSERT INTO pg_catalog.pg_depend VALUES "
+ "(%d, %d, 0, %d, '%s'::pg_catalog.regclass, %d, 'i');\n",
+ AttrCompressionRelationId,
+ item->acoid,
+ RelationRelationId,
+ cminfo->acrelid,
+ cminfo->acattnum);
+ }
+ }
+
+ ArchiveEntry(fout,
+ cminfo->dobj.catId,
+ cminfo->dobj.dumpId,
+ cminfo->dobj.name,
+ NULL,
+ NULL,
+ "", false,
+ "ATTRIBUTE COMPRESSION", SECTION_PRE_DATA,
+ query->data, "", NULL,
+ NULL, 0,
+ NULL, NULL);
+
+ destroyPQExpBuffer(query);
+}
+
static void
binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
@@ -8104,6 +8340,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
@@ -8133,7 +8371,47 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
resetPQExpBuffer(q);
- if (fout->remoteVersion >= 110000)
+ if (fout->remoteVersion >= 120000)
+ {
+ /* attcompresssion is new in 12 */
+ appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
+ "a.attstattarget, a.attstorage, t.typstorage, "
+ "a.attnotnull, a.atthasdef, a.attisdropped, "
+ "a.attlen, a.attalign, a.attislocal, "
+ "pg_catalog.format_type(t.oid,a.atttypmod) AS atttypname, "
+ "array_to_string(a.attoptions, ', ') AS attoptions, "
+ "CASE WHEN a.attcollation <> t.typcollation "
+ "THEN a.attcollation ELSE 0 END AS attcollation, "
+ "a.attidentity, "
+ /* fdw */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(attfdwoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attfdwoptions ,"
+ /* compression */
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions, "
+ "c.acname AS attcmname, "
+ /* atthasmissing and attmissingval */
+ "CASE WHEN a.atthasmissing AND NOT a.attisdropped "
+ "THEN a.attmissingval ELSE null END AS attmissingval "
+ /* --- */
+ "FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
+ "ON a.atttypid = t.oid "
+ "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid "
+ "WHERE a.attrelid = '%u'::pg_catalog.oid "
+ "AND a.attnum > 0::pg_catalog.int2 "
+ "ORDER BY a.attnum",
+ tbinfo->dobj.catId.oid);
+ }
+ else if (fout->remoteVersion >= 110000)
{
/* atthasmissing and attmissingval are new in 11 */
appendPQExpBuffer(q, "SELECT a.attnum, a.attname, a.atttypmod, "
@@ -8180,6 +8458,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"FROM pg_catalog.pg_options_to_table(attfdwoptions) "
"ORDER BY option_name"
"), E',\n ') AS attfdwoptions ,"
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname, "
"NULL as attmissingval "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
@@ -8208,6 +8488,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"ORDER BY option_name"
"), E',\n ') AS attfdwoptions, "
"NULL as attmissingval "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8233,6 +8515,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"THEN a.attcollation ELSE 0 END AS attcollation, "
"NULL AS attfdwoptions, "
"NULL as attmissingval "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8252,6 +8536,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"0 AS attcollation, "
"NULL AS attfdwoptions, "
"NULL as attmissingval "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8270,6 +8556,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"'' AS attoptions, 0 AS attcollation, "
"NULL AS attfdwoptions, "
"NULL as attmissingval "
+ "NULL AS attcmoptions, "
+ "NULL AS attcmname "
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
"ON a.atttypid = t.oid "
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8300,6 +8588,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8317,6 +8607,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8345,6 +8637,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -9835,6 +10129,9 @@ dumpDumpableObject(Archive *fout, DumpableObject *dobj)
case DO_SUBSCRIPTION:
dumpSubscription(fout, (SubscriptionInfo *) dobj);
break;
+ case DO_ATTRIBUTE_COMPRESSION:
+ dumpAttributeCompression(fout, (AttributeCompressionInfo *) dobj);
+ break;
case DO_PRE_DATA_BOUNDARY:
case DO_POST_DATA_BOUNDARY:
/* never dumped, nothing to do */
@@ -12597,6 +12894,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15522,6 +15822,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15580,6 +15888,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even we're skipping compression the attribute
+ * will get builtin compression. It's the task for ALTER
+ * command to change to right one and remove dependency to
+ * builtin compression.
+ */
+ if (tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j])
+ && !(dopt->binary_upgrade && has_custom_compression))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -17886,6 +18213,7 @@ addBoundaryDependencies(DumpableObject **dobjs, int numObjs,
case DO_FOREIGN_SERVER:
case DO_TRANSFORM:
case DO_BLOB:
+ case DO_ATTRIBUTE_COMPRESSION:
/* Pre-data objects: must come before the pre-data boundary */
addObjectDependency(preDataBound, dobj->dumpId);
break;
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1448005f30..b1f89b9cae 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -84,7 +84,8 @@ typedef enum
DO_POLICY,
DO_PUBLICATION,
DO_PUBLICATION_REL,
- DO_SUBSCRIPTION
+ DO_SUBSCRIPTION,
+ DO_ATTRIBUTE_COMPRESSION
} DumpableObjectType;
/* component types of an object which can be selected for dumping */
@@ -317,6 +318,8 @@ typedef struct _tableInfo
Oid *attcollation; /* per-attribute collation selection */
char **attfdwoptions; /* per-attribute fdw options */
char **attmissingval; /* per attribute missing value */
+ char **attcmoptions; /* per-attribute compression options */
+ char **attcmnames; /* per-attribute compression method names */
bool *notnull; /* NOT NULL constraints on attributes */
bool *inhNotNull; /* true if NOT NULL is inherited */
struct _attrDefInfo **attrdefs; /* DEFAULT expressions */
@@ -625,6 +628,28 @@ typedef struct _SubscriptionInfo
char *subpublications;
} SubscriptionInfo;
+typedef struct _AttributeCompressionItem
+{
+ Oid acoid;
+ char *acname;
+ char *acoptions;
+ char *parsedoptions;
+ bool iscurrent;
+} AttributeCompressionItem;
+
+/* The AttributeCompressionInfo struct is used to represent attribute compression */
+typedef struct _AttributeCompressionInfo
+{
+ DumpableObject dobj;
+ char *acrelid;
+ uint16 acattnum;
+ char *attname;
+ char *preserve;
+
+ int nitems;
+ AttributeCompressionItem *items;
+} AttributeCompressionInfo;
+
/*
* We build an array of these with an entry for each object that is an
* extension member according to pg_depend.
@@ -725,5 +750,6 @@ extern void getPublications(Archive *fout);
extern void getPublicationTables(Archive *fout, TableInfo tblinfo[],
int numTables);
extern void getSubscriptions(Archive *fout);
+extern void getAttributeCompressionData(Archive *fout);
#endif /* PG_DUMP_H */
diff --git a/src/bin/pg_dump/pg_dump_sort.c b/src/bin/pg_dump/pg_dump_sort.c
index d2b0949d6b..6ef978a50c 100644
--- a/src/bin/pg_dump/pg_dump_sort.c
+++ b/src/bin/pg_dump/pg_dump_sort.c
@@ -49,7 +49,7 @@ static const int dbObjectTypePriority[] =
6, /* DO_FUNC */
7, /* DO_AGG */
8, /* DO_OPERATOR */
- 8, /* DO_ACCESS_METHOD */
+ 7, /* DO_ACCESS_METHOD */
9, /* DO_OPCLASS */
9, /* DO_OPFAMILY */
3, /* DO_COLLATION */
@@ -85,7 +85,8 @@ static const int dbObjectTypePriority[] =
35, /* DO_POLICY */
36, /* DO_PUBLICATION */
37, /* DO_PUBLICATION_REL */
- 38 /* DO_SUBSCRIPTION */
+ 38, /* DO_SUBSCRIPTION */
+ 21 /* DO_ATTRIBUTE_COMPRESSION */
};
static DumpId preDataBoundId;
@@ -1470,6 +1471,11 @@ describeDumpableObject(DumpableObject *obj, char *buf, int bufsize)
"POST-DATA BOUNDARY (ID %d)",
obj->dumpId);
return;
+ case DO_ATTRIBUTE_COMPRESSION:
+ snprintf(buf, bufsize,
+ "ATTRIBUTE COMPRESSION %s (ID %d OID %u)",
+ obj->name, obj->dumpId, obj->catId.oid);
+ return;
}
/* shouldn't get here */
snprintf(buf, bufsize,
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 54db0cd174..41a05bb383 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -135,6 +136,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
@@ -404,6 +406,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
@@ -618,6 +622,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 501d7cea72..78758107f2 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 7eee870259..ed41cc9b20 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -631,6 +631,62 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 2, NULL);\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \s+\QSET COMPRESSION pglz2 PRESERVE (pglz2);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz', 'dump_test.test_table_compression'::pg_catalog.regclass, 3, '{min_input_size=1000}');\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \s+\QSET COMPRESSION pglz WITH (min_input_size '1000') PRESERVE (pglz);\E\n
+ .*
+ \QINSERT INTO pg_catalog.pg_attr_compression\E\n
+ \s+\QVALUES (\E\d+\Q, 'pglz2', 'dump_test.test_table_compression'::pg_catalog.regclass, 4, '{min_input_size=1000}');\E\n
+ \QWITH t AS (SELECT oid FROM pg_am WHERE amname = 'pglz2')\E\n
+ \s+\QINSERT INTO pg_catalog.pg_depend SELECT 4001, \E\d+\Q, 0, 2601, t.oid, 0, 'n' FROM t;\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \s+\QSET COMPRESSION pglz2 WITH (min_input_size '1000') PRESERVE (pglz2);\E\n
+ /xms,
+ like => {
+ binary_upgrade => 1, },
+ unlike => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1,
+ data_only => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ role => 1,
+ section_pre_data => 1,
+ section_data => 1, }, },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1381,6 +1437,39 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => {
+ binary_upgrade => 1,
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_dump_test_schema => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ with_oids => 1, },
+ unlike => {
+ only_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1,
+ test_schema_plus_blobs => 1, }, },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2214,9 +2303,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz,\E\n
+ \s+\Qcol3 text COMPRESSION pglz,\E\n
+ \s+\Qcol4 text COMPRESSION pglz,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2256,7 +2345,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION pglz\E
\n\);
/xm,
like =>
@@ -2366,7 +2455,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION pglz,\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2384,7 +2473,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION pglz\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
@@ -2401,6 +2490,49 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 53,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text COMPRESSION pglz,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '1000')\E\n
+ \);
+ /xm,
+ like => {
+ clean => 1,
+ clean_if_exists => 1,
+ createdb => 1,
+ defaults => 1,
+ exclude_test_table => 1,
+ exclude_test_table_data => 1,
+ no_blobs => 1,
+ no_privs => 1,
+ no_owner => 1,
+ only_dump_test_schema => 1,
+ pg_dumpall_dbprivs => 1,
+ schema_only => 1,
+ section_pre_data => 1,
+ test_schema_plus_blobs => 1,
+ with_oids => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ only_dump_test_table => 1,
+ pg_dumpall_globals => 1,
+ pg_dumpall_globals_clean => 1,
+ role => 1,
+ section_post_data => 1, }, },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6e08515857..4f5a821f53 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1755,6 +1755,22 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
appendPQExpBufferStr(&buf, ",\n a.attstorage");
+
+ /* compresssion info */
+ if (pset.sversion >= 110000)
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ else
+ appendPQExpBufferStr(&buf, "\n NULL AS attcmname");
+
appendPQExpBufferStr(&buf, ",\n CASE WHEN a.attstattarget=-1 THEN NULL ELSE a.attstattarget END AS attstattarget");
/*
@@ -1871,6 +1887,11 @@ describeOneTableDetails(const char *schemaname,
if (verbose)
{
headers[cols++] = gettext_noop("Storage");
+
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ headers[cols++] = gettext_noop("Compression");
+
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_INDEX ||
@@ -1968,6 +1989,28 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
+ /* Column compression. */
+ if (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, firstvcol + 1);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1976,7 +2019,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
{
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 1),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
false, false);
}
@@ -1987,7 +2030,7 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_COMPOSITE_TYPE ||
tableinfo.relkind == RELKIND_FOREIGN_TABLE ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE)
- printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 2),
+ printTableAddCell(&cont, PQgetvalue(res, i, firstvcol + 3),
false, false);
}
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index bb696f8ee9..8cfb0304a8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2161,11 +2161,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
--
2.18.0
0007-Add-tests-for-compression-methods-v17.patchtext/x-patchDownload
From f3d7e7b1f133a25422fc7301e07bd8281092ebdb Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/backend/access/compression/cm_zlib.c | 2 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 405 +++++++++++++++++
src/test/regress/expected/create_cm_1.out | 409 ++++++++++++++++++
src/test/regress/expected/create_table.out | 132 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 282 ++++++------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 ++---
src/test/regress/expected/opr_sanity.out | 4 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 203 +++++++++
src/test/regress/sql/opr_sanity.sql | 4 +-
22 files changed, 1488 insertions(+), 462 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/expected/create_cm_1.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index 0dcb56ddf3..dc8666f309 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -182,7 +182,6 @@ zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
}
pfree(tmp);
-#endif
return NULL;
}
@@ -231,6 +230,7 @@ zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
pfree(zp);
return result;
}
+#endif
Datum
zlibhandler(PG_FUNCTION_ARGS)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e606a5fda4..37b9ddc3d7 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..de14d87885
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,405 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..755d40caef
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,409 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 672719e5d5..e4d802b1db 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -415,11 +415,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -428,11 +428,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -590,10 +590,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -729,11 +729,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -742,11 +742,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -756,11 +756,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -793,46 +793,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -864,11 +864,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -877,10 +877,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND (((a)::anyarray OPERATOR(pg_catalog.=) '{1}'::integer[]) OR ((a)::anyarray OPERATOR(pg_catalog.=) '{2}'::integer[])))
@@ -890,10 +890,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 0b5a9041b0..36151d0b06 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 75365501d4..54471fec02 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1337,12 +1337,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1358,12 +1358,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1382,12 +1382,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1425,12 +1425,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1450,17 +1450,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1482,17 +1482,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1524,17 +1524,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1562,12 +1562,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1599,12 +1599,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1643,12 +1643,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1674,12 +1674,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1701,12 +1701,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1728,12 +1728,12 @@ Inherits: fd_pt1
-- OID system column
ALTER TABLE fd_pt1 SET WITH OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1758,12 +1758,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE fd_pt1 SET WITHOUT OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1789,12 +1789,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1853,12 +1853,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1898,12 +1898,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1925,12 +1925,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1953,12 +1953,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1983,12 +1983,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2011,12 +2011,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 4f29d9f891..ea61f79883 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 5edf269367..ef1479f395 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index a1e18a6ceb..19f476a787 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1725,7 +1725,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index f1ae40df61..fd4110b5f7 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index ae0cd253d5..9f9adc3dcc 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2803,11 +2803,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2823,11 +2823,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 0aa5357917..4395152a94 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -105,6 +107,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 16f979c8d9..b8473c0568 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 42632be675..a4e3a771e2 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d7e6c5eba2
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,203 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index a593d37643..9c2d8fe89d 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1161,7 +1161,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
--
2.18.0
0008-Add-documentation-for-custom-compression-methods-v17.patchtext/x-patchDownload
From 17629c272a1616c696ae64651376681381ead517 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 178 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3ed9021c2f..216f8dd7dc 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support procedures</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index f010cd4c3b..78df1cf2be 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 24c3405f91..94ea769cfc 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 0070603fc3..d957f9b08b 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -252,6 +252,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 1cce00eaf9..5ecd2d4b69 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 2a1eac9592..fecaf7c5fa 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -860,6 +861,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 8ef2ac8010..4524eb65fc 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.18.0
Re: [HACKERS] Custom compression methods
Hi!
On Mon, Jul 2, 2018 at 3:56 PM Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:
On Mon, 18 Jun 2018 17:30:45 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:On Tue, 24 Apr 2018 14:05:20 +0300
Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:Yes, this patch definitely lacks of good usage example. That may
lead to some misunderstanding of its purpose. Good use-cases
should be shown before we can consider committing this. I think
Ildus should try to implement at least custom dictionary compression
method where dictionary is specified by user in parameters.Hi,
attached v16 of the patch. I have splitted the patch to 8 parts so now
it should be easier to make a review. The main improvement is zlib
compression method with dictionary support like you mentioned. My
synthetic tests showed that zlib gives more compression but usually
slower than pglz.I have noticed that my patch is failing to apply on cputube. Attached a
rebased version of the patch. Nothing have really changed, just added
and fixed some tests for zlib and improved documentation.
I'm going to review this patch. Could you please rebase it? It
doesn't apply for me due to changes made in src/bin/psql/describe.c.
patching file src/bin/psql/describe.c
Hunk #1 FAILED at 1755.
Hunk #2 FAILED at 1887.
Hunk #3 FAILED at 1989.
Hunk #4 FAILED at 2019.
Hunk #5 FAILED at 2030.
5 out of 5 hunks FAILED -- saving rejects to file src/bin/psql/describe.c.rej
Also, please not that PostgreSQL 11 already passed feature freeze some
time ago. So, please adjust your patch to expect PostgreSQL 12 in the
lines like this:
+ if (pset.sversion >= 110000)
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, 23 Jul 2018 16:16:19 +0300
Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
I'm going to review this patch. Could you please rebase it? It
doesn't apply for me due to changes made in src/bin/psql/describe.c.patching file src/bin/psql/describe.c
Hunk #1 FAILED at 1755.
Hunk #2 FAILED at 1887.
Hunk #3 FAILED at 1989.
Hunk #4 FAILED at 2019.
Hunk #5 FAILED at 2030.
5 out of 5 hunks FAILED -- saving rejects to file
src/bin/psql/describe.c.rejAlso, please not that PostgreSQL 11 already passed feature freeze some
time ago. So, please adjust your patch to expect PostgreSQL 12 in the
lines like this:+ if (pset.sversion >= 110000)
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Hi, attached latest set of patches. Rebased and fixed pg_upgrade errors
related with zlib support.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
0001-Make-syntax-changes-for-custom-compression-metho-v19.patchtext/x-patchDownload
From 7595df4969afb77b6ea0184b11a4da6dda7c7fae Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/parser/gram.y | 76 ++++++++++++++++++++++++++++++----
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 ++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 89 insertions(+), 9 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 4bd2223f26..12ad9e5210 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -399,6 +400,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -585,6 +587,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -617,9 +623,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2214,6 +2220,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3380,11 +3395,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3394,8 +3410,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3442,6 +3458,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3647,6 +3700,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5324,12 +5378,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -15042,6 +15101,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 26cb234987..a3b75c5256 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -51,6 +51,7 @@ typedef FormData_pg_am *Form_pg_am;
* Allowed values for amtype
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 697d3d7a5f..c84c67a1f2 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -474,6 +474,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 07ab1a3dde..c32a4546e7 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,6 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -683,6 +698,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1790,7 +1806,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..76c33cbc5f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.18.0
0002-Add-compression-catalog-tables-and-the-basic-inf-v19.patchtext/x-patchDownload
From 59d721744dd610ceea2d3d0bacaffe630905cb37 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 3 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 4 +-
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 84 +++
src/backend/commands/compressioncmds.c | 652 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 3 +-
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/catalog/toasting.h | 1 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 3 +-
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
47 files changed, 2129 insertions(+), 195 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index aa52a96259..9a1cebbe2f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index db84da0678..bf836305a9 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -962,11 +962,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 56f1d82f96..7a91896fa1 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2684,8 +2684,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -4121,7 +4122,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 85f92973c9..4857656601 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -655,7 +655,8 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
HEAP_INSERT_SKIP_FSM |
(state->rs_use_wal ?
- 0 : HEAP_INSERT_SKIP_WAL));
+ 0 : HEAP_INSERT_SKIP_WAL),
+ NULL);
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index cd42c50b09..413b0b2b9c 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +61,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -421,7 +462,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +552,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +655,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +667,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +802,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +886,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +901,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +919,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1064,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1353,6 +1504,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1368,54 +1531,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1666,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1687,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1699,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2059,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2244,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2440,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2561,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate caches so we need to call it before
+ * we're putting something to cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreateExtended(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index cdd71a9bc3..4de480fca4 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -740,6 +740,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 0865240f11..5560fc05b8 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 4f1d365357..61ebd316ba 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -170,7 +171,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1280,6 +1282,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2537,6 +2543,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5206d5e516..65625f6ae7 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -633,6 +633,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1605,6 +1606,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4b29120f19..60d4f47aa7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -413,8 +413,9 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attcacheoff = -1;
to->atttypmod = exprTypmod(indexkey);
to->attislocal = true;
+ to->attcompression = InvalidOid;
to->attcollation = (i < numkeyatts) ?
- collationObjectId[i] : InvalidOid;
+ collationObjectId[i] : InvalidOid;
ReleaseSysCache(tuple);
@@ -501,6 +502,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 7db942dcba..828240040b 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -489,6 +490,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3546,6 +3559,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4070,6 +4114,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4614,6 +4662,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..e1b41964f2
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,652 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(acform->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index eecc85d14e..ee67d07b2f 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1209,6 +1209,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index e5dd9958a4..efd0007a04 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e96512e051..9976a24929 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -22,6 +22,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -33,6 +34,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -40,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -89,6 +92,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -370,6 +374,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -468,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -522,6 +530,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -696,6 +705,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -742,6 +753,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -787,6 +805,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2147,6 +2182,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2174,6 +2222,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2383,6 +2432,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3464,6 +3520,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3939,6 +3996,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4313,6 +4371,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4378,8 +4441,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5540,6 +5604,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5714,6 +5788,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5836,6 +5911,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5858,6 +5957,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6701,6 +6900,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9604,6 +9807,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9704,7 +9913,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9729,6 +9939,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9737,6 +9975,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12756,6 +12999,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7c8220cf65..9a24b85c86 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2871,6 +2871,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2890,6 +2891,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5568,6 +5581,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 378f2facb8..6faf983efc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2549,6 +2549,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2568,6 +2569,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3627,6 +3638,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index a10014f755..07ec871980 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3648,6 +3648,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3655,6 +3657,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b5af904c18..5dee3630b2 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2870,6 +2870,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2887,6 +2888,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4195,6 +4206,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index f5c1e2a0d7..79e16e81f3 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1062,6 +1062,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 658fbe6494..7d2848557c 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3108,7 +3108,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 4022c14a83..c9302347e8 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -259,7 +259,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -270,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..793367ff58 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 254fbef1f7..5f2270f93d 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -87,6 +87,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..563aa65a2a
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index dc36753ede..08e074ea4d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index 9fffdef379..49643b934e 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -36,7 +36,7 @@
reloftype => '0', relowner => 'PGUID', relam => '0', relfilenode => '0',
reltablespace => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasoids => 'f', relhasrules => 'f', relhastriggers => 'f',
relhassubclass => 'f', relrowsecurity => 'f', relforcerowsecurity => 'f',
relispopulated => 't', relreplident => 'n', relispartition => 'f',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 860571440a..31768c9fa2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6915,6 +6915,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7084,6 +7087,13 @@
{ oid => '3312', descr => 'I/O',
proname => 'tsm_handler_out', prorettype => 'cstring',
proargtypes => 'tsm_handler', prosrc => 'tsm_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 48e01cd694..722b11674f 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -929,6 +929,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
typcategory => 'P', typinput => 'tsm_handler_in',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index f259890e43..e5408e2763 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -71,6 +71,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 4187, 4188);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1d05a4bcdc..988986afce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -156,8 +157,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index c84c67a1f2..a297cefc65 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -506,7 +506,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b596fcb513..916c75c7d8 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9fe950b29d..1b0d714c36 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -351,6 +351,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -376,6 +377,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.18.0
0003-Add-rewrite-rules-and-tupdesc-flags-v19.patchtext/x-patchDownload
From 0be409e623860965314ef01fc60056f30d5aa2b9 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 68 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 156 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index d8b06bca7e..a13600f3c5 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -214,6 +214,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -244,6 +245,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -282,6 +286,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -305,6 +318,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -341,7 +357,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -365,6 +381,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -375,6 +392,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -793,6 +811,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -917,6 +936,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -935,6 +955,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -987,6 +1008,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -997,6 +1019,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1152,6 +1175,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1810,6 +1834,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 9a1cebbe2f..38fbdf7205 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b0434b4672..0628416f0d 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -74,6 +75,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -96,8 +98,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -138,6 +149,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -302,6 +315,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -468,6 +484,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -553,6 +571,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -659,6 +678,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 7a91896fa1..b7877f4cb8 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -96,7 +96,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2361,13 +2362,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2454,7 +2456,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2623,7 +2625,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2685,8 +2687,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2725,7 +2730,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* We're about to do the actual inserts -- but check for conflict first,
@@ -4020,6 +4025,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 413b0b2b9c..f75ca6d9b2 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1197,6 +1197,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1425,6 +1426,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 9bc67ce60f..4d622afae8 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2634,7 +2634,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 3d82edbf58..c7059d5f62 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -569,7 +569,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index e1eb7c374b..969c8160c2 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9976a24929..89d53b173e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -166,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -4672,7 +4674,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4943,6 +4945,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -9049,6 +9069,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9947,7 +10006,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -9955,7 +10014,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -13058,7 +13117,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index b1b6883c19..b0e6dfc996 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -814,6 +814,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a4fc001103..8d9bc1f9c6 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -577,6 +577,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ca5cad7497..755682a4f8 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -22,6 +22,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -146,7 +147,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 1867a70f6f..9e255154ad 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -273,7 +273,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -686,6 +688,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -801,7 +806,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 708160f645..5bac03168a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -46,6 +48,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -83,6 +89,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 793367ff58..50f20611ab 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.18.0
0004-Add-pglz-compression-method-v19.patchtext/x-patchDownload
From e8b0e1e2bbb1fdb5c7a4dfb898fe2f37c8289729 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index bef53a319a..6f7ad79613 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -30,5 +30,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 31768c9fa2..276d2b9b26 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -956,6 +956,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.18.0
0005-Add-zlib-compression-method-v19.patchtext/x-patchDownload
From 0ee23a8186284e75992bd1af3a9f4ad5b7b6b3cf Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 3a58bf6685..9e984b34b6 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6f7ad79613..b5754b9759 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 276d2b9b26..53891aacc0 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -961,6 +961,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.18.0
0006-Add-psql-pg_dump-and-pg_upgrade-support-v19.patchtext/x-patchDownload
From 81a534d1e0dd29d5212a6354264626a086b548a5 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/commands/compressioncmds.c | 80 ++++++---
src/backend/commands/tablecmds.c | 14 +-
src/backend/utils/adt/pg_upgrade_support.c | 10 ++
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_dump.c | 200 ++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 17 ++
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 95 ++++++++++
src/bin/psql/describe.c | 42 +++++
src/bin/psql/tab-complete.c | 5 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 +
13 files changed, 434 insertions(+), 45 deletions(-)
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index e1b41964f2..f3a5a1f7fb 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -36,6 +36,9 @@
#include "utils/syscache.h"
#include "utils/snapmgr.h"
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
/*
* When conditions of compression satisfies one if builtin attribute
* compresssion tuples the compressed attribute will be linked to
@@ -129,11 +132,12 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
tup_amoid;
Datum values[Natts_pg_attr_compression];
bool nulls[Natts_pg_attr_compression];
+ char *amname;
heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
- tup_amoid = get_am_oid(
- NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
if (previous_amoids)
*previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
@@ -150,17 +154,15 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
if (DatumGetPointer(acoptions) == NULL)
result = acoid;
}
- else
+ else if (DatumGetPointer(acoptions) != NULL)
{
bool equal;
/* check if arrays for WITH options are equal */
equal = DatumGetBool(CallerFInfoFunctionCall2(
- array_eq,
- &arrayeq_info,
- InvalidOid,
- acoptions,
- values[Anum_pg_attr_compression_acoptions - 1]));
+ array_eq, &arrayeq_info, InvalidOid, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
if (equal)
result = acoid;
}
@@ -227,6 +229,16 @@ CreateAttributeCompression(Form_pg_attribute att,
/* Try to find builtin compression first */
acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
/*
* attrelid will be invalid on CREATE TABLE, no need for table rewrite
* check.
@@ -252,16 +264,10 @@ CreateAttributeCompression(Form_pg_attribute att,
*/
if (need_rewrite != NULL)
{
- /* no rewrite by default */
- *need_rewrite = false;
-
Assert(preserved_amoids != NULL);
if (compression->preserve == NIL)
- {
- Assert(!IsBinaryUpgrade);
*need_rewrite = true;
- }
else
{
ListCell *cell;
@@ -294,7 +300,7 @@ CreateAttributeCompression(Form_pg_attribute att,
* In binary upgrade list will not be free since it contains
* Oid of builtin compression access method.
*/
- if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ if (list_length(previous_amoids) != 0)
*need_rewrite = true;
}
}
@@ -303,9 +309,6 @@ CreateAttributeCompression(Form_pg_attribute att,
list_free(previous_amoids);
}
- if (IsBinaryUpgrade && !OidIsValid(acoid))
- elog(ERROR, "could not restore attribute compression data");
-
/* Return Oid if we already found identical compression on this column */
if (OidIsValid(acoid))
{
@@ -315,6 +318,7 @@ CreateAttributeCompression(Form_pg_attribute att,
return acoid;
}
+add_tuple:
/* Initialize buffers for new tuple values */
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
@@ -323,13 +327,27 @@ CreateAttributeCompression(Form_pg_attribute att,
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
- acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
- Anum_pg_attr_compression_acoid);
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
if (acoid < FirstNormalObjectId)
{
- /* this is database initialization */
+ /* this is built-in attribute compression */
heap_close(rel, RowExclusiveLock);
- return DefaultCompressionOid;
+ return acoid;
}
/* we need routine only to call cmcheck function */
@@ -393,8 +411,8 @@ RemoveAttributeCompression(Oid acoid)
/*
* CleanupAttributeCompression
*
- * Remove entries in pg_attr_compression except current attribute compression
- * and related with specified list of access methods.
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
*/
void
CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
@@ -422,9 +440,7 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
ReleaseSysCache(attrtuple);
Assert(relid > 0 && attnum > 0);
-
- if (IsBinaryUpgrade)
- goto builtin_removal;
+ Assert(!IsBinaryUpgrade);
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
@@ -441,7 +457,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
true, NULL, 2, key);
- /* Remove attribute compression tuples and collect removed Oids to list */
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
while (HeapTupleIsValid(tuple = systable_getnext(scan)))
{
Form_pg_attr_compression acform;
@@ -463,7 +482,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
systable_endscan(scan);
heap_close(rel, RowExclusiveLock);
- /* Now remove dependencies */
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
rel = heap_open(DependRelationId, RowExclusiveLock);
foreach(lc, removed)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 89d53b173e..5ef48e3005 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -756,10 +756,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
- if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression = CreateAttributeCompression(attr,
- colDef->compression,
- NULL, NULL);
+ colDef->compression, NULL, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -13133,14 +13133,6 @@ ATExecSetCompression(AlteredTableInfo *tab,
/* make changes visible */
CommandCounterIncrement();
- /*
- * Normally cleanup is done in rewrite but in binary upgrade we should do
- * it explicitly.
- */
- if (IsBinaryUpgrade)
- CleanupAttributeCompression(RelationGetRelid(rel),
- attnum, preserved_amoids);
-
ObjectAddressSet(address, AttrCompressionRelationId, acoid);
return address;
}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index b8b7777c31..1082eab4dc 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -116,6 +116,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 42cf441aaf..11b0da8221 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -151,6 +152,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f0ea83e6a9..b76e059d67 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -375,6 +377,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
@@ -842,13 +845,13 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
- sortDumpableObjectsByTypeName(dobjs, numObjs);
-
- /* If we do a parallel dump, we want the largest tables to go first */
if (archiveFormat == archDirectory && numWorkers > 1)
sortDataAndIndexObjectsBySize(dobjs, numObjs);
+ sortDumpableObjectsByTypeName(dobjs, numObjs);
sortDumpableObjects(dobjs, numObjs,
boundaryObjs[0].dumpId, boundaryObjs[1].dumpId);
@@ -8133,9 +8136,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
+ bool createWithCompression;
for (i = 0; i < numTables; i++)
{
@@ -8178,6 +8184,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attislocal,\n"
"pg_catalog.format_type(t.oid, a.atttypmod) AS atttypname,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBuffer(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8228,7 +8251,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBuffer(q,
/* need left join here to not fail on dropped columns ... */
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8256,6 +8285,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8273,9 +8304,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (j = 0; j < ntups; j++)
@@ -8301,6 +8335,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -8518,6 +8554,104 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ if (g_verbose)
+ write_msg(NULL, "finding compression info for table \"%s.%s\"\n",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -12575,6 +12709,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15500,6 +15637,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15558,6 +15703,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -15973,6 +16137,34 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "OPTIONS (\n %s\n);\n",
tbinfo->attfdwoptions[j]);
}
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
}
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1448005f30..582d661dd2 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,10 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
+ char **attcmoptions; /* per-attribute current compression options */
+ char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
+
/*
* Stuff computed only for dumpable tables.
*/
@@ -346,6 +350,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index eb29d318a4..61d55c7082 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -136,6 +137,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -406,6 +408,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -622,6 +626,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 501d7cea72..78758107f2 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -364,6 +366,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec751a7c23..432b65ef00 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -650,6 +650,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1400,6 +1437,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2420,6 +2468,53 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4ca0db1d0c..58bc222c0d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1467,6 +1467,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1835,6 +1836,24 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1954,6 +1973,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2025,6 +2046,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index bb696f8ee9..8cfb0304a8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2161,11 +2161,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches7("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches6("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH_LIST5("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH_LIST6("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH_LIST2("n_distinct", "n_distinct_inherited");
+ else if (Matches9("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches8("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH_CONST("WITH (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches8("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches7("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index abc6e1ae1d..1e95a3863a 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -25,6 +25,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 53891aacc0..06a0576bd8 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10061,6 +10061,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4012', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# replication/origin.h
{ oid => '6003', descr => 'create a replication origin',
--
2.18.0
0007-Add-tests-for-compression-methods-v19.patchtext/x-patchDownload
From a5bedd2dc35b0276a0de771598a803f35c0d1a37 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/backend/access/compression/cm_zlib.c | 2 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 405 +++++++++++++++++
src/test/regress/expected/create_cm_1.out | 409 ++++++++++++++++++
src/test/regress/expected/create_table.out | 132 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 282 ++++++------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 ++---
src/test/regress/expected/opr_sanity.out | 4 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 203 +++++++++
src/test/regress/sql/opr_sanity.sql | 4 +-
22 files changed, 1488 insertions(+), 462 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/expected/create_cm_1.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index 0dcb56ddf3..dc8666f309 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -182,7 +182,6 @@ zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
}
pfree(tmp);
-#endif
return NULL;
}
@@ -231,6 +230,7 @@ zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
pfree(zp);
return result;
}
+#endif
Datum
zlibhandler(PG_FUNCTION_ARGS)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index eb9e4b9774..36e7b2fa72 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..de14d87885
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,405 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..755d40caef
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,409 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 744b9990a6..86e09a9c23 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -414,11 +414,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -427,11 +427,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -589,10 +589,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -730,11 +730,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -743,11 +743,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -757,11 +757,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -794,46 +794,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -865,11 +865,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -878,10 +878,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -891,10 +891,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 0b5a9041b0..36151d0b06 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 75365501d4..54471fec02 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1337,12 +1337,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1358,12 +1358,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1382,12 +1382,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1425,12 +1425,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1450,17 +1450,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1482,17 +1482,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1524,17 +1524,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1562,12 +1562,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1599,12 +1599,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1643,12 +1643,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1674,12 +1674,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1701,12 +1701,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1728,12 +1728,12 @@ Inherits: fd_pt1
-- OID system column
ALTER TABLE fd_pt1 SET WITH OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1758,12 +1758,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE fd_pt1 SET WITHOUT OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1789,12 +1789,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1853,12 +1853,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1898,12 +1898,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1925,12 +1925,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1953,12 +1953,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1983,12 +1983,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2011,12 +2011,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 4f29d9f891..ea61f79883 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 5edf269367..ef1479f395 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 3c6d853ffb..3ac81daf90 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1760,7 +1760,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index bc16ca4c43..9f7d16bef5 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 078129f251..e41f5f5c09 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2803,11 +2803,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2823,11 +2823,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 0aa5357917..4395152a94 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -105,6 +107,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 16f979c8d9..b8473c0568 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 42632be675..a4e3a771e2 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -43,6 +43,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d7e6c5eba2
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,203 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index a593d37643..9c2d8fe89d 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1161,7 +1161,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
--
2.18.0
0008-Add-documentation-for-custom-compression-methods-v19.patchtext/x-patchDownload
From 2d4218e4085fff4e1f90bc494af7e384dbc8148f Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 178 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0179deea2e..1d5262b67a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index f010cd4c3b..78df1cf2be 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index beb99d1831..cf96689747 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 0070603fc3..d957f9b08b 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -252,6 +252,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index ec6b4c3311..2409cc9d34 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 5a19f94ce9..0b63c41901 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -921,6 +922,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 8ef2ac8010..4524eb65fc 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.18.0
Re: [HACKERS] Custom compression methods
On Thu, 6 Sep 2018 18:27:13 +0300
Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
Hi, attached latest set of patches. Rebased and fixed pg_upgrade
errors related with zlib support.
Hi, just updated patches to current master. Nothing new.
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
0001-Make-syntax-changes-for-custom-compression-metho-v20.patchtext/x-patchDownload
From c7d93bdba8d2008900f0d7da39d95591ffb23f62 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/parser/gram.y | 76 ++++++++++++++++++++++++++++++----
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 ++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 89 insertions(+), 9 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 6d23bfb0b3..ffcfb3c05b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -399,6 +400,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -585,6 +587,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -617,9 +623,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2213,6 +2219,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3379,11 +3394,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3393,8 +3409,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3441,6 +3457,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3646,6 +3699,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5323,12 +5377,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -15029,6 +15088,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 26cb234987..a3b75c5256 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -51,6 +51,7 @@ typedef FormData_pg_am *Form_pg_am;
* Allowed values for amtype
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index cac6ff0eda..a392fcdd2c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -474,6 +474,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index aa4a0dba2a..dde4697425 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -622,6 +622,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -645,6 +659,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -682,6 +697,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1803,7 +1819,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..76c33cbc5f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.19.1
0002-Add-compression-catalog-tables-and-the-basic-inf-v20.patchtext/x-patchDownload
From 3a381237b22cd139f861c8085641e9e4a1d3697a Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 2 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 84 +++
src/backend/commands/compressioncmds.c | 649 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 3 +-
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/catalog/toasting.h | 1 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 3 +-
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
47 files changed, 2124 insertions(+), 194 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index aa52a96259..9a1cebbe2f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index db84da0678..bf836305a9 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -962,11 +962,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index fb63471a0e..d963cc50e8 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2702,8 +2702,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -4142,7 +4143,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 7127788964..f479a70c2f 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -667,7 +667,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
options |= HEAP_INSERT_NO_LOGICAL;
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
- options);
+ options, NULL);
}
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index cd42c50b09..785efc113f 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +61,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -421,7 +462,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +552,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +655,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +667,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +802,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -741,12 +886,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -754,7 +901,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -770,10 +919,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -914,7 +1064,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1353,6 +1504,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1368,54 +1531,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1510,19 +1666,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1530,7 +1687,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1539,7 +1699,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1899,7 +2059,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2084,7 +2244,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2280,15 +2440,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2390,3 +2561,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate system cache so we need to call it before
+ * we're putting something to our cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreate(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 578af2e66d..1343efe76d 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -748,6 +748,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 0865240f11..5560fc05b8 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 7dfa3278a5..0d3bf0dd05 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -170,7 +171,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1280,6 +1282,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2538,6 +2544,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3c9c03c997..ce29d0d070 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -689,6 +689,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1661,6 +1662,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 5885899c9b..150cb91839 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -410,6 +410,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -496,6 +497,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 593e6f7022..2429e9b678 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -489,6 +490,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3548,6 +3561,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4072,6 +4116,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4616,6 +4664,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index eff325cc7d..eb49cfc54a 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index f2173450ad..8f69f9e33b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -185,6 +249,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -225,6 +299,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -263,6 +339,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..7841c7700a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,649 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(((Form_pg_attr_compression) GETSTRUCT(tup))->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 20a3a78692..20eeee60cc 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1209,6 +1209,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index e5dd9958a4..efd0007a04 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -48,50 +48,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -178,7 +134,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 153aec263e..f18b97add1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -22,6 +22,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -33,6 +34,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -40,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -89,6 +92,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -370,6 +374,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepAddOids(List **wqueue, Relation rel, bool recurse,
AlterTableCmd *cmd, LOCKMODE lockmode);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
@@ -468,6 +474,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -522,6 +530,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -696,6 +705,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -742,6 +753,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -787,6 +805,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2147,6 +2182,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2174,6 +2222,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2383,6 +2432,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3464,6 +3520,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3939,6 +3996,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4313,6 +4371,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4378,8 +4441,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5540,6 +5604,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5714,6 +5788,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5836,6 +5911,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5858,6 +5957,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE SET WITH OIDS
*
@@ -6702,6 +6901,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9586,6 +9789,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9686,7 +9895,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9711,6 +9921,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9719,6 +9957,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12742,6 +12985,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e8ea59e34a..120245084c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2869,6 +2869,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2888,6 +2889,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5567,6 +5580,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3bb91c9595..3ddfe063af 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2549,6 +2549,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2568,6 +2569,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3628,6 +3639,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index a10014f755..07ec871980 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3648,6 +3648,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3655,6 +3657,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 69731ccdea..afe7319883 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2870,6 +2870,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2887,6 +2888,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4211,6 +4222,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a6a2de94ea..e296a45299 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1062,6 +1062,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index bed63c768e..9eb8e05dbe 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3123,7 +3123,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 2b381782a3..0c97851cbd 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 4022c14a83..c9302347e8 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -259,7 +259,6 @@ extern void add_string_reloption(bits32 kinds, const char *name, const char *des
extern Datum transformRelOptions(Datum oldOptions, List *defList,
const char *namspace, char *validnsps[],
bool ignoreOids, bool isReset);
-extern List *untransformRelOptions(Datum options);
extern bytea *extractRelOptions(HeapTuple tuple, TupleDesc tupdesc,
amoptions_function amoptions);
extern relopt_value *parseRelOptions(Datum options, bool validate,
@@ -270,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..793367ff58 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 254fbef1f7..5f2270f93d 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -87,6 +87,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..563aa65a2a
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId) BKI_WITHOUT_OIDS
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index dc36753ede..08e074ea4d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_WITHOUT_OIDS BK
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index 9fffdef379..49643b934e 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -36,7 +36,7 @@
reloftype => '0', relowner => 'PGUID', relam => '0', relfilenode => '0',
reltablespace => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasoids => 'f', relhasrules => 'f', relhastriggers => 'f',
relhassubclass => 'f', relrowsecurity => 'f', relforcerowsecurity => 'f',
relispopulated => 't', relreplident => 'n', relispartition => 'f',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4026018ba9..02431b45ef 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6740,6 +6740,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -6909,6 +6912,13 @@
{ oid => '3312', descr => 'I/O',
proname => 'tsm_handler_out', prorettype => 'cstring',
proargtypes => 'tsm_handler', prosrc => 'tsm_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index d295eae1b9..0fb34cb680 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -574,6 +574,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index f259890e43..e5408e2763 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -71,6 +71,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 4187, 4188);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1d05a4bcdc..988986afce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -156,8 +157,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index a392fcdd2c..95f1f935e7 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -506,7 +506,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b596fcb513..916c75c7d8 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 4f333586ee..48051b8294 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9fe950b29d..1b0d714c36 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -351,6 +351,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -376,6 +377,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.19.1
0003-Add-rewrite-rules-and-tupdesc-flags-v20.patchtext/x-patchDownload
From 412cdde360396538fe79ab39d4212acfc1b5330c Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 68 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 156 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 00316b899c..cb9024863d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 28127b311f..2d46e207c4 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -177,6 +177,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -207,6 +208,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -245,6 +249,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -268,6 +281,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -304,7 +320,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -328,6 +344,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -338,6 +355,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -756,6 +774,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -871,6 +890,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -889,6 +909,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -941,6 +962,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -951,6 +973,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1106,6 +1129,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1620,6 +1644,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 9a1cebbe2f..38fbdf7205 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index b0434b4672..0628416f0d 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -74,6 +75,7 @@ CreateTemplateTupleDesc(int natts, bool hasoid)
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
desc->tdhasoid = hasoid;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -96,8 +98,17 @@ CreateTupleDesc(int natts, bool hasoid, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts, hasoid);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -138,6 +149,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -302,6 +315,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdhasoid != tupdesc2->tdhasoid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -468,6 +484,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -553,6 +571,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -659,6 +678,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d963cc50e8..4ff0bcf33b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -96,7 +96,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2373,13 +2374,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2471,7 +2473,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2641,7 +2643,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2703,8 +2705,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2746,7 +2751,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* We're about to do the actual inserts -- but check for conflict first,
@@ -4041,6 +4046,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 785efc113f..3c6896b6b9 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1197,6 +1197,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1425,6 +1426,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index b58a74f4e3..9ba328d24a 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2637,7 +2637,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index d5cb62da15..9709347739 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -570,7 +570,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index e1eb7c374b..969c8160c2 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f18b97add1..4ecff835c0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -166,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -4672,7 +4674,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4943,6 +4945,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -9052,6 +9072,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9929,7 +9988,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -9937,7 +9996,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -13044,7 +13103,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 9aa6f3aac5..055d84f079 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -814,6 +814,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index fd3d010b77..7989a02bce 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -577,6 +577,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 40e153f71a..d8da9f1f90 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -22,6 +22,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -147,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 97d240fdbb..b58c84aa8a 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -273,7 +273,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -686,6 +688,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -801,7 +806,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 708160f645..5bac03168a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -46,6 +48,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -83,6 +89,7 @@ typedef struct tupleDesc
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
bool tdhasoid; /* tuple has oid attribute in its header */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 793367ff58..50f20611ab 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0e1959462e..4fc2ed8c68 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
#define AT_REWRITE_ALTER_OID 0x08
+#define AT_REWRITE_ALTER_COMPRESSION 0x10
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.19.1
0004-Add-pglz-compression-method-v20.patchtext/x-patchDownload
From e1839d0d76b462742192123581ec1c959d5047f3 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index bef53a319a..6f7ad79613 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -30,5 +30,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 02431b45ef..0b2dc896e7 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -851,6 +851,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.19.1
0005-Add-zlib-compression-method-v20.patchtext/x-patchDownload
From 21bcfd8dfc73438382fa6b29e009ed2caf711f65 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 3a58bf6685..9e984b34b6 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6f7ad79613..b5754b9759 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 0b2dc896e7..fa82fbed05 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -856,6 +856,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.19.1
0006-Add-psql-pg_dump-and-pg_upgrade-support-v20.patchtext/x-patchDownload
From 17bc4bf335e26bf0a85f6dde2ebd03734ef9e103 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/commands/compressioncmds.c | 80 ++++++---
src/backend/commands/tablecmds.c | 14 +-
src/backend/utils/adt/pg_upgrade_support.c | 10 ++
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_dump.c | 196 ++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 17 ++
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 95 ++++++++++
src/bin/psql/describe.c | 42 +++++
src/bin/psql/tab-complete.c | 5 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 +
13 files changed, 433 insertions(+), 42 deletions(-)
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 7841c7700a..89ffa227b0 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -36,6 +36,9 @@
#include "utils/syscache.h"
#include "utils/snapmgr.h"
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
/*
* When conditions of compression satisfies one if builtin attribute
* compresssion tuples the compressed attribute will be linked to
@@ -129,11 +132,12 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
tup_amoid;
Datum values[Natts_pg_attr_compression];
bool nulls[Natts_pg_attr_compression];
+ char *amname;
heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
- tup_amoid = get_am_oid(
- NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
if (previous_amoids)
*previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
@@ -150,17 +154,15 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
if (DatumGetPointer(acoptions) == NULL)
result = acoid;
}
- else
+ else if (DatumGetPointer(acoptions) != NULL)
{
bool equal;
/* check if arrays for WITH options are equal */
equal = DatumGetBool(CallerFInfoFunctionCall2(
- array_eq,
- &arrayeq_info,
- InvalidOid,
- acoptions,
- values[Anum_pg_attr_compression_acoptions - 1]));
+ array_eq, &arrayeq_info, InvalidOid, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
if (equal)
result = acoid;
}
@@ -227,6 +229,16 @@ CreateAttributeCompression(Form_pg_attribute att,
/* Try to find builtin compression first */
acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
/*
* attrelid will be invalid on CREATE TABLE, no need for table rewrite
* check.
@@ -252,16 +264,10 @@ CreateAttributeCompression(Form_pg_attribute att,
*/
if (need_rewrite != NULL)
{
- /* no rewrite by default */
- *need_rewrite = false;
-
Assert(preserved_amoids != NULL);
if (compression->preserve == NIL)
- {
- Assert(!IsBinaryUpgrade);
*need_rewrite = true;
- }
else
{
ListCell *cell;
@@ -294,7 +300,7 @@ CreateAttributeCompression(Form_pg_attribute att,
* In binary upgrade list will not be free since it contains
* Oid of builtin compression access method.
*/
- if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ if (list_length(previous_amoids) != 0)
*need_rewrite = true;
}
}
@@ -303,9 +309,6 @@ CreateAttributeCompression(Form_pg_attribute att,
list_free(previous_amoids);
}
- if (IsBinaryUpgrade && !OidIsValid(acoid))
- elog(ERROR, "could not restore attribute compression data");
-
/* Return Oid if we already found identical compression on this column */
if (OidIsValid(acoid))
{
@@ -315,6 +318,7 @@ CreateAttributeCompression(Form_pg_attribute att,
return acoid;
}
+add_tuple:
/* Initialize buffers for new tuple values */
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
@@ -323,13 +327,27 @@ CreateAttributeCompression(Form_pg_attribute att,
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
- acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
- Anum_pg_attr_compression_acoid);
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
if (acoid < FirstNormalObjectId)
{
- /* this is database initialization */
+ /* this is built-in attribute compression */
heap_close(rel, RowExclusiveLock);
- return DefaultCompressionOid;
+ return acoid;
}
/* we need routine only to call cmcheck function */
@@ -390,8 +408,8 @@ RemoveAttributeCompression(Oid acoid)
/*
* CleanupAttributeCompression
*
- * Remove entries in pg_attr_compression except current attribute compression
- * and related with specified list of access methods.
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
*/
void
CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
@@ -419,9 +437,7 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
ReleaseSysCache(attrtuple);
Assert(relid > 0 && attnum > 0);
-
- if (IsBinaryUpgrade)
- goto builtin_removal;
+ Assert(!IsBinaryUpgrade);
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
@@ -438,7 +454,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
true, NULL, 2, key);
- /* Remove attribute compression tuples and collect removed Oids to list */
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
while (HeapTupleIsValid(tuple = systable_getnext(scan)))
{
Form_pg_attr_compression acform;
@@ -460,7 +479,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
systable_endscan(scan);
heap_close(rel, RowExclusiveLock);
- /* Now remove dependencies */
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
rel = heap_open(DependRelationId, RowExclusiveLock);
foreach(lc, removed)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 4ecff835c0..a778f48cbb 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -756,10 +756,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
- if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression = CreateAttributeCompression(attr,
- colDef->compression,
- NULL, NULL);
+ colDef->compression, NULL, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -13119,14 +13119,6 @@ ATExecSetCompression(AlteredTableInfo *tab,
/* make changes visible */
CommandCounterIncrement();
- /*
- * Normally cleanup is done in rewrite but in binary upgrade we should do
- * it explicitly.
- */
- if (IsBinaryUpgrade)
- CleanupAttributeCompression(RelationGetRelid(rel),
- attnum, preserved_amoids);
-
ObjectAddressSet(address, AttrCompressionRelationId, acoid);
return address;
}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index b8b7777c31..1082eab4dc 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -116,6 +116,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index ba798213be..00853989d0 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -151,6 +152,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c8d01ed4a4..e59b8ca9d5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -376,6 +378,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
@@ -843,6 +846,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8143,9 +8148,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
+ bool createWithCompression;
for (i = 0; i < numTables; i++)
{
@@ -8188,6 +8196,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attislocal,\n"
"pg_catalog.format_type(t.oid, a.atttypmod) AS atttypname,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBuffer(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8240,7 +8265,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8268,6 +8299,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8285,9 +8318,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (j = 0; j < ntups; j++)
@@ -8313,6 +8349,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -8530,6 +8568,104 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ if (g_verbose)
+ write_msg(NULL, "finding compression info for table \"%s.%s\"\n",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -12606,6 +12742,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15531,6 +15670,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15589,6 +15736,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -16004,6 +16170,34 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "OPTIONS (\n %s\n);\n",
tbinfo->attfdwoptions[j]);
}
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
}
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 685ad78669..9bff630c93 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,10 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
+ char **attcmoptions; /* per-attribute current compression options */
+ char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
+
/*
* Stuff computed only for dumpable tables.
*/
@@ -346,6 +350,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 5176626476..d1e13c643a 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -136,6 +137,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -406,6 +408,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -622,6 +626,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 44012ff44d..fd45a0d7c2 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -375,6 +377,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec751a7c23..432b65ef00 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -650,6 +650,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1400,6 +1437,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2420,6 +2468,53 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4ca0db1d0c..58bc222c0d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1467,6 +1467,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1835,6 +1836,24 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1954,6 +1973,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2025,6 +2046,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a980f92e11..a9383ab939 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1853,11 +1853,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index abc6e1ae1d..1e95a3863a 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -25,6 +25,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fa82fbed05..4528a3c2c1 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9868,6 +9868,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4012', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# replication/origin.h
{ oid => '6003', descr => 'create a replication origin',
--
2.19.1
0007-Add-tests-for-compression-methods-v20.patchtext/x-patchDownload
From 32085cf06568d4ebe26d6a985c6e0cedf8c2f165 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/backend/access/compression/cm_zlib.c | 2 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 405 +++++++++++++++++
src/test/regress/expected/create_cm_1.out | 409 ++++++++++++++++++
src/test/regress/expected/create_table.out | 132 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 282 ++++++------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 ++---
src/test/regress/expected/opr_sanity.out | 4 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 203 +++++++++
src/test/regress/sql/opr_sanity.sql | 4 +-
22 files changed, 1488 insertions(+), 462 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/expected/create_cm_1.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index 0dcb56ddf3..dc8666f309 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -182,7 +182,6 @@ zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
}
pfree(tmp);
-#endif
return NULL;
}
@@ -231,6 +230,7 @@ zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
pfree(zp);
return result;
}
+#endif
Datum
zlibhandler(PG_FUNCTION_ARGS)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index eb9e4b9774..36e7b2fa72 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -438,10 +438,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..de14d87885
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,405 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..755d40caef
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,409 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 744b9990a6..86e09a9c23 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -414,11 +414,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -427,11 +427,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -589,10 +589,10 @@ CREATE TABLE oids_parted (
) PARTITION BY RANGE (a) WITH OIDS;
CREATE TABLE part_forced_oids PARTITION OF oids_parted FOR VALUES FROM (1) TO (10) WITHOUT OIDS;
\d+ part_forced_oids
- Table "public.part_forced_oids"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_forced_oids"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: oids_parted FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a >= 1) AND (a < 10))
Has OIDs: yes
@@ -730,11 +730,11 @@ CREATE TABLE part_c PARTITION OF parted (b WITH OPTIONS NOT NULL DEFAULT 0) FOR
CREATE TABLE part_c_1_10 PARTITION OF part_c FOR VALUES FROM (1) TO (10);
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -743,11 +743,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -757,11 +757,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -794,46 +794,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -865,11 +865,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -878,10 +878,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -891,10 +891,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 8d4543bfe8..f046eee45f 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 0b5a9041b0..36151d0b06 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 75365501d4..54471fec02 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1337,12 +1337,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1358,12 +1358,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1382,12 +1382,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1425,12 +1425,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1450,17 +1450,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1482,17 +1482,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1524,17 +1524,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1562,12 +1562,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1599,12 +1599,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1643,12 +1643,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1674,12 +1674,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1701,12 +1701,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1728,12 +1728,12 @@ Inherits: fd_pt1
-- OID system column
ALTER TABLE fd_pt1 SET WITH OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1758,12 +1758,12 @@ ALTER TABLE ft2 SET WITHOUT OIDS; -- ERROR
ERROR: cannot drop inherited column "oid"
ALTER TABLE fd_pt1 SET WITHOUT OIDS;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1789,12 +1789,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1853,12 +1853,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1898,12 +1898,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1925,12 +1925,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1953,12 +1953,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1983,12 +1983,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2011,12 +2011,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 4f29d9f891..ea61f79883 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1023,13 +1023,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1042,14 +1042,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1059,14 +1059,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1106,33 +1106,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1143,27 +1143,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1173,37 +1173,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 5edf269367..ef1479f395 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 69ea821a0c..82dc565a36 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1735,7 +1735,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 67c34a92a4..b86883ce2d 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -158,13 +158,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index bc16ca4c43..9f7d16bef5 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 735dd37acf..f418ab58f9 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2803,11 +2803,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2823,11 +2823,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 9c7a60c092..a1a7228610 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -105,6 +107,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index b5e15501dd..d65ee16df0 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 49329ffbb6..096f9bff05 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -40,6 +40,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d7e6c5eba2
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,203 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index b155d90b11..248031e687 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1166,7 +1166,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
--
2.19.1
0008-Add-documentation-for-custom-compression-methods-v20.patchtext/x-patchDownload
From d205f1103c8410e79630b5294a896cfad02c5550 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 178 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 4256516c08..3910dfe8bd 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 48ac14a838..083e01e3ad 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index d758a4987d..c2ffe11cdc 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 0070603fc3..d957f9b08b 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -252,6 +252,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index f13a6cd944..499c32f243 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -351,6 +352,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 10428f8ff0..96b3487df9 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -921,6 +922,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 8ef2ac8010..4524eb65fc 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.19.1
Re: [HACKERS] Custom compression methods
On Thu, Sep 6, 2018 at 5:27 PM Ildus Kurbangaliev <i.kurbangaliev@postgrespro.ru> wrote:
Hi, attached latest set of patches. Rebased and fixed pg_upgrade errors
related with zlib support.
Thank you for working on this patch, I believe the ideas mentioned in this
thread are quite important for Postgres improvement. Unfortunately, patch has
some conflicts now, could you post a rebased version one more time?
On Mon, 23 Jul 2018 16:16:19 +0300
Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:I'm going to review this patch. Could you please rebase it? It
doesn't apply for me due to changes made in src/bin/psql/describe.c.
Is there any review underway, could you share the results?
Re: [HACKERS] Custom compression methods
On Fri, 30 Nov 2018 15:08:39 +0100
Dmitry Dolgov <9erthalion6@gmail.com> wrote:
On Thu, Sep 6, 2018 at 5:27 PM Ildus Kurbangaliev
<i.kurbangaliev@postgrespro.ru> wrote:Hi, attached latest set of patches. Rebased and fixed pg_upgrade
errors related with zlib support.Thank you for working on this patch, I believe the ideas mentioned in
this thread are quite important for Postgres improvement.
Unfortunately, patch has some conflicts now, could you post a rebased
version one more time?
Hi, here is a rebased version. I hope it will get some review :)
--
---
Ildus Kurbangaliev
Postgres Professional: http://www.postgrespro.com
Russian Postgres Company
Attachments:
0001-Make-syntax-changes-for-custom-compression-metho-v20.patchtext/x-patchDownload
From 74db207a6f68e53724c6af952a9772059ef808ac Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/parser/gram.y | 76 ++++++++++++++++++++++++++++++----
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 ++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 89 insertions(+), 9 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 2c2208ffb7..2087eeb08d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -399,6 +400,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -585,6 +587,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound partbound_datum_list range_datum_list
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -617,9 +623,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2213,6 +2219,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3358,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3371,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3417,6 +3433,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3622,6 +3675,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5298,12 +5352,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -15003,6 +15062,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 57d65f830f..f990e52025 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -53,6 +53,7 @@ typedef FormData_pg_am *Form_pg_am;
* Allowed values for amtype
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index cac6ff0eda..a392fcdd2c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -474,6 +474,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e5bdc1cec5..d7be502409 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -622,6 +622,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -645,6 +659,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -681,6 +696,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1800,7 +1816,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 23db40147b..76c33cbc5f 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.19.2
0002-Add-compression-catalog-tables-and-the-basic-inf-v20.patchtext/x-patchDownload
From fe239bc0db25eaae6eb16be51f273070d12caa64 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 2 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 84 +++
src/backend/commands/compressioncmds.c | 649 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 2 +
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/catalog/toasting.h | 1 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 3 +-
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
47 files changed, 2124 insertions(+), 193 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index bd93a6a8d1..4cd4fd9715 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index aa52a96259..9a1cebbe2f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index eece89aa21..92c72c8665 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -977,11 +977,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 9650145642..b843efd01e 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2676,8 +2676,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -4101,7 +4102,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 44caeca336..5c3126d826 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -666,7 +666,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
options |= HEAP_INSERT_NO_LOGICAL;
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
- options);
+ options, NULL);
}
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index fdbaf38126..c9753ad2f9 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
#include "utils/tqual.h"
@@ -53,19 +61,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -421,7 +462,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -511,6 +552,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -522,6 +655,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -532,7 +667,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -667,27 +802,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -739,12 +884,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -752,7 +899,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -768,10 +917,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -912,7 +1062,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1340,6 +1491,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1355,54 +1518,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1497,19 +1653,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1517,7 +1674,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1526,7 +1686,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1886,7 +2046,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2071,7 +2231,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2267,15 +2427,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2377,3 +2548,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate system cache so we need to call it before
+ * we're putting something to our cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreate(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index f395cb1ab4..353cd4bc39 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 7caab64ce7..08f7756e01 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -748,6 +748,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 0865240f11..5560fc05b8 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 7dfa3278a5..0d3bf0dd05 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -170,7 +171,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1280,6 +1282,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2538,6 +2544,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 11debaa780..87851cdbb0 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -677,6 +677,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1615,6 +1616,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 8709e8c22c..1914ac8afb 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -398,6 +398,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -484,6 +485,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index d5e30649ff..df62edb809 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -23,6 +23,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -523,6 +524,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3594,6 +3607,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4120,6 +4164,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4666,6 +4714,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index 4a6c99e090..e23abf64f1 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 21e9d3916a..71017b8173 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -631,6 +631,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 4367290a27..1f9128b7e4 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -179,6 +179,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -189,6 +253,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -229,6 +303,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -267,6 +343,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..7841c7700a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,649 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(((Form_pg_attr_compression) GETSTRUCT(tup))->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 3e7c1067d8..98104c4e70 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1216,6 +1216,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 10e9d7f562..58397e9d5d 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -179,7 +135,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 843ed48aa7..5617e16c09 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -22,6 +22,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -33,6 +34,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -40,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -89,6 +92,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -369,6 +373,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +472,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -520,6 +528,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -669,6 +678,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -715,6 +726,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -758,6 +776,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2122,6 +2157,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2148,6 +2196,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2357,6 +2406,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3435,6 +3491,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3894,6 +3951,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4251,6 +4309,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4316,8 +4379,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5460,6 +5524,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5624,6 +5698,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5746,6 +5821,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5768,6 +5867,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*
@@ -6583,6 +6782,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9429,6 +9632,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -9529,7 +9738,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -9554,6 +9764,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
heap_close(attrelation, RowExclusiveLock);
@@ -9562,6 +9800,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -12588,6 +12831,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index db49968409..d0849ebba0 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2869,6 +2869,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2887,6 +2888,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5566,6 +5579,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3a084b4d1f..2124c228d1 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2549,6 +2549,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2567,6 +2568,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3627,6 +3638,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index a10014f755..07ec871980 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3648,6 +3648,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3655,6 +3657,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0c396530d..a52e72fec4 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2870,6 +2870,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2886,6 +2887,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4210,6 +4221,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 52582d0a13..f2d62b0530 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1032,6 +1032,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 23466bade2..368beb9a64 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3118,7 +3118,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index dbe67cdb4c..4a97099cd0 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index c26808a833..1f0402029c 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 50690b9c9e..deb8196689 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -270,6 +270,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f99291e30d..793367ff58 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 46c271a46c..cb19cb3e41 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -180,10 +180,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 2359b4c629..859a27fe33 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -87,6 +87,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..90a8244a9b
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId)
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 1bac59bf26..69008693f7 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index 5a884a852b..e6d5e2cdda 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -36,7 +36,7 @@
reloftype => '0', relowner => 'PGUID', relam => '0', relfilenode => '0',
reltablespace => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasrules => 'f', relhastriggers => 'f', relhassubclass => 'f',
relrowsecurity => 'f', relforcerowsecurity => 'f', relispopulated => 't',
relreplident => 'n', relispartition => 'f', relrewrite => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 034a41eb55..2278d87d4d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6749,6 +6749,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -6918,6 +6921,13 @@
{ oid => '3312', descr => 'I/O',
proname => 'tsm_handler_out', prorettype => 'cstring',
proargtypes => 'tsm_handler', prosrc => 'tsm_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index d295eae1b9..0fb34cb680 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -574,6 +574,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index f259890e43..e5408e2763 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -71,6 +71,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 4187, 4188);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1d05a4bcdc..988986afce 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -156,8 +157,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index a392fcdd2c..95f1f935e7 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -506,7 +506,8 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
- T_CallContext /* in nodes/parsenodes.h */
+ T_CallContext, /* in nodes/parsenodes.h */
+ T_CompressionAmRoutine /* in access/cmapi.h */
} NodeTag;
/*
diff --git a/src/include/postgres.h b/src/include/postgres.h
index b596fcb513..916c75c7d8 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 6f290c7214..c87da19cc2 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9fe950b29d..1b0d714c36 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -351,6 +351,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -376,6 +377,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.19.2
0003-Add-rewrite-rules-and-tupdesc-flags-v20.patchtext/x-patchDownload
From 0437922f4de2442b2e9eb5fc19d002e0dc78786c Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 68 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 156 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index c82bbbaa7f..9a6aa844db 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 06dd628a5b..9f8307bd24 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -179,6 +179,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -209,6 +210,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -247,6 +251,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -270,6 +283,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -306,7 +322,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -330,6 +346,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -340,6 +357,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -754,6 +772,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -866,6 +885,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -884,6 +904,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -936,6 +957,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -946,6 +968,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1095,6 +1118,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1417,6 +1441,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 9a1cebbe2f..38fbdf7205 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 5354a04639..8e781fd67e 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -73,6 +74,7 @@ CreateTemplateTupleDesc(int natts)
desc->constr = NULL;
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -95,8 +97,17 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -137,6 +148,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -216,6 +228,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -301,6 +314,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -415,6 +429,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -465,6 +481,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -550,6 +568,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -655,6 +674,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index b843efd01e..404f92a207 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -96,7 +96,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2373,13 +2374,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2471,7 +2473,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2639,7 +2641,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2677,8 +2679,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2720,7 +2725,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* We're about to do the actual inserts -- but check for conflict first,
@@ -4000,6 +4005,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index c9753ad2f9..8d3a2d7897 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1191,6 +1191,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1412,6 +1413,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 4aa8890fe8..0ad51c249c 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2631,7 +2631,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index d01b258b65..c677f91d2f 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -557,7 +557,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index a171ebabf8..a73f9bd3f6 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -462,7 +462,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5617e16c09..bf36cbb844 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -166,6 +166,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -4610,7 +4612,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4873,6 +4875,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
heap_close(oldrel, NoLock);
if (newrel)
{
@@ -8895,6 +8915,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -9772,7 +9831,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -9780,7 +9839,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -12890,7 +12949,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 5561b741e9..fbe50092bc 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -808,6 +808,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index c3071db1cd..4c955e75ce 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -574,6 +574,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 64cfdbd2f0..fc5a066a71 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -22,6 +22,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -147,7 +148,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index 9993d5be70..b491bf6338 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -32,6 +32,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 708f73f0ea..855cc1b0cd 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -274,7 +274,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -673,6 +675,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -782,7 +787,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 06af39ff2a..afa3cd39c6 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -46,6 +48,10 @@ typedef struct tupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -82,6 +88,7 @@ typedef struct tupleDesc
int natts; /* number of attributes in the tuple */
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 793367ff58..50f20611ab 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 5a2fc6c8d7..10a85e08bd 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -31,6 +31,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.19.2
0004-Add-pglz-compression-method-v20.patchtext/x-patchDownload
From b8459f868449771a4d99aea0944fbeced26a8245 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index bef53a319a..6f7ad79613 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -30,5 +30,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2278d87d4d..9f855857e3 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -851,6 +851,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.19.2
0005-Add-zlib-compression-method-v20.patchtext/x-patchDownload
From 4d58c9dee48d34222bae8585828bbc0c3663643b Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 25eb043941..7d28a056e9 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6f7ad79613..b5754b9759 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9f855857e3..44e1c1c852 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -856,6 +856,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.19.2
0006-Add-psql-pg_dump-and-pg_upgrade-support-v20.patchtext/x-patchDownload
From 3c35a8073d9caeb356dad6b1636d6fe9abb8aa94 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/commands/compressioncmds.c | 80 ++++++---
src/backend/commands/tablecmds.c | 14 +-
src/backend/utils/adt/pg_upgrade_support.c | 10 ++
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_dump.c | 196 ++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 17 ++
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 95 ++++++++++
src/bin/psql/describe.c | 42 +++++
src/bin/psql/tab-complete.c | 5 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 +
13 files changed, 433 insertions(+), 42 deletions(-)
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 7841c7700a..89ffa227b0 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -36,6 +36,9 @@
#include "utils/syscache.h"
#include "utils/snapmgr.h"
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
/*
* When conditions of compression satisfies one if builtin attribute
* compresssion tuples the compressed attribute will be linked to
@@ -129,11 +132,12 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
tup_amoid;
Datum values[Natts_pg_attr_compression];
bool nulls[Natts_pg_attr_compression];
+ char *amname;
heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
- tup_amoid = get_am_oid(
- NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
if (previous_amoids)
*previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
@@ -150,17 +154,15 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
if (DatumGetPointer(acoptions) == NULL)
result = acoid;
}
- else
+ else if (DatumGetPointer(acoptions) != NULL)
{
bool equal;
/* check if arrays for WITH options are equal */
equal = DatumGetBool(CallerFInfoFunctionCall2(
- array_eq,
- &arrayeq_info,
- InvalidOid,
- acoptions,
- values[Anum_pg_attr_compression_acoptions - 1]));
+ array_eq, &arrayeq_info, InvalidOid, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
if (equal)
result = acoid;
}
@@ -227,6 +229,16 @@ CreateAttributeCompression(Form_pg_attribute att,
/* Try to find builtin compression first */
acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
/*
* attrelid will be invalid on CREATE TABLE, no need for table rewrite
* check.
@@ -252,16 +264,10 @@ CreateAttributeCompression(Form_pg_attribute att,
*/
if (need_rewrite != NULL)
{
- /* no rewrite by default */
- *need_rewrite = false;
-
Assert(preserved_amoids != NULL);
if (compression->preserve == NIL)
- {
- Assert(!IsBinaryUpgrade);
*need_rewrite = true;
- }
else
{
ListCell *cell;
@@ -294,7 +300,7 @@ CreateAttributeCompression(Form_pg_attribute att,
* In binary upgrade list will not be free since it contains
* Oid of builtin compression access method.
*/
- if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ if (list_length(previous_amoids) != 0)
*need_rewrite = true;
}
}
@@ -303,9 +309,6 @@ CreateAttributeCompression(Form_pg_attribute att,
list_free(previous_amoids);
}
- if (IsBinaryUpgrade && !OidIsValid(acoid))
- elog(ERROR, "could not restore attribute compression data");
-
/* Return Oid if we already found identical compression on this column */
if (OidIsValid(acoid))
{
@@ -315,6 +318,7 @@ CreateAttributeCompression(Form_pg_attribute att,
return acoid;
}
+add_tuple:
/* Initialize buffers for new tuple values */
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
@@ -323,13 +327,27 @@ CreateAttributeCompression(Form_pg_attribute att,
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
- acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
- Anum_pg_attr_compression_acoid);
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
if (acoid < FirstNormalObjectId)
{
- /* this is database initialization */
+ /* this is built-in attribute compression */
heap_close(rel, RowExclusiveLock);
- return DefaultCompressionOid;
+ return acoid;
}
/* we need routine only to call cmcheck function */
@@ -390,8 +408,8 @@ RemoveAttributeCompression(Oid acoid)
/*
* CleanupAttributeCompression
*
- * Remove entries in pg_attr_compression except current attribute compression
- * and related with specified list of access methods.
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
*/
void
CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
@@ -419,9 +437,7 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
ReleaseSysCache(attrtuple);
Assert(relid > 0 && attnum > 0);
-
- if (IsBinaryUpgrade)
- goto builtin_removal;
+ Assert(!IsBinaryUpgrade);
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
@@ -438,7 +454,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
true, NULL, 2, key);
- /* Remove attribute compression tuples and collect removed Oids to list */
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
while (HeapTupleIsValid(tuple = systable_getnext(scan)))
{
Form_pg_attr_compression acform;
@@ -460,7 +479,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
systable_endscan(scan);
heap_close(rel, RowExclusiveLock);
- /* Now remove dependencies */
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
rel = heap_open(DependRelationId, RowExclusiveLock);
foreach(lc, removed)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index bf36cbb844..54e6671a72 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -729,10 +729,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
- if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression = CreateAttributeCompression(attr,
- colDef->compression,
- NULL, NULL);
+ colDef->compression, NULL, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -12965,14 +12965,6 @@ ATExecSetCompression(AlteredTableInfo *tab,
/* make changes visible */
CommandCounterIncrement();
- /*
- * Normally cleanup is done in rewrite but in binary upgrade we should do
- * it explicitly.
- */
- if (IsBinaryUpgrade)
- CleanupAttributeCompression(RelationGetRelid(rel),
- attnum, preserved_amoids);
-
ObjectAddressSet(address, AttrCompressionRelationId, acoid);
return address;
}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index b8b7777c31..1082eab4dc 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -116,6 +116,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 4a2e122e2d..0f1d15ed82 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -150,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d583154fba..ec45d52f7d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -375,6 +377,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
@@ -831,6 +834,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8083,9 +8088,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
+ bool createWithCompression;
for (i = 0; i < numTables; i++)
{
@@ -8128,6 +8136,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attislocal,\n"
"pg_catalog.format_type(t.oid, a.atttypmod) AS atttypname,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBuffer(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8180,7 +8205,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8208,6 +8239,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8225,9 +8258,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (j = 0; j < ntups; j++)
@@ -8253,6 +8289,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -8470,6 +8508,104 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ if (g_verbose)
+ write_msg(NULL, "finding compression info for table \"%s.%s\"\n",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -12546,6 +12682,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15477,6 +15616,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15535,6 +15682,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -15950,6 +16116,34 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "OPTIONS (\n %s\n);\n",
tbinfo->attfdwoptions[j]);
}
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
}
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 789d6a24e2..4af95caade 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,10 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
+ char **attcmoptions; /* per-attribute current compression options */
+ char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
+
/*
* Stuff computed only for dumpable tables.
*/
@@ -346,6 +350,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 5176626476..d1e13c643a 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -136,6 +137,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -406,6 +408,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -622,6 +626,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 44012ff44d..fd45a0d7c2 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -375,6 +377,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 2afd950591..09ab16527c 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -631,6 +631,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1332,6 +1369,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2324,6 +2372,53 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 0a181b01d9..0d06efe58f 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1467,6 +1467,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1852,6 +1853,24 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1978,6 +1997,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2049,6 +2070,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fa44b2820b..3cc5dbf6d1 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1853,11 +1853,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index abc6e1ae1d..1e95a3863a 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -25,6 +25,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 44e1c1c852..73697a9f08 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9877,6 +9877,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4012', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# replication/origin.h
{ oid => '6003', descr => 'create a replication origin',
--
2.19.2
0007-Add-tests-for-compression-methods-v20.patchtext/x-patchDownload
From 2df3c676b2e83d70c2762035fea3d5e064fedf3c Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/backend/access/compression/cm_zlib.c | 2 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 405 +++++++++++++++++
src/test/regress/expected/create_cm_1.out | 409 ++++++++++++++++++
src/test/regress/expected/create_table.out | 124 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 ++---
src/test/regress/expected/opr_sanity.out | 4 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 203 +++++++++
src/test/regress/sql/opr_sanity.sql | 4 +-
22 files changed, 1472 insertions(+), 446 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/expected/create_cm_1.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index b7c76469fc..71251cb8cb 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -420,12 +420,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -434,12 +434,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -447,12 +447,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -465,13 +465,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index 0dcb56ddf3..dc8666f309 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -182,7 +182,6 @@ zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
}
pfree(tmp);
-#endif
return NULL;
}
@@ -231,6 +230,7 @@ zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
pfree(zp);
return result;
}
+#endif
Datum
zlibhandler(PG_FUNCTION_ARGS)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 19bb538411..3df7930aee 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -426,10 +426,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..de14d87885
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,405 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..755d40caef
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,409 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index b26b4e7b6d..34939f356e 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -431,11 +431,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -444,11 +444,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -750,11 +750,11 @@ create table parted_collate_must_match2 partition of parted_collate_must_match
drop table parted_collate_must_match;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
Check constraints:
@@ -763,11 +763,11 @@ Check constraints:
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -777,11 +777,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
Check constraints:
@@ -814,46 +814,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -885,11 +885,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -898,10 +898,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -911,10 +911,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index b582211270..d830ce830c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 0b5a9041b0..36151d0b06 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 4d82d3a7e8..8c8a03cfc2 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1332,12 +1332,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1353,12 +1353,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1377,12 +1377,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1420,12 +1420,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1445,17 +1445,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1477,17 +1477,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1519,17 +1519,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1557,12 +1557,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1594,12 +1594,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1638,12 +1638,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1669,12 +1669,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1696,12 +1696,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1727,12 +1727,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1791,12 +1791,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1836,12 +1836,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1863,12 +1863,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1921,12 +1921,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1949,12 +1949,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index f259d07535..1584230f35 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -992,13 +992,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1011,14 +1011,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1028,14 +1028,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1075,33 +1075,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1112,27 +1112,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1142,37 +1142,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 1cf6531c01..37f96ea7a2 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 6072f6bdb1..eedcfbc678 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1745,7 +1745,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 175ecd2879..7cf1927971 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 1d12b01068..5653637c30 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 735dd37acf..f418ab58f9 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2803,11 +2803,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2823,11 +2823,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 009a89fc1a..89a6ac91e8 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -106,6 +108,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index d09326c182..cdc93b2816 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -659,14 +659,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index cc0bbf5db9..6a31f7c352 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0c10c7100c..ac9b36f8d4 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -40,6 +40,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d7e6c5eba2
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,203 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index 91c68f4204..1d41ffef2f 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1174,7 +1174,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
--
2.19.2
0008-Add-documentation-for-custom-compression-methods-v20.patchtext/x-patchDownload
From f1f38e36fb21f99b4a89f77c52ea07060b11ba87 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 178 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index c134bca809..8ba21e0fbb 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 5dfdf54815..520cc5c5f2 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -90,6 +90,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index d758a4987d..c2ffe11cdc 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 96d196d229..17f2cc98b3 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index be1647937d..a0365f7ff1 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -350,6 +351,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index d3e33132f3..ebf8ef5f0e 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">action</replaceable> ] [ ON UPDATE <replaceable class="parameter">action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -918,6 +919,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 8ef2ac8010..4524eb65fc 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.19.2
Re: [HACKERS] Custom compression methods
On Mon, Dec 03, 2018 at 03:43:32PM +0300, Ildus Kurbangaliev wrote:
Hi, here is a rebased version. I hope it will get some review :)
This patch set is failing to apply, so moved to next CF, waiting for
author.
--
Michael
Re: [HACKERS] Custom compression methods
Hi,
there are another set of patches.
Only rebased to current master.
Also I will change status on commitfest to 'Needs review'.
--
Regards,
Ildus Kurbangaliev
Attachments:
0007-Add-tests-for-compression-methods-v21.patchtext/x-patchDownload
From a77fba6dd2a090a0fb5fc06568853483ce9dd73d Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/backend/access/compression/cm_zlib.c | 2 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 405 +++++++++++++++++
src/test/regress/expected/create_cm_1.out | 409 ++++++++++++++++++
src/test/regress/expected/create_table.out | 132 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 ++---
src/test/regress/expected/opr_sanity.out | 4 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 203 +++++++++
src/test/regress/sql/opr_sanity.sql | 4 +-
22 files changed, 1476 insertions(+), 450 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/expected/create_cm_1.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 2bd28e6d15..01947e0696 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index 0dcb56ddf3..dc8666f309 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -182,7 +182,6 @@ zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
}
pfree(tmp);
-#endif
return NULL;
}
@@ -231,6 +230,7 @@ zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
pfree(zp);
return result;
}
+#endif
Datum
zlibhandler(PG_FUNCTION_ARGS)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 75d4119eaa..54fe30b3c0 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..de14d87885
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,405 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..755d40caef
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,409 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index d51e547278..bd84883175 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -445,11 +445,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -458,11 +458,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -479,10 +479,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -832,21 +832,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -854,11 +854,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -887,46 +887,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -958,11 +958,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -971,10 +971,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -984,10 +984,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index b582211270..d830ce830c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 4ff1b4af41..5255951060 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 4d82d3a7e8..8c8a03cfc2 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1332,12 +1332,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1353,12 +1353,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1377,12 +1377,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1420,12 +1420,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1445,17 +1445,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1477,17 +1477,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1519,17 +1519,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1557,12 +1557,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1594,12 +1594,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1638,12 +1638,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1669,12 +1669,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1696,12 +1696,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1727,12 +1727,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1791,12 +1791,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1836,12 +1836,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1863,12 +1863,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1921,12 +1921,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1949,12 +1949,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 565d947b6d..285e36fb21 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1033,13 +1033,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1052,14 +1052,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1069,14 +1069,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1116,33 +1116,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1153,27 +1153,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1183,37 +1183,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 1cf6531c01..37f96ea7a2 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index ce25ee044a..d8cd5d8fde 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1806,7 +1806,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
oid | amname | oid | proname
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 175ecd2879..7cf1927971 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 2e170497c9..df49e98251 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 98f417cb57..a0b03839d5 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2814,11 +2814,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2834,11 +2834,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 6cd937eb52..de8be73a54 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -106,6 +108,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index 2083345c8e..b245059e37 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -690,14 +690,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 4051a4ad4e..f591a981ce 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index ac1ea622d6..ae4724e9e7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -40,6 +40,7 @@ test: inet
test: macaddr
test: macaddr8
test: tstypes
+test: create_cm
test: geometry
test: horology
test: regex
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d7e6c5eba2
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,203 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index e2014fc2b5..64bbe004b0 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1206,7 +1206,9 @@ WHERE p1.amhandler = 0;
SELECT p1.oid, p1.amname, p2.oid, p2.proname
FROM pg_am AS p1, pg_proc AS p2
WHERE p2.oid = p1.amhandler AND
- (p2.prorettype != 'index_am_handler'::regtype OR p2.proretset
+ ((p2.prorettype != 'index_am_handler'::regtype
+ AND p2.prorettype != 'compression_am_handler'::regtype)
+ OR p2.proretset
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
--
2.20.1
0001-Make-syntax-changes-for-custom-compression-metho-v21.patchtext/x-patchDownload
From 083134470ea3cb41ae69a4ab7f0c0567ca5bb760 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/parser/gram.y | 76 ++++++++++++++++++++++++++++++----
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 ++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 89 insertions(+), 9 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0279013120..5f4194f6e3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -311,6 +311,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
opt_grant_grant_option opt_grant_admin_option
opt_nowait opt_if_exists opt_with_data
%type <ival> opt_nowait_or_skip
+%type <ival> AccessMethodType
%type <list> OptRoleList AlterOptRoleList
%type <defelt> CreateOptRoleElem AlterOptRoleElem
@@ -399,6 +400,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -584,6 +586,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -616,9 +622,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2213,6 +2219,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3313,11 +3328,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3326,8 +3342,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3372,6 +3388,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3577,6 +3630,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5253,12 +5307,17 @@ row_security_cmd:
*
*****************************************************************************/
-CreateAmStmt: CREATE ACCESS METHOD name TYPE_P INDEX HANDLER handler_name
+AccessMethodType:
+ INDEX { $$ = AMTYPE_INDEX; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
+ ;
+
+CreateAmStmt: CREATE ACCESS METHOD name TYPE_P AccessMethodType HANDLER handler_name
{
CreateAmStmt *n = makeNode(CreateAmStmt);
n->amname = $4;
n->handler_name = $8;
- n->amtype = AMTYPE_INDEX;
+ n->amtype = $6;
$$ = (Node *) n;
}
;
@@ -14986,6 +15045,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index eb3495c36a..b341e9e88e 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -53,6 +53,7 @@ typedef FormData_pg_am *Form_pg_am;
* Allowed values for amtype
*/
#define AMTYPE_INDEX 'i' /* index access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index f9389257c6..9bc8c444ca 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -475,6 +475,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index a7e859dc90..fe54b80b72 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -622,6 +622,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -645,6 +659,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -681,6 +696,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1811,7 +1827,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f05444008c..a722891c99 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.20.1
0002-Add-compression-catalog-tables-and-the-basic-inf-v21.patchtext/x-patchDownload
From e34210094c0049e4f1482e79a4e991609075c86c Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 2 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 84 +++
src/backend/commands/compressioncmds.c | 649 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 2 +
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/catalog/toasting.h | 1 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 1 +
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
47 files changed, 2123 insertions(+), 192 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ca00737c53 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- table tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 6a22b17203..91c229f688 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index cdf1f4af62..20cc73e485 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -968,11 +968,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index dc3499349b..188923231b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2386,8 +2386,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -3808,7 +3809,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index f5cf9ffc9c..6082c48629 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -665,7 +665,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
options |= HEAP_INSERT_NO_LOGICAL;
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
- options);
+ options, NULL);
}
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index cd921a4600..6082ee9a38 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -52,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -82,7 +122,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -420,7 +461,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -510,6 +551,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -521,6 +654,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -531,7 +666,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -666,27 +801,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -738,12 +883,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -751,7 +898,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -767,10 +916,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -911,7 +1061,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1339,6 +1490,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1354,54 +1517,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1496,19 +1652,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1516,7 +1673,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1525,7 +1685,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1885,7 +2045,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2072,7 +2232,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2265,15 +2425,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2375,3 +2546,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate system cache so we need to call it before
+ * we're putting something to our cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreate(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index 450a7dce1f..6bc0a56e2b 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 4d7ed8ad1a..1330ebac42 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -758,6 +758,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index f186198fc6..31aed9bb9a 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 2048d71535..8f4271a8f8 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -25,6 +25,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -181,7 +182,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1475,6 +1477,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2753,6 +2759,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 7dba4e50dd..fe87b6c2e0 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -688,6 +688,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1632,6 +1633,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index d16c3d0ea5..2c845d1774 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -396,6 +396,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -482,6 +483,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8b51ec7f39..5881298a65 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -26,6 +26,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -525,6 +526,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3596,6 +3609,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4125,6 +4169,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4671,6 +4719,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d64628566d..ce7ef52e7a 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 9229fe1a45..ff4ef9b55b 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -632,6 +632,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index c84507b5d0..a665fa9b47 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -179,6 +179,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -189,6 +253,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -229,6 +303,8 @@ get_am_type_string(char amtype)
{
case AMTYPE_INDEX:
return "INDEX";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -267,6 +343,14 @@ lookup_index_am_handler_func(List *handler_name, char amtype)
NameListToString(handler_name),
"index_am_handler")));
break;
+ case AMTYPE_COMPRESSION:
+ if (get_func_rettype(handlerOid) != COMPRESSION_AM_HANDLEROID)
+ ereport(ERROR,
+ (errcode(ERRCODE_WRONG_OBJECT_TYPE),
+ errmsg("function %s must return type %s",
+ NameListToString(handler_name),
+ "compression_am_handler")));
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..7841c7700a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,649 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(((Form_pg_attr_compression) GETSTRUCT(tup))->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index adb77d8f69..eff30c937c 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1216,6 +1216,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 413ce3fcb6..7701dd8b80 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -179,7 +135,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a93b13c2fe..2f31eea25c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -22,6 +22,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -33,6 +34,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -40,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -86,6 +89,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -365,6 +369,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(Relation rel, bool recurse, bool recursing);
@@ -466,6 +472,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -520,6 +528,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -729,6 +738,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -775,6 +786,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -818,6 +836,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2182,6 +2217,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2208,6 +2256,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2417,6 +2466,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3501,6 +3557,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -3960,6 +4017,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4319,6 +4377,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4384,8 +4447,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5531,6 +5595,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5695,6 +5769,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5817,6 +5892,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5839,6 +5938,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*
@@ -6654,6 +6853,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9913,6 +10116,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -10013,7 +10222,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -10103,6 +10313,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -10111,6 +10349,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -13143,6 +13386,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index e15724bb0e..08820e6681 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2870,6 +2870,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2888,6 +2889,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5567,6 +5580,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 31499eb798..dca0d5130e 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2550,6 +2550,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2568,6 +2569,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3632,6 +3643,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 8ed30c011a..3caae15f7c 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3688,6 +3688,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3695,6 +3697,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 65302fe65b..77e89dbc95 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2773,6 +2773,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2789,6 +2790,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4115,6 +4126,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a37d1f18be..90b407b0f8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1037,6 +1037,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2b486b5e9f..0fe6681fd7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3122,7 +3122,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 6194dcd2fe..1d1f30957f 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -418,3 +418,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index ac98c19155..01b2459719 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 7ade18ea46..820015affc 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -269,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 4bfefffbf3..fad14bad85 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b235a23f5d..e4a5957ab4 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index 833fad1f6a..1d1502d550 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -87,6 +87,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..90a8244a9b
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId)
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a6ec122389..ed75a09131 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index cccad25c14..84744a15ca 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -36,7 +36,7 @@
reloftype => '0', relowner => 'PGUID', relam => '0', relfilenode => '0',
reltablespace => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasrules => 'f', relhastriggers => 'f', relhassubclass => 'f',
relrowsecurity => 'f', relforcerowsecurity => 'f', relispopulated => 't',
relreplident => 'n', relispartition => 'f', relrewrite => '0',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a4e173b484..be2918be55 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6812,6 +6812,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -6981,6 +6984,13 @@
{ oid => '3312', descr => 'I/O',
proname => 'tsm_handler_out', prorettype => 'cstring',
proargtypes => 'tsm_handler', prosrc => 'tsm_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 4b7750d439..4dbd611ba6 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -574,6 +574,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 37969719a0..0053806cd1 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -71,6 +71,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 4187, 4188);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e592a914a4..62ccd203a8 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -156,8 +157,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 9bc8c444ca..df403c4813 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -505,6 +505,7 @@ typedef enum NodeTag
T_InlineCodeBlock, /* in nodes/parsenodes.h */
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
+ T_CompressionAmRoutine, /* in access/cmapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 057a3413ac..a8e818347b 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 95ee48954e..812e4fd43a 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 3d3c76d251..3cc38690cc 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -351,6 +351,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -376,6 +377,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.20.1
0003-Add-rewrite-rules-and-tupdesc-flags-v21.patchtext/x-patchDownload
From 31c016f2203d83f4c351b8d9ed1e43c8aca09d29 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 68 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 156 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a5bc6f5749..df391bcc78 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 783b04a3cb..b05dee6195 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -177,6 +177,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -207,6 +208,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -245,6 +249,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -268,6 +281,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -304,7 +320,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -328,6 +344,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -338,6 +355,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -752,6 +770,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -864,6 +883,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -882,6 +902,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -934,6 +955,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -944,6 +966,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1093,6 +1116,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1415,6 +1439,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 91c229f688..dd133684ee 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 47e80ae186..8a80a74a15 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -21,6 +21,7 @@
#include "access/hash.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -73,6 +74,7 @@ CreateTemplateTupleDesc(int natts)
desc->constr = NULL;
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -95,8 +97,17 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -137,6 +148,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -216,6 +228,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -301,6 +314,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -415,6 +429,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -465,6 +481,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -550,6 +568,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -655,6 +674,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 188923231b..3fb0c0f189 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -85,7 +85,8 @@ static HeapScanDesc heap_beginscan_internal(Relation relation,
static void heap_parallelscan_startblock_init(HeapScanDesc scan);
static BlockNumber heap_parallelscan_nextpage(HeapScanDesc scan);
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -2086,13 +2087,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -2182,7 +2184,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2349,7 +2351,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2387,8 +2389,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2430,7 +2435,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* We're about to do the actual inserts -- but check for conflict first,
@@ -3707,6 +3712,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 6082ee9a38..176416a943 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1190,6 +1190,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1411,6 +1412,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 5dd6fe02c6..a787bc50f7 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2661,7 +2661,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 6517ecb738..4fd8dce3cc 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -558,7 +558,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 5a47be4b33..982a287e8c 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -464,7 +464,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2f31eea25c..8e6e22c1e4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -162,6 +162,8 @@ typedef struct AlteredTableInfo
/* Information saved by Phases 1/2 for Phase 3: */
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
@@ -4678,7 +4680,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -4944,6 +4946,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
table_close(oldrel, NoLock);
if (newrel)
{
@@ -9351,6 +9371,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -10321,7 +10380,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -10329,7 +10388,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -13445,7 +13504,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 9971abd71f..71fffc53b8 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -808,6 +808,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 54a40ef00b..df892441d4 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -569,6 +569,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ab0879138f..07fa8b11f7 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -24,6 +24,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -129,7 +130,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index cec087cb1a..84c736cea2 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -31,6 +31,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 6d51f9062b..1ab905a131 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -274,7 +274,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -673,6 +675,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -779,7 +784,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 66d1b2fc40..c878a0636a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -44,6 +46,10 @@ typedef struct TupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct TupleDescData
int natts; /* number of attributes in the tuple */
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fad14bad85..aaf3b155dc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0b8c7cca21..9252ad3d3a 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -31,6 +31,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.20.1
0004-Add-pglz-compression-method-v21.patchtext/x-patchDownload
From ca4dc711c2669718ad10cd47ec9cbfe0631a891e Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 08f331d4e1..adfc10c443 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -30,5 +30,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index be2918be55..5b57d46afe 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -902,6 +902,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.20.1
0005-Add-zlib-compression-method-v21.patchtext/x-patchDownload
From 7273857dd0d91c5b357b4ac996950d0db873c77b Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 478a96db9b..bd5009e190 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index adfc10c443..8c101ed157 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5b57d46afe..e3dbdb623b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -907,6 +907,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.20.1
0006-Add-psql-pg_dump-and-pg_upgrade-support-v21.patchtext/x-patchDownload
From ceed42f608155c45ef3bff70c7aec59ec4a1499c Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/commands/compressioncmds.c | 80 ++++++---
src/backend/commands/tablecmds.c | 14 +-
src/backend/utils/adt/pg_upgrade_support.c | 10 ++
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_dump.c | 196 ++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 17 ++
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 95 ++++++++++
src/bin/psql/describe.c | 42 +++++
src/bin/psql/tab-complete.c | 5 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 +
13 files changed, 433 insertions(+), 42 deletions(-)
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 7841c7700a..89ffa227b0 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -36,6 +36,9 @@
#include "utils/syscache.h"
#include "utils/snapmgr.h"
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
/*
* When conditions of compression satisfies one if builtin attribute
* compresssion tuples the compressed attribute will be linked to
@@ -129,11 +132,12 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
tup_amoid;
Datum values[Natts_pg_attr_compression];
bool nulls[Natts_pg_attr_compression];
+ char *amname;
heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
- tup_amoid = get_am_oid(
- NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
if (previous_amoids)
*previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
@@ -150,17 +154,15 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
if (DatumGetPointer(acoptions) == NULL)
result = acoid;
}
- else
+ else if (DatumGetPointer(acoptions) != NULL)
{
bool equal;
/* check if arrays for WITH options are equal */
equal = DatumGetBool(CallerFInfoFunctionCall2(
- array_eq,
- &arrayeq_info,
- InvalidOid,
- acoptions,
- values[Anum_pg_attr_compression_acoptions - 1]));
+ array_eq, &arrayeq_info, InvalidOid, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
if (equal)
result = acoid;
}
@@ -227,6 +229,16 @@ CreateAttributeCompression(Form_pg_attribute att,
/* Try to find builtin compression first */
acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
/*
* attrelid will be invalid on CREATE TABLE, no need for table rewrite
* check.
@@ -252,16 +264,10 @@ CreateAttributeCompression(Form_pg_attribute att,
*/
if (need_rewrite != NULL)
{
- /* no rewrite by default */
- *need_rewrite = false;
-
Assert(preserved_amoids != NULL);
if (compression->preserve == NIL)
- {
- Assert(!IsBinaryUpgrade);
*need_rewrite = true;
- }
else
{
ListCell *cell;
@@ -294,7 +300,7 @@ CreateAttributeCompression(Form_pg_attribute att,
* In binary upgrade list will not be free since it contains
* Oid of builtin compression access method.
*/
- if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ if (list_length(previous_amoids) != 0)
*need_rewrite = true;
}
}
@@ -303,9 +309,6 @@ CreateAttributeCompression(Form_pg_attribute att,
list_free(previous_amoids);
}
- if (IsBinaryUpgrade && !OidIsValid(acoid))
- elog(ERROR, "could not restore attribute compression data");
-
/* Return Oid if we already found identical compression on this column */
if (OidIsValid(acoid))
{
@@ -315,6 +318,7 @@ CreateAttributeCompression(Form_pg_attribute att,
return acoid;
}
+add_tuple:
/* Initialize buffers for new tuple values */
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
@@ -323,13 +327,27 @@ CreateAttributeCompression(Form_pg_attribute att,
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
- acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
- Anum_pg_attr_compression_acoid);
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
if (acoid < FirstNormalObjectId)
{
- /* this is database initialization */
+ /* this is built-in attribute compression */
heap_close(rel, RowExclusiveLock);
- return DefaultCompressionOid;
+ return acoid;
}
/* we need routine only to call cmcheck function */
@@ -390,8 +408,8 @@ RemoveAttributeCompression(Oid acoid)
/*
* CleanupAttributeCompression
*
- * Remove entries in pg_attr_compression except current attribute compression
- * and related with specified list of access methods.
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
*/
void
CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
@@ -419,9 +437,7 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
ReleaseSysCache(attrtuple);
Assert(relid > 0 && attnum > 0);
-
- if (IsBinaryUpgrade)
- goto builtin_removal;
+ Assert(!IsBinaryUpgrade);
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
@@ -438,7 +454,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
true, NULL, 2, key);
- /* Remove attribute compression tuples and collect removed Oids to list */
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
while (HeapTupleIsValid(tuple = systable_getnext(scan)))
{
Form_pg_attr_compression acform;
@@ -460,7 +479,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
systable_endscan(scan);
heap_close(rel, RowExclusiveLock);
- /* Now remove dependencies */
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
rel = heap_open(DependRelationId, RowExclusiveLock);
foreach(lc, removed)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8e6e22c1e4..af81480735 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -789,10 +789,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
- if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression = CreateAttributeCompression(attr,
- colDef->compression,
- NULL, NULL);
+ colDef->compression, NULL, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -13520,14 +13520,6 @@ ATExecSetCompression(AlteredTableInfo *tab,
/* make changes visible */
CommandCounterIncrement();
- /*
- * Normally cleanup is done in rewrite but in binary upgrade we should do
- * it explicitly.
- */
- if (IsBinaryUpgrade)
- CleanupAttributeCompression(RelationGetRelid(rel),
- attnum, preserved_amoids);
-
ObjectAddressSet(address, AttrCompressionRelationId, acoid);
return address;
}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 99db5ba389..0e81e70e09 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -116,6 +116,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 4a2e122e2d..0f1d15ed82 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -150,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index a08bc4ecae..7824cad64a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -40,11 +40,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -380,6 +382,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
@@ -846,6 +849,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8115,9 +8120,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
+ bool createWithCompression;
for (i = 0; i < numTables; i++)
{
@@ -8160,6 +8168,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attislocal,\n"
"pg_catalog.format_type(t.oid, a.atttypmod) AS atttypname,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBuffer(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8212,7 +8237,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8240,6 +8271,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8257,9 +8290,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (j = 0; j < ntups; j++)
@@ -8285,6 +8321,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -8502,6 +8540,104 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ if (g_verbose)
+ write_msg(NULL, "finding compression info for table \"%s.%s\"\n",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -12604,6 +12740,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_INDEX:
appendPQExpBuffer(q, "TYPE INDEX ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
+ break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
aminfo->amtype, qamname);
@@ -15517,6 +15656,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15575,6 +15722,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -15990,6 +16156,34 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "OPTIONS (\n %s\n);\n",
tbinfo->attfdwoptions[j]);
}
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 21d2ab05b0..180b3dc9a0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,10 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
+ char **attcmoptions; /* per-attribute current compression options */
+ char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
+
/*
* Stuff computed only for dumpable tables.
*/
@@ -346,6 +350,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 44c3350887..8178377d1c 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -74,6 +74,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -137,6 +138,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -412,6 +414,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -629,6 +633,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 428e040acb..03f84933c6 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -375,6 +377,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 0233fcb47f..84b3e8982f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -631,6 +631,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1332,6 +1369,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2379,6 +2427,53 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 4da6719ce7..1170e6352d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1467,6 +1467,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1852,6 +1853,24 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1978,6 +1997,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2049,6 +2070,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 7b7a88fda3..2215cbe117 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1919,11 +1919,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index 2927b7a4d3..00f91e4b90 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -25,6 +25,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index e3dbdb623b..db5be57e06 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9950,6 +9950,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4012', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# conversion functions
{ oid => '4300',
--
2.20.1
0008-Add-documentation-for-custom-compression-methods-v21.patchtext/x-patchDownload
From 852fe15115798cf10ae1c8fcb95cc78fc015e3bd Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 178 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0fd792ff1a..c8551c2fc3 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a03ea1427b..8f482a60dd 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -90,6 +90,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 05102724ea..54110050d1 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 96d196d229..17f2cc98b3 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 0aa0f093f2..122a17c1cf 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -350,6 +351,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 22dbc07b23..66d94df3dd 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -920,6 +921,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index cbdad0c3fb..89a5889d0f 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.20.1
Re: Re: [HACKERS] Custom compression methods
On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I can see
since last January 2018.
As Andres noted in [1]/messages/by-id/20190216054526.zss2cufdxfeudr4i@alap3.anarazel.de, I think that we need to decide if this is a
feature that we want rather than just continuing to push it from CF to CF.
--
-David
david@pgmasters.net
[1]: /messages/by-id/20190216054526.zss2cufdxfeudr4i@alap3.anarazel.de
/messages/by-id/20190216054526.zss2cufdxfeudr4i@alap3.anarazel.de
Re: Re: [HACKERS] Custom compression methods
On Thu, Mar 7, 2019 at 10:43 AM David Steele <david@pgmasters.net> wrote:
On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I can see
since last January 2018.As Andres noted in [1], I think that we need to decide if this is a
feature that we want rather than just continuing to push it from CF to CF.
Yes. I took a look at code of this patch. I think it's in pretty good
shape. But high level review/discussion is required.
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: Re: Re: [HACKERS] Custom compression methods
On 3/7/19 11:50 AM, Alexander Korotkov wrote:
On Thu, Mar 7, 2019 at 10:43 AM David Steele <david@pgmasters.net
<mailto:david@pgmasters.net>> wrote:On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I can see
since last January 2018.As Andres noted in [1], I think that we need to decide if this is a
feature that we want rather than just continuing to push it from CF
to CF.Yes. I took a look at code of this patch. I think it's in pretty good
shape. But high level review/discussion is required.
OK, but I think this patch can only be pushed one more time, maximum,
before it should be rejected.
Regards,
--
-David
david@pgmasters.net
Re: [HACKERS] Custom compression methods
On Fri, 15 Mar 2019 14:07:14 +0400
David Steele <david@pgmasters.net> wrote:
On 3/7/19 11:50 AM, Alexander Korotkov wrote:
On Thu, Mar 7, 2019 at 10:43 AM David Steele <david@pgmasters.net
<mailto:david@pgmasters.net>> wrote:On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I
can see since last January 2018.As Andres noted in [1], I think that we need to decide if this
is a feature that we want rather than just continuing to push it
from CF to CF.Yes. I took a look at code of this patch. I think it's in pretty
good shape. But high level review/discussion is required.OK, but I think this patch can only be pushed one more time, maximum,
before it should be rejected.Regards,
Hi,
in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.
Attached latest set of patches.
--
Best regards,
Ildus Kurbangaliev
Attachments:
0001-Make-syntax-changes-for-custom-compression-metho-v22.patchtext/x-patchDownload
From cd888a98435ccc4212f9b9bcee6da0bcb3bc9f4a Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 12:25:50 +0300
Subject: [PATCH 1/8] Make syntax changes for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/parser/gram.y | 67 +++++++++++++++++++++++++++++++---
src/include/catalog/pg_am.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 +++++++++-
src/include/parser/kwlist.h | 1 +
5 files changed, 82 insertions(+), 7 deletions(-)
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index e23e68fdb3..cd2f791ea9 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -402,6 +402,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -587,6 +588,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -619,9 +624,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2216,6 +2221,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3323,11 +3337,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3336,8 +3351,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3382,6 +3397,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3587,6 +3639,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5285,6 +5338,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
@@ -15015,6 +15069,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 706b5e81cb..8199f5c05d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -54,6 +54,7 @@ typedef FormData_pg_am *Form_pg_am;
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ffb4cd4bcc..2451d650ab 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -475,6 +475,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fe35783359..6800fcf7c4 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -622,6 +622,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -645,6 +659,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -681,6 +696,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 4,
CREATE_TABLE_LIKE_STATISTICS = 1 << 5,
CREATE_TABLE_LIKE_STORAGE = 1 << 6,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 7,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1811,7 +1827,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index f05444008c..a722891c99 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
--
2.21.0
0002-Add-compression-catalog-tables-and-the-basic-inf-v22.patchtext/x-patchDownload
From 2f599a0a25b604b8e49a685dcd68a018ae27d8ec Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:26:13 +0300
Subject: [PATCH 2/8] Add compression catalog tables and the basic
infrastructure
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/Makefile | 4 +-
src/backend/access/common/indextuple.c | 25 +-
src/backend/access/common/reloptions.c | 89 +++
src/backend/access/compression/Makefile | 17 +
src/backend/access/compression/cmapi.c | 96 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/rewriteheap.c | 2 +-
src/backend/access/heap/tuptoaster.c | 475 +++++++++++--
src/backend/access/index/amapi.c | 47 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 11 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/objectaddress.c | 57 ++
src/backend/commands/Makefile | 4 +-
src/backend/commands/alter.c | 1 +
src/backend/commands/amcmds.c | 79 +++
src/backend/commands/compressioncmds.c | 649 ++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/tablecmds.c | 336 ++++++++-
src/backend/nodes/copyfuncs.c | 16 +
src/backend/nodes/equalfuncs.c | 14 +
src/backend/nodes/nodeFuncs.c | 10 +
src/backend/nodes/outfuncs.c | 14 +
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 12 +
src/include/access/cmapi.h | 80 +++
src/include/access/reloptions.h | 2 +
src/include/access/tuptoaster.h | 42 +-
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 23 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_class.dat | 2 +-
src/include/catalog/pg_proc.dat | 10 +
src/include/catalog/pg_type.dat | 5 +
src/include/catalog/toasting.h | 1 +
src/include/commands/defrem.h | 14 +
src/include/nodes/nodes.h | 1 +
src/include/postgres.h | 26 +-
src/include/utils/syscache.h | 1 +
src/tools/pgindent/typedefs.list | 3 +
47 files changed, 2118 insertions(+), 192 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ca00737c53 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- table tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 32c0ebb93a..77c742491f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -77,13 +77,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +112,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 3b0b138f24..9138373a42 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -968,11 +968,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..a09dc787ed
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 3c8a5da0bc..1ecc16888e 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2146,8 +2146,9 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options, NULL);
else
return tup;
}
@@ -3568,7 +3569,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index bce4274362..2a1a70fa10 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -665,7 +665,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
options |= HEAP_INSERT_NO_LOGICAL;
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
- options);
+ options, NULL);
}
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index cd921a4600..6082ee9a38 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -52,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -82,7 +122,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -420,7 +461,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -510,6 +551,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -521,6 +654,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -531,7 +666,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -666,27 +801,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -738,12 +883,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -751,7 +898,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -767,10 +916,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -911,7 +1061,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1339,6 +1490,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1354,54 +1517,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1496,19 +1652,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1516,7 +1673,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1525,7 +1685,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1885,7 +2045,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2072,7 +2232,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2265,15 +2425,26 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr)) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr)) < 0)
+ elog(ERROR, "compressed data is corrupted");
+ }
return result;
}
@@ -2375,3 +2546,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate system cache so we need to call it before
+ * we're putting something to our cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreate(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index 450a7dce1f..6bc0a56e2b 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -17,6 +17,7 @@
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
+#include "commands/defrem.h"
#include "utils/builtins.h"
#include "utils/syscache.h"
@@ -55,52 +56,12 @@ GetIndexAmRoutine(Oid amhandler)
IndexAmRoutine *
GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
{
- HeapTuple tuple;
- Form_pg_am amform;
regproc amhandler;
/* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
- if (!HeapTupleIsValid(tuple))
- {
- if (noerror)
- return NULL;
- elog(ERROR, "cache lookup failed for access method %u",
- amoid);
- }
- amform = (Form_pg_am) GETSTRUCT(tuple);
-
- /* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
- }
-
- amhandler = amform->amhandler;
-
- /* Complain if handler OID is invalid */
- if (!RegProcedureIsValid(amhandler))
- {
- if (noerror)
- {
- ReleaseSysCache(tuple);
- return NULL;
- }
- ereport(ERROR,
- (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
- NameStr(amform->amname))));
- }
-
- ReleaseSysCache(tuple);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_INDEX, noerror);
+ if (noerror && !RegProcedureIsValid(amhandler))
+ return NULL;
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index d8776e192e..4116b7ca1b 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -759,6 +759,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index f186198fc6..31aed9bb9a 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h \
+ pg_statistic_ext.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 2048d71535..8f4271a8f8 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -25,6 +25,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -181,7 +182,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1475,6 +1477,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2753,6 +2759,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index c7b5ff62f9..d0b5bd24c8 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -691,6 +691,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1653,6 +1654,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* We don't want to keep stats for it anymore */
attStruct->attstattarget = 0;
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index c339a2bb77..1913ff58dc 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -397,6 +397,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -483,6 +484,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8b51ec7f39..5881298a65 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -26,6 +26,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -525,6 +526,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3596,6 +3609,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4125,6 +4169,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4671,6 +4719,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d64628566d..ce7ef52e7a 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 9229fe1a45..ff4ef9b55b 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -632,6 +632,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 24ca18018e..750a5df95b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -179,6 +179,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -189,6 +253,16 @@ get_index_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_INDEX, missing_ok);
}
+/*
+ * get_index_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an index AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -231,6 +305,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -268,6 +344,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..7841c7700a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,649 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
+#include "catalog/pg_type.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ tup_amoid = get_am_oid(
+ NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq,
+ &arrayeq_info,
+ InvalidOid,
+ acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ /* no rewrite by default */
+ *need_rewrite = false;
+
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ {
+ Assert(!IsBinaryUpgrade);
+ *need_rewrite = true;
+ }
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ if (IsBinaryUpgrade && !OidIsValid(acoid))
+ elog(ERROR, "could not restore attribute compression data");
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is database initialization */
+ heap_close(rel, RowExclusiveLock);
+ return DefaultCompressionOid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(((Form_pg_attr_compression) GETSTRUCT(tup))->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression except current attribute compression
+ * and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+
+ if (IsBinaryUpgrade)
+ goto builtin_removal;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /* Remove attribute compression tuples and collect removed Oids to list */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index adb77d8f69..eff30c937c 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1216,6 +1216,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index 413ce3fcb6..7701dd8b80 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -179,7 +135,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 515c29072c..90361f2a59 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -34,6 +35,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -87,6 +90,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -367,6 +371,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(Relation rel, bool recurse, bool recursing);
@@ -472,6 +478,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void copy_relation_data(SMgrRelation rel, SMgrRelation dst,
ForkNumber forkNum, char relpersistence);
@@ -526,6 +534,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -737,6 +746,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -783,6 +794,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
+
+ if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -863,6 +881,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/* Process and store partition bound, if any. */
if (stmt->partbound)
{
@@ -2227,6 +2262,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2253,6 +2301,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2462,6 +2511,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3546,6 +3602,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4008,6 +4065,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4367,6 +4425,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4432,8 +4495,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -5603,6 +5667,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5765,6 +5839,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5887,6 +5962,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -5909,6 +6008,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*
@@ -6771,6 +6970,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -10088,6 +10291,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -10188,7 +10397,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!(foundDep->refclassid == TypeRelationId &&
foundDep->refobjid == attTup->atttypid) &&
!(foundDep->refclassid == CollationRelationId &&
- foundDep->refobjid == attTup->attcollation))
+ foundDep->refobjid == attTup->attcollation) &&
+ foundDep->refclassid != AttrCompressionRelationId)
elog(ERROR, "found unexpected dependency for column");
CatalogTupleDelete(depRel, &depTup->t_self);
@@ -10278,6 +10488,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ /* TODO: call the rewrite function here */
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ /* TODO: call the rewrite function here */;
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -10286,6 +10524,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -13318,6 +13561,93 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ /* TODO: set up rewrite rules here */;
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ /*
+ * Normally cleanup is done in rewrite but in binary upgrade we should do
+ * it explicitly.
+ */
+ if (IsBinaryUpgrade)
+ CleanupAttributeCompression(RelationGetRelid(rel),
+ attnum, preserved_amoids);
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index a8a735c247..a5ab5967e6 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2872,6 +2872,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2890,6 +2891,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5570,6 +5583,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3cab90e9f8..33d109db5d 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2552,6 +2552,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2570,6 +2571,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3634,6 +3645,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 8ed30c011a..3caae15f7c 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3688,6 +3688,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3695,6 +3697,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 69179a07c3..da6b95aa46 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2776,6 +2776,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2792,6 +2793,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4118,6 +4129,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a37d1f18be..90b407b0f8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1037,6 +1037,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 2cfdf1c9ac..f3f248b7eb 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3122,7 +3122,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 5c886cfe96..db18724625 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -419,3 +419,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index ac98c19155..01b2459719 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -187,6 +188,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9e48f0d49f
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,80 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (InvalidOid)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 7ade18ea46..820015affc 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -269,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index 4bfefffbf3..fad14bad85 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,9 +13,11 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +103,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +120,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +137,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +155,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +231,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +257,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index b235a23f5d..e4a5957ab4 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index f253613650..84e5dc81a5 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -90,6 +90,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 4003, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 4003
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 4004, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 4004
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..30faae0de4
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,23 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..90a8244a9b
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,4001,AttrCompressionRelationId)
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a6ec122389..ed75a09131 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -184,10 +187,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index c89710bc60..9bcf28676d 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -34,7 +34,7 @@
relname => 'pg_attribute', reltype => 'pg_attribute', relam => 'heap',
relfilenode => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '24', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
relhasrules => 'f', relhastriggers => 'f', relhassubclass => 'f',
relrowsecurity => 'f', relforcerowsecurity => 'f', relispopulated => 't',
relreplident => 'n', relispartition => 'f', relfrozenxid => '3',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c4b012cf4c..4cfe44a8da 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -6836,6 +6836,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '4008', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7012,6 +7015,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '4006', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '4007', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 2495ed6b17..703e02e8dc 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -567,6 +567,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '4005',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 5ee628c837..ac420cd3d6 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -74,6 +74,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 4187, 4188);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e592a914a4..62ccd203a8 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -156,8 +157,21 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 2451d650ab..5f71e8531d 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -506,6 +506,7 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
+ T_CompressionAmRoutine, /* in access/cmapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 057a3413ac..a8e818347b 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 95ee48954e..812e4fd43a 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b821df9e71..1beb3261cf 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -351,6 +351,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -376,6 +377,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
--
2.21.0
0003-Add-rewrite-rules-and-tupdesc-flags-v22.patchtext/x-patchDownload
From 9a0801b91fedbdd391c5cbed73d86adc2398e119 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:34:39 +0300
Subject: [PATCH 3/8] Add rewrite rules and tupdesc flags
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/heaptuple.c | 27 +++++++++-
src/backend/access/common/indextuple.c | 2 +
src/backend/access/common/tupdesc.c | 20 ++++++++
src/backend/access/heap/heapam.c | 19 ++++---
src/backend/access/heap/tuptoaster.c | 2 +
src/backend/commands/copy.c | 2 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 69 ++++++++++++++++++++++++--
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/cache/relcache.c | 4 ++
src/include/access/heapam.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/htup_details.h | 9 +++-
src/include/access/tupdesc.h | 7 +++
src/include/access/tuptoaster.h | 1 -
src/include/commands/event_trigger.h | 1 +
18 files changed, 157 insertions(+), 18 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a5bc6f5749..df391bcc78 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 783b04a3cb..b05dee6195 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -177,6 +177,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -207,6 +208,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -245,6 +249,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -268,6 +281,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -304,7 +320,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -328,6 +344,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -338,6 +355,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -752,6 +770,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -864,6 +883,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -882,6 +902,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -934,6 +955,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -944,6 +966,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1093,6 +1116,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1415,6 +1439,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 77c742491f..0c73bac69b 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -162,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 0158950a43..cb344b55aa 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts)
desc->constr = NULL;
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -136,6 +147,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -215,6 +227,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -300,6 +313,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasdef = false;
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -414,6 +428,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -464,6 +480,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -549,6 +567,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -654,6 +673,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1ecc16888e..34dd671430 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -70,7 +70,8 @@
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options,
+ BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -1846,13 +1847,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -1942,7 +1944,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2109,7 +2111,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2147,8 +2149,11 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
return tup;
}
else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options, NULL);
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2190,7 +2195,7 @@ heap_multi_insert(Relation relation, HeapTuple *tuples, int ntuples,
heaptuples = palloc(ntuples * sizeof(HeapTuple));
for (i = 0; i < ntuples; i++)
heaptuples[i] = heap_prepare_insert(relation, tuples[i],
- xid, cid, options);
+ xid, cid, options, bistate);
/*
* We're about to do the actual inserts -- but check for conflict first,
@@ -3467,6 +3472,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 6082ee9a38..176416a943 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -1190,6 +1190,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1411,6 +1412,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index 218a6e01cb..acf00191b8 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2662,7 +2662,7 @@ CopyFrom(CopyState cstate)
values = (Datum *) palloc(tupDesc->natts * sizeof(Datum));
nulls = (bool *) palloc(tupDesc->natts * sizeof(bool));
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
econtext = GetPerTupleExprContext(estate);
/* Set up callback to identify error line number */
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 36e3d44aad..8a52a53472 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -559,7 +559,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->hi_options = HEAP_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : HEAP_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 5a47be4b33..982a287e8c 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -464,7 +464,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->hi_options = HEAP_INSERT_SKIP_FSM | HEAP_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->hi_options |= HEAP_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 90361f2a59..185c9650c7 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -165,6 +165,9 @@ typedef struct AlteredTableInfo
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
bool verify_new_notnull; /* T if we should recheck NOT NULL */
+ bool new_notnull; /* T if we added new NOT NULL constraints */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
bool chgPersistence; /* T if SET LOGGED/UNLOGGED is used */
@@ -4727,7 +4730,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
hi_options = HEAP_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -5016,6 +5019,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
table_close(oldrel, NoLock);
if (newrel)
{
@@ -9505,6 +9526,45 @@ createForeignKeyTriggers(Relation rel, Oid refRelOid, Constraint *fkconstraint,
CommandCounterIncrement();
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -10496,7 +10556,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
{
/* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- /* TODO: call the rewrite function here */
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
}
else if (OidIsValid(attTup->attcompression))
{
@@ -10504,7 +10564,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
if (!IsBuiltinCompression(attTup->attcompression))
- /* TODO: call the rewrite function here */;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
}
@@ -13620,7 +13680,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
* toast_insert_or_update
*/
if (need_rewrite)
- /* TODO: set up rewrite rules here */;
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
atttableform->attcompression = acoid;
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 9971abd71f..71fffc53b8 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -808,6 +808,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index eba77491fd..6fa67242ef 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -569,6 +569,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 1b6607fe90..aa54b52476 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -25,6 +25,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -162,7 +163,7 @@ extern void heap_get_latest_tid(Relation relation, Snapshot snapshot,
ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index cec087cb1a..84c736cea2 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -31,6 +31,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 6d51f9062b..1ab905a131 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -274,7 +274,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -673,6 +675,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -779,7 +784,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index 66d1b2fc40..c878a0636a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -44,6 +46,10 @@ typedef struct TupleConstr
bool has_not_null;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -80,6 +86,7 @@ typedef struct TupleDescData
int natts; /* number of attributes in the tuple */
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index fad14bad85..aaf3b155dc 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -13,7 +13,6 @@
#ifndef TUPTOASTER_H
#define TUPTOASTER_H
-#include "access/cmapi.h"
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 0b8c7cca21..9252ad3d3a 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -31,6 +31,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
--
2.21.0
0004-Add-pglz-compression-method-v22.patchtext/x-patchDownload
From f985335dc149a805646cfc17ca0ec2888cb4eb51 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:48:28 +0300
Subject: [PATCH 4/8] Add pglz compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_pglz.c | 166 ++++++++++++++++++++
src/include/access/cmapi.h | 2 +-
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 2 +-
src/include/catalog/pg_proc.dat | 6 +
6 files changed, 178 insertions(+), 3 deletions(-)
create mode 100644 src/backend/access/compression/cm_pglz.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index a09dc787ed..14286920d3 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cmapi.o
+OBJS = cm_pglz.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..b693cd09f2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 resultlen;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ resultlen = VARRAWSIZE_4B_C(value) + VARHDRSZ;
+ result = (struct varlena *) palloc(resultlen);
+
+ SET_VARSIZE(result, resultlen);
+ if (pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value)) < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index 9e48f0d49f..1be98a60a5 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -19,7 +19,7 @@
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (InvalidOid)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 393b41dd68..4bf1c49d11 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 30faae0de4..4e72bde16c 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -18,6 +18,6 @@
[
-
+{ acoid => '4002', acname => 'pglz' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4cfe44a8da..5ff8e886bd 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -909,6 +909,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4009', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
--
2.21.0
0005-Add-zlib-compression-method-v22.patchtext/x-patchDownload
From 592ba861a18bcd936534f5d014d8debcc09abee6 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:51:07 +0300
Subject: [PATCH 5/8] Add zlib compression method
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/Makefile | 2 +-
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 252 ++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_attr_compression.dat | 1 +
src/include/catalog/pg_proc.dat | 4 +
6 files changed, 262 insertions(+), 2 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 478a96db9b..bd5009e190 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..0dcb56ddf3
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,252 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") != 0 &&
+ strcmp(defGetString(def), "best_compression") != 0 &&
+ strcmp(defGetString(def), "default") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"", defGetString(def))));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ if (strcmp(defGetString(def), "best_speed") == 0)
+ state->level = Z_BEST_SPEED;
+ else if (strcmp(defGetString(def), "best_compression") == 0)
+ state->level = Z_BEST_COMPRESSION;
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+#endif
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 4bf1c49d11..16a098d633 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -36,5 +36,8 @@
{ oid => '4002', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4011', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
index 4e72bde16c..a7216fe963 100644
--- a/src/include/catalog/pg_attr_compression.dat
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -19,5 +19,6 @@
[
{ acoid => '4002', acname => 'pglz' },
+{ acoid => '4011', acname => 'zlib' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 5ff8e886bd..2dce2c087d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -914,6 +914,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4010', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
--
2.21.0
0006-Add-psql-pg_dump-and-pg_upgrade-support-v22.patchtext/x-patchDownload
From b93d6f436fab78905251f3dea592b101951b6154 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:57:13 +0300
Subject: [PATCH 6/8] Add psql, pg_dump and pg_upgrade support
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
src/backend/commands/compressioncmds.c | 80 ++++++---
src/backend/commands/tablecmds.c | 14 +-
src/backend/utils/adt/pg_upgrade_support.c | 10 ++
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_dump.c | 195 ++++++++++++++++++++-
src/bin/pg_dump/pg_dump.h | 17 ++
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/pg_dump/t/002_pg_dump.pl | 95 ++++++++++
src/bin/psql/describe.c | 42 +++++
src/bin/psql/tab-complete.c | 5 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 +
13 files changed, 432 insertions(+), 42 deletions(-)
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 7841c7700a..89ffa227b0 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -36,6 +36,9 @@
#include "utils/syscache.h"
#include "utils/snapmgr.h"
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
/*
* When conditions of compression satisfies one if builtin attribute
* compresssion tuples the compressed attribute will be linked to
@@ -129,11 +132,12 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
tup_amoid;
Datum values[Natts_pg_attr_compression];
bool nulls[Natts_pg_attr_compression];
+ char *amname;
heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
- tup_amoid = get_am_oid(
- NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1])), false);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
if (previous_amoids)
*previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
@@ -150,17 +154,15 @@ lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
if (DatumGetPointer(acoptions) == NULL)
result = acoid;
}
- else
+ else if (DatumGetPointer(acoptions) != NULL)
{
bool equal;
/* check if arrays for WITH options are equal */
equal = DatumGetBool(CallerFInfoFunctionCall2(
- array_eq,
- &arrayeq_info,
- InvalidOid,
- acoptions,
- values[Anum_pg_attr_compression_acoptions - 1]));
+ array_eq, &arrayeq_info, InvalidOid, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
if (equal)
result = acoid;
}
@@ -227,6 +229,16 @@ CreateAttributeCompression(Form_pg_attribute att,
/* Try to find builtin compression first */
acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
/*
* attrelid will be invalid on CREATE TABLE, no need for table rewrite
* check.
@@ -252,16 +264,10 @@ CreateAttributeCompression(Form_pg_attribute att,
*/
if (need_rewrite != NULL)
{
- /* no rewrite by default */
- *need_rewrite = false;
-
Assert(preserved_amoids != NULL);
if (compression->preserve == NIL)
- {
- Assert(!IsBinaryUpgrade);
*need_rewrite = true;
- }
else
{
ListCell *cell;
@@ -294,7 +300,7 @@ CreateAttributeCompression(Form_pg_attribute att,
* In binary upgrade list will not be free since it contains
* Oid of builtin compression access method.
*/
- if (!IsBinaryUpgrade && list_length(previous_amoids) != 0)
+ if (list_length(previous_amoids) != 0)
*need_rewrite = true;
}
}
@@ -303,9 +309,6 @@ CreateAttributeCompression(Form_pg_attribute att,
list_free(previous_amoids);
}
- if (IsBinaryUpgrade && !OidIsValid(acoid))
- elog(ERROR, "could not restore attribute compression data");
-
/* Return Oid if we already found identical compression on this column */
if (OidIsValid(acoid))
{
@@ -315,6 +318,7 @@ CreateAttributeCompression(Form_pg_attribute att,
return acoid;
}
+add_tuple:
/* Initialize buffers for new tuple values */
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
@@ -323,13 +327,27 @@ CreateAttributeCompression(Form_pg_attribute att,
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
- acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
- Anum_pg_attr_compression_acoid);
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
if (acoid < FirstNormalObjectId)
{
- /* this is database initialization */
+ /* this is built-in attribute compression */
heap_close(rel, RowExclusiveLock);
- return DefaultCompressionOid;
+ return acoid;
}
/* we need routine only to call cmcheck function */
@@ -390,8 +408,8 @@ RemoveAttributeCompression(Oid acoid)
/*
* CleanupAttributeCompression
*
- * Remove entries in pg_attr_compression except current attribute compression
- * and related with specified list of access methods.
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
*/
void
CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
@@ -419,9 +437,7 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
ReleaseSysCache(attrtuple);
Assert(relid > 0 && attnum > 0);
-
- if (IsBinaryUpgrade)
- goto builtin_removal;
+ Assert(!IsBinaryUpgrade);
rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
@@ -438,7 +454,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
true, NULL, 2, key);
- /* Remove attribute compression tuples and collect removed Oids to list */
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
while (HeapTupleIsValid(tuple = systable_getnext(scan)))
{
Form_pg_attr_compression acform;
@@ -460,7 +479,10 @@ CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
systable_endscan(scan);
heap_close(rel, RowExclusiveLock);
- /* Now remove dependencies */
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
rel = heap_open(DependRelationId, RowExclusiveLock);
foreach(lc, removed)
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 185c9650c7..b3ff97a5cb 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -798,10 +798,10 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->identity)
attr->attidentity = colDef->identity;
- if (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE)
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression = CreateAttributeCompression(attr,
- colDef->compression,
- NULL, NULL);
+ colDef->compression, NULL, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -13696,14 +13696,6 @@ ATExecSetCompression(AlteredTableInfo *tab,
/* make changes visible */
CommandCounterIncrement();
- /*
- * Normally cleanup is done in rewrite but in binary upgrade we should do
- * it explicitly.
- */
- if (IsBinaryUpgrade)
- CleanupAttributeCompression(RelationGetRelid(rel),
- attnum, preserved_amoids);
-
ObjectAddressSet(address, AttrCompressionRelationId, acoid);
return address;
}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 99db5ba389..0e81e70e09 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -116,6 +116,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 7ab27391fb..2f97e244c7 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -150,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 4c98ae4d7f..2e32d4e126 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -41,11 +41,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -389,6 +391,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -885,6 +888,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8227,9 +8232,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
+ bool createWithCompression;
for (i = 0; i < numTables; i++)
{
@@ -8272,6 +8280,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"a.attislocal,\n"
"pg_catalog.format_type(t.oid, a.atttypmod) AS atttypname,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBuffer(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8324,7 +8349,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8352,6 +8383,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8369,9 +8402,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (j = 0; j < ntups; j++)
@@ -8397,6 +8433,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -8614,6 +8652,104 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ if (g_verbose)
+ write_msg(NULL, "finding compression info for table \"%s.%s\"\n",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -12729,6 +12865,8 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
break;
case AMTYPE_TABLE:
appendPQExpBuffer(q, "TYPE TABLE ");
+ case AMTYPE_COMPRESSION:
+ appendPQExpBuffer(q, "TYPE COMPRESSION ");
break;
default:
write_msg(NULL, "WARNING: invalid type \"%c\" of access method \"%s\"\n",
@@ -15650,6 +15788,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
(!tbinfo->inhNotNull[j] ||
dopt->binary_upgrade));
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ bool has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/*
* Skip column if fully defined by reloftype or the
* partition parent.
@@ -15708,6 +15854,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
fmtQualifiedDumpable(coll));
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (has_default)
appendPQExpBuffer(q, " DEFAULT %s",
tbinfo->attrdefs[j]->adef_expr);
@@ -16123,6 +16288,34 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "OPTIONS (\n %s\n);\n",
tbinfo->attfdwoptions[j]);
}
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 2e1b90acd0..810d001e84 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,10 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmoptions; /* per-attribute current compression options */
+ char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
+
/*
* Stuff computed only for dumpable tables.
*/
@@ -347,6 +351,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index a86965e670..f09925a85d 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -76,6 +76,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -143,6 +144,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -432,6 +434,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -656,6 +660,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 428e040acb..03f84933c6 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -375,6 +377,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index de6895122e..87a397303f 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -651,6 +651,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1367,6 +1404,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2414,6 +2462,53 @@ my %tests = (
unlike => { exclude_dump_test_schema => 1, },
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 779e48437c..428f839d03 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1467,6 +1467,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1859,6 +1860,24 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -1985,6 +2004,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2056,6 +2077,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 10ae21cc61..19ef736b98 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1919,11 +1919,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index 2927b7a4d3..00f91e4b90 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -25,6 +25,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 2dce2c087d..8ca3f53b96 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -9981,6 +9981,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4012', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# conversion functions
{ oid => '4300',
--
2.21.0
0007-Add-tests-for-compression-methods-v22.patchtext/x-patchDownload
From bb1270061d5ef0bbf344452790109552c4c4162b Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 15:59:52 +0300
Subject: [PATCH 7/8] Add tests for compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
contrib/test_decoding/expected/ddl.out | 50 +--
src/backend/access/compression/cm_zlib.c | 2 +-
src/bin/pg_dump/pg_dump.c | 1 +
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 405 +++++++++++++++++
src/test/regress/expected/create_cm_1.out | 409 ++++++++++++++++++
src/test/regress/expected/create_table.out | 132 +++---
.../regress/expected/create_table_like.out | 68 +--
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++------
src/test/regress/expected/inherit.out | 138 +++---
src/test/regress/expected/insert.out | 118 ++---
src/test/regress/expected/opr_sanity.out | 12 +
src/test/regress/expected/psql.out | 40 +-
src/test/regress/expected/publication.out | 40 +-
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 20 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/expected/update.out | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 203 +++++++++
src/test/regress/sql/opr_sanity.sql | 10 +
24 files changed, 1513 insertions(+), 468 deletions(-)
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/expected/create_cm_1.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 2bd28e6d15..01947e0696 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index 0dcb56ddf3..dc8666f309 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -182,7 +182,6 @@ zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
}
pfree(tmp);
-#endif
return NULL;
}
@@ -231,6 +230,7 @@ zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
pfree(zp);
return result;
}
+#endif
Datum
zlibhandler(PG_FUNCTION_ARGS)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2e32d4e126..dc857b2cc6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -12865,6 +12865,7 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
break;
case AMTYPE_TABLE:
appendPQExpBuffer(q, "TYPE TABLE ");
+ break;
case AMTYPE_COMPRESSION:
appendPQExpBuffer(q, "TYPE COMPRESSION ");
break;
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index 75d4119eaa..54fe30b3c0 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..de14d87885
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,405 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: unexpected value for zlib compression level: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..755d40caef
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,409 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz2, pglz1
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 4001 | 0 | n
+ 4001 | 0 | 1259 | 1 | i
+ 4001 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz2, pglz1
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index d51e547278..bd84883175 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -445,11 +445,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -458,11 +458,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -479,10 +479,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -832,21 +832,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -854,11 +854,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -887,46 +887,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -958,11 +958,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -971,10 +971,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -984,10 +984,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index b582211270..d830ce830c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -158,32 +158,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -197,12 +197,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -212,12 +212,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -231,11 +231,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 4ff1b4af41..5255951060 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index 4d82d3a7e8..8c8a03cfc2 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1332,12 +1332,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1353,12 +1353,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1377,12 +1377,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1420,12 +1420,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1445,17 +1445,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1477,17 +1477,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1519,17 +1519,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1557,12 +1557,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1594,12 +1594,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1638,12 +1638,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1669,12 +1669,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1696,12 +1696,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1727,12 +1727,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1791,12 +1791,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1836,12 +1836,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1863,12 +1863,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1891,12 +1891,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1921,12 +1921,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1949,12 +1949,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 565d947b6d..285e36fb21 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1033,13 +1033,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1052,14 +1052,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1069,14 +1069,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1116,33 +1116,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1153,27 +1153,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1183,37 +1183,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 1cf6531c01..37f96ea7a2 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -799,11 +799,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -815,74 +815,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 49a0acc0ee..7d2c2618c3 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1826,6 +1826,18 @@ WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND
-----+--------+-----+---------
(0 rows)
+-- Check for compression amhandler functions with the wrong signature
+SELECT p1.oid, p1.amname, p2.oid, p2.proname
+FROM pg_am AS p1, pg_proc AS p2
+WHERE p2.oid = p1.amhandler AND p1.amtype = 'c' AND
+ (p2.prorettype != 'compression_am_handler'::regtype
+ OR p2.proretset
+ OR p2.pronargs != 1
+ OR p2.proargtypes[0] != 'internal'::regtype);
+ oid | amname | oid | proname
+-----+--------+-----+---------
+(0 rows)
+
-- **************** pg_amop ****************
-- Look for illegal values in pg_amop fields
SELECT p1.amopfamily, p1.amopstrategy
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index aa101de906..a2d9f145ea 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2779,34 +2779,34 @@ CREATE ACCESS METHOD heap_psql TYPE TABLE HANDLER heap_tableam_handler;
CREATE TABLE tbl_heap_psql(f1 int, f2 char(100)) using heap_psql;
CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
\d+ tbl_heap_psql
- Table "public.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "public.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "public.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "public.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
\set HIDE_TABLEAM on
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index afbbdd543d..4f386a84e7 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 175ecd2879..7cf1927971 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 2e170497c9..df49e98251 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index f104dc4a62..e5ad0e0051 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2815,11 +2815,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2835,11 +2835,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 29682e3866..b565c1a74d 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index 2083345c8e..b245059e37 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -690,14 +690,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 030a71f3a4..3a3464b8a0 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -41,6 +41,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 3e53d7d8f3..9cb37b9b89 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -79,6 +79,7 @@ test: updatable_views
test: rolenames
test: roleattributes
test: create_am
+test: create_cm
test: hash_func
test: sanity_check
test: errors
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..d7e6c5eba2
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,203 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 4001 OR classid = 4001) AND
+ (objid = 'cmtest'::regclass OR refobjid = 'cmtest'::regclass)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::regclass OR acrelid = 'cmtest'::regclass;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_compression');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'best_speed');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level 'default');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index 1227ef79f0..fe65d44041 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1221,6 +1221,16 @@ WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
+-- Check for compression amhandler functions with the wrong signature
+
+SELECT p1.oid, p1.amname, p2.oid, p2.proname
+FROM pg_am AS p1, pg_proc AS p2
+WHERE p2.oid = p1.amhandler AND p1.amtype = 'c' AND
+ (p2.prorettype != 'compression_am_handler'::regtype
+ OR p2.proretset
+ OR p2.pronargs != 1
+ OR p2.proargtypes[0] != 'internal'::regtype);
+
-- **************** pg_amop ****************
-- Look for illegal values in pg_amop fields
--
2.21.0
0008-Add-documentation-for-custom-compression-methods-v22.patchtext/x-patchDownload
From e51539968dec49235cf8579afb142a6fb3529967 Mon Sep 17 00:00:00 2001
From: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
Date: Mon, 18 Jun 2018 16:00:43 +0300
Subject: [PATCH 8/8] Add documentation for custom compression methods
Signed-off-by: Ildus Kurbangaliev <i.kurbangaliev@gmail.com>
---
doc/src/sgml/catalogs.sgml | 19 ++-
doc/src/sgml/compression-am.sgml | 178 +++++++++++++++++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +++
doc/src/sgml/ref/create_access_method.sgml | 5 +-
doc/src/sgml/ref/create_table.sgml | 13 ++
doc/src/sgml/storage.sgml | 6 +-
9 files changed, 236 insertions(+), 7 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 0fd792ff1a..c8551c2fc3 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -57,7 +57,7 @@
<row>
<entry><link linkend="catalog-pg-am"><structname>pg_am</structname></link></entry>
- <entry>index access methods</entry>
+ <entry>access methods</entry>
</row>
<row>
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -587,8 +592,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only indexes have access methods. The requirements for index
- access methods are discussed in detail in <xref linkend="indexam"/>.
+ Currently, compression and index access methods are supported.
+ The requirements for index access methods are discussed in detail
+ in <xref linkend="indexam"/>, for compression access methods
+ could be found in <xref linkend="compression-am"/>.
</para>
<table>
@@ -892,6 +899,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index a03ea1427b..8f482a60dd 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -90,6 +90,7 @@
<!ENTITY brin SYSTEM "brin.sgml">
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 05102724ea..54110050d1 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -47,7 +47,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the access
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 96d196d229..17f2cc98b3 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -251,6 +251,7 @@
&custom-scan;
&geqo;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index e360728c02..09ad7c8aa2 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -359,6 +360,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 851c5e63be..a35005aca3 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>INDEX</literal> is supported at present.
+ <literal>INDEX</literal> and <literal>COMPRESSION</literal> types are supported at present.
</para>
</listitem>
</varlistentry>
@@ -79,6 +79,9 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
be <type>index_am_handler</type>. The C-level API that the handler
function must implement varies depending on the type of access method.
The index access method API is described in <xref linkend="indexam"/>.
+ For <literal>COMPRESSION</literal> access methods, the type must be
+ <type>compression_am_handler</type>. The compression access method API
+ is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index e94fe2c3b6..2d192148de 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -65,6 +65,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -920,6 +921,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index cbdad0c3fb..89a5889d0f 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -385,10 +385,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
--
2.21.0
Re: Re: Re: [HACKERS] Custom compression methods
On Fri, Mar 15, 2019 at 6:07 PM David Steele <david@pgmasters.net> wrote:
On 3/7/19 11:50 AM, Alexander Korotkov wrote:
On Thu, Mar 7, 2019 at 10:43 AM David Steele <david@pgmasters.net
<mailto:david@pgmasters.net>> wrote:On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I can
see
since last January 2018.
As Andres noted in [1], I think that we need to decide if this is a
feature that we want rather than just continuing to push it from CF
to CF.Yes. I took a look at code of this patch. I think it's in pretty good
shape. But high level review/discussion is required.OK, but I think this patch can only be pushed one more time, maximum,
before it should be rejected.
As a note, we believe at Adjust that this would be very helpful for some of
our use cases and some other general use cases. I think as a feature,
custom compression methods are a good thing but we are not the only ones
with interests here and would be interested in pushing this forward if
possible or finding ways to contribute to better approaches in this
particular field.
Regards,
--
-David
david@pgmasters.net
--
Best Regards,
Chris Travers
Head of Database
Tel: +49 162 9037 210 | Skype: einhverfr | www.adjust.com
Saarbrücker Straße 37a, 10405 Berlin
Re: [HACKERS] Custom compression methods
Hi Ildus,
On Fri, Mar 15, 2019 at 12:52 PM Ildus Kurbangaliev <
i.kurbangaliev@gmail.com> wrote:
Hi,
in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.Attached latest set of patches.
As I understand, the only thing changed since my last review is an
additional
compression method for zlib.
The code looks good. I have one suggestion though. Currently you only
predefine
two compression levels: `best_speed` and `best_compression`. But zlib itself
allows a fine gradation between those two. It is possible to set level to
the
values from 0 to 9 (where zero means no compression at all which I guess
isn't
useful in our case). So I think we should allow user choose between either
textual representation (as you already did) or numeral. Another thing is
that one
can specify, for instance, `best_speed` level, but not `BEST_SPEED`, which
can
be a bit frustrating for user.
Regards,
Ildar Musin
Re: [HACKERS] Custom compression methods
On 3/15/19 12:52 PM, Ildus Kurbangaliev wrote:
On Fri, 15 Mar 2019 14:07:14 +0400
David Steele <david@pgmasters.net> wrote:On 3/7/19 11:50 AM, Alexander Korotkov wrote:
On Thu, Mar 7, 2019 at 10:43 AM David Steele <david@pgmasters.net
<mailto:david@pgmasters.net>> wrote:On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I
can see since last January 2018.As Andres noted in [1], I think that we need to decide if this
is a feature that we want rather than just continuing to push it
from CF to CF.Yes. I took a look at code of this patch. I think it's in pretty
good shape. But high level review/discussion is required.OK, but I think this patch can only be pushed one more time, maximum,
before it should be rejected.Regards,
Hi,
in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.
That may be one of the reasons, yes. But there are other reasons, which
I think may be playing a bigger role.
There's one practical issue with how the patch is structured - the docs
and tests are in separate patches towards the end of the patch series,
which makes it impossible to commit the preceding parts. This needs to
change. Otherwise the patch size kills the patch as a whole.
But there's a more important cost/benefit issue, I think. When I look at
patches as a committer, I naturally have to weight how much time I spend
on getting it in (and then dealing with fallout from bugs etc) vs. what
I get in return (measured in benefits for community, users). This patch
is pretty large and complex, so the "costs" are quite high, while the
benefits from the patch itself is the ability to pick between pg_lz and
zlib. Which is not great, and so people tend to pick other patches.
Now, I understand there's a lot of potential benefits further down the
line, like column-level compression (which I think is the main goal
here). But that's not included in the patch, so the gains are somewhat
far in the future.
But hey, I think there are committers working for postgrespro, who might
have the motivation to get this over the line. Of course, assuming that
there are no serious objections to having this functionality or how it's
implemented ... But I don't think that was the case.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Mar 18, 2019 at 11:09 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
On 3/15/19 12:52 PM, Ildus Kurbangaliev wrote:
On Fri, 15 Mar 2019 14:07:14 +0400
David Steele <david@pgmasters.net> wrote:On 3/7/19 11:50 AM, Alexander Korotkov wrote:
On Thu, Mar 7, 2019 at 10:43 AM David Steele <david@pgmasters.net
<mailto:david@pgmasters.net>> wrote:On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
there are another set of patches.
Only rebased to current master.Also I will change status on commitfest to 'Needs review'.
This patch has seen periodic rebases but no code review that I
can see since last January 2018.As Andres noted in [1], I think that we need to decide if this
is a feature that we want rather than just continuing to push it
from CF to CF.Yes. I took a look at code of this patch. I think it's in pretty
good shape. But high level review/discussion is required.OK, but I think this patch can only be pushed one more time, maximum,
before it should be rejected.Regards,
Hi,
in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.That may be one of the reasons, yes. But there are other reasons, which
I think may be playing a bigger role.There's one practical issue with how the patch is structured - the docs
and tests are in separate patches towards the end of the patch series,
which makes it impossible to commit the preceding parts. This needs to
change. Otherwise the patch size kills the patch as a whole.But there's a more important cost/benefit issue, I think. When I look at
patches as a committer, I naturally have to weight how much time I spend
on getting it in (and then dealing with fallout from bugs etc) vs. what
I get in return (measured in benefits for community, users). This patch
is pretty large and complex, so the "costs" are quite high, while the
benefits from the patch itself is the ability to pick between pg_lz and
zlib. Which is not great, and so people tend to pick other patches.Now, I understand there's a lot of potential benefits further down the
line, like column-level compression (which I think is the main goal
here). But that's not included in the patch, so the gains are somewhat
far in the future.
Not discussing whether any particular committer should pick this up but I
want to discuss an important use case we have at Adjust for this sort of
patch.
The PostgreSQL compression strategy is something we find inadequate for at
least one of our large deployments (a large debug log spanning 10PB+). Our
current solution is to set storage so that it does not compress and then
run on ZFS to get compression speedups on spinning disks.
But running PostgreSQL on ZFS has some annoying costs because we have
copy-on-write on copy-on-write, and when you add file fragmentation... I
would really like to be able to get away from having to do ZFS as an
underlying filesystem. While we have good write throughput, read
throughput is not as good as I would like.
An approach that would give us better row-level compression would allow us
to ditch the COW filesystem under PostgreSQL approach.
So I think the benefits are actually quite high particularly for those
dealing with volume/variety problems where things like JSONB might be a
go-to solution. Similarly I could totally see having systems which handle
large amounts of specialized text having extensions for dealing with these.
But hey, I think there are committers working for postgrespro, who might
have the motivation to get this over the line. Of course, assuming that
there are no serious objections to having this functionality or how it's
implemented ... But I don't think that was the case.
While I am not currently able to speak for questions of how it is
implemented, I can say with very little doubt that we would almost
certainly use this functionality if it were there and I could see plenty of
other cases where this would be a very appropriate direction for some other
projects as well.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Best Regards,
Chris Travers
Head of Database
Tel: +49 162 9037 210 | Skype: einhverfr | www.adjust.com
Saarbrücker Straße 37a, 10405 Berlin
Re: [HACKERS] Custom compression methods
On 3/19/19 10:59 AM, Chris Travers wrote:
On Mon, Mar 18, 2019 at 11:09 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:On 3/15/19 12:52 PM, Ildus Kurbangaliev wrote:
On Fri, 15 Mar 2019 14:07:14 +0400
David Steele <david@pgmasters.net <mailto:david@pgmasters.net>> wrote:On 3/7/19 11:50 AM, Alexander Korotkov wrote:
On Thu, Mar 7, 2019 at 10:43 AM David Steele
<david@pgmasters.net <mailto:david@pgmasters.net>
<mailto:david@pgmasters.net <mailto:david@pgmasters.net>>> wrote:
On 2/28/19 5:44 PM, Ildus Kurbangaliev wrote:
> there are another set of patches.
> Only rebased to current master.
>
> Also I will change status on commitfest to 'Needs review'.This patch has seen periodic rebases but no code review that I
can see since last January 2018.As Andres noted in [1], I think that we need to decide if this
is a feature that we want rather than just continuing to push it
from CF to CF.Yes. I took a look at code of this patch. I think it's in pretty
good shape. But high level review/discussion is required.OK, but I think this patch can only be pushed one more time,
maximum,
before it should be rejected.
Regards,
Hi,
in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.That may be one of the reasons, yes. But there are other reasons, which
I think may be playing a bigger role.There's one practical issue with how the patch is structured - the docs
and tests are in separate patches towards the end of the patch series,
which makes it impossible to commit the preceding parts. This needs to
change. Otherwise the patch size kills the patch as a whole.But there's a more important cost/benefit issue, I think. When I look at
patches as a committer, I naturally have to weight how much time I spend
on getting it in (and then dealing with fallout from bugs etc) vs. what
I get in return (measured in benefits for community, users). This patch
is pretty large and complex, so the "costs" are quite high, while the
benefits from the patch itself is the ability to pick between pg_lz and
zlib. Which is not great, and so people tend to pick other patches.Now, I understand there's a lot of potential benefits further down the
line, like column-level compression (which I think is the main goal
here). But that's not included in the patch, so the gains are somewhat
far in the future.Not discussing whether any particular committer should pick this up but
I want to discuss an important use case we have at Adjust for this sort
of patch.The PostgreSQL compression strategy is something we find inadequate for
at least one of our large deployments (a large debug log spanning
10PB+). Our current solution is to set storage so that it does not
compress and then run on ZFS to get compression speedups on spinning disks.But running PostgreSQL on ZFS has some annoying costs because we have
copy-on-write on copy-on-write, and when you add file fragmentation... I
would really like to be able to get away from having to do ZFS as an
underlying filesystem. While we have good write throughput, read
throughput is not as good as I would like.An approach that would give us better row-level compression would allow
us to ditch the COW filesystem under PostgreSQL approach.So I think the benefits are actually quite high particularly for those
dealing with volume/variety problems where things like JSONB might be a
go-to solution. Similarly I could totally see having systems which
handle large amounts of specialized text having extensions for dealing
with these.
Sure, I don't disagree - the proposed compression approach may be a big
win for some deployments further down the road, no doubt about it. But
as I said, it's unclear when we get there (or if the interesting stuff
will be in some sort of extension, which I don't oppose in principle).
But hey, I think there are committers working for postgrespro, who might
have the motivation to get this over the line. Of course, assuming that
there are no serious objections to having this functionality or how it's
implemented ... But I don't think that was the case.While I am not currently able to speak for questions of how it is
implemented, I can say with very little doubt that we would almost
certainly use this functionality if it were there and I could see plenty
of other cases where this would be a very appropriate direction for some
other projects as well.
Well, I guess the best thing you can do to move this patch forward is to
actually try that on your real-world use case, and report your results
and possibly do a review of the patch.
IIRC there was an extension [1]/messages/by-id/20171130182009.1b492eb2@wp.localdomain leveraging this custom compression
interface for better jsonb compression, so perhaps that would work for
you (not sure if it's up to date with the current patch, though).
[1]: /messages/by-id/20171130182009.1b492eb2@wp.localdomain
/messages/by-id/20171130182009.1b492eb2@wp.localdomain
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Tue, Mar 19, 2019 at 12:19 PM Tomas Vondra <tomas.vondra@2ndquadrant.com>
wrote:
On 3/19/19 10:59 AM, Chris Travers wrote:
Not discussing whether any particular committer should pick this up but
I want to discuss an important use case we have at Adjust for this sort
of patch.The PostgreSQL compression strategy is something we find inadequate for
at least one of our large deployments (a large debug log spanning
10PB+). Our current solution is to set storage so that it does not
compress and then run on ZFS to get compression speedups on spinningdisks.
But running PostgreSQL on ZFS has some annoying costs because we have
copy-on-write on copy-on-write, and when you add file fragmentation... I
would really like to be able to get away from having to do ZFS as an
underlying filesystem. While we have good write throughput, read
throughput is not as good as I would like.An approach that would give us better row-level compression would allow
us to ditch the COW filesystem under PostgreSQL approach.So I think the benefits are actually quite high particularly for those
dealing with volume/variety problems where things like JSONB might be a
go-to solution. Similarly I could totally see having systems which
handle large amounts of specialized text having extensions for dealing
with these.Sure, I don't disagree - the proposed compression approach may be a big
win for some deployments further down the road, no doubt about it. But
as I said, it's unclear when we get there (or if the interesting stuff
will be in some sort of extension, which I don't oppose in principle).
I would assume that if extensions are particularly stable and useful they
could be moved into core.
But I would also assume that at first, this area would be sufficiently
experimental that folks (like us) would write our own extensions for it.
But hey, I think there are committers working for postgrespro, who
might
have the motivation to get this over the line. Of course, assuming
that
there are no serious objections to having this functionality or how
it's
implemented ... But I don't think that was the case.
While I am not currently able to speak for questions of how it is
implemented, I can say with very little doubt that we would almost
certainly use this functionality if it were there and I could see plenty
of other cases where this would be a very appropriate direction for some
other projects as well.Well, I guess the best thing you can do to move this patch forward is to
actually try that on your real-world use case, and report your results
and possibly do a review of the patch.
Yeah, I expect to do this within the next month or two.
IIRC there was an extension [1] leveraging this custom compression
interface for better jsonb compression, so perhaps that would work for
you (not sure if it's up to date with the current patch, though).[1]
/messages/by-id/20171130182009.1b492eb2@wp.localdomain
Yeah I will be looking at a couple different approaches here and reporting
back. I don't expect it will be a full production workload but I do expect
to be able to report on benchmarks in both storage and performance.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
--
Best Regards,
Chris Travers
Head of Database
Tel: +49 162 9037 210 | Skype: einhverfr | www.adjust.com
Saarbrücker Straße 37a, 10405 Berlin
Re: [HACKERS] Custom compression methods
On 3/19/19 4:44 PM, Chris Travers wrote:
On Tue, Mar 19, 2019 at 12:19 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com <mailto:tomas.vondra@2ndquadrant.com>> wrote:On 3/19/19 10:59 AM, Chris Travers wrote:
Not discussing whether any particular committer should pick this
up but
I want to discuss an important use case we have at Adjust for this
sort
of patch.
The PostgreSQL compression strategy is something we find
inadequate for
at least one of our large deployments (a large debug log spanning
10PB+). Our current solution is to set storage so that it does not
compress and then run on ZFS to get compression speedups onspinning disks.
But running PostgreSQL on ZFS has some annoying costs because we have
copy-on-write on copy-on-write, and when you add filefragmentation... I
would really like to be able to get away from having to do ZFS as an
underlying filesystem. While we have good write throughput, read
throughput is not as good as I would like.An approach that would give us better row-level compression would
allow
us to ditch the COW filesystem under PostgreSQL approach.
So I think the benefits are actually quite high particularly for those
dealing with volume/variety problems where things like JSONB mightbe a
go-to solution. Similarly I could totally see having systems which
handle large amounts of specialized text having extensions for dealing
with these.Sure, I don't disagree - the proposed compression approach may be a big
win for some deployments further down the road, no doubt about it. But
as I said, it's unclear when we get there (or if the interesting stuff
will be in some sort of extension, which I don't oppose in principle).I would assume that if extensions are particularly stable and useful
they could be moved into core.But I would also assume that at first, this area would be sufficiently
experimental that folks (like us) would write our own extensions for it.
But hey, I think there are committers working for postgrespro,
who might
have the motivation to get this over the line. Of course,
assuming that
there are no serious objections to having this functionality
or how it's
implemented ... But I don't think that was the case.
While I am not currently able to speak for questions of how it is
implemented, I can say with very little doubt that we would almost
certainly use this functionality if it were there and I could seeplenty
of other cases where this would be a very appropriate direction
for some
other projects as well.
Well, I guess the best thing you can do to move this patch forward is to
actually try that on your real-world use case, and report your results
and possibly do a review of the patch.Yeah, I expect to do this within the next month or two.
IIRC there was an extension [1] leveraging this custom compression
interface for better jsonb compression, so perhaps that would work for
you (not sure if it's up to date with the current patch, though).[1]
/messages/by-id/20171130182009.1b492eb2@wp.localdomainYeah I will be looking at a couple different approaches here and
reporting back. I don't expect it will be a full production workload but
I do expect to be able to report on benchmarks in both storage and
performance.
FWIW I was a bit curious how would that jsonb compression affect the
data set I'm using for testing jsonpath patches, so I spent a bit of
time getting it to work with master. It attached patch gets it to
compile, but unfortunately then it fails like this:
ERROR: jsonbd: worker has detached
It seems there's some bug in how sh_mq is used, but I don't have time
investigate that further.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Attachments:
jsonbd-master-fix.patchtext/x-patch; name=jsonbd-master-fix.patchDownload
diff --git a/jsonbd.c b/jsonbd.c
index 511cfcf..9b86e1f 100644
--- a/jsonbd.c
+++ b/jsonbd.c
@@ -766,11 +766,6 @@ jsonbd_cminitstate(Oid acoid, List *options)
return NULL;
}
-static void
-jsonbd_cmdrop(Oid acoid)
-{
- /* TODO: if there is no compression options, remove the dictionary */
-}
static struct varlena *
jsonbd_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *data)
@@ -863,7 +858,6 @@ jsonbd_compression_handler(PG_FUNCTION_ARGS)
CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
routine->cmcheck = jsonbd_cmcheck;
- routine->cmdrop = jsonbd_cmdrop; /* no drop behavior */
routine->cminitstate = jsonbd_cminitstate;
routine->cmcompress = jsonbd_cmcompress;
routine->cmdecompress = jsonbd_cmdecompress;
diff --git a/jsonbd_utils.c b/jsonbd_utils.c
index 3ab9347..029e2e3 100644
--- a/jsonbd_utils.c
+++ b/jsonbd_utils.c
@@ -2,9 +2,11 @@
#include "jsonbd_utils.h"
#include "postgres.h"
+#include "access/genam.h"
#include "access/htup_details.h"
#include "access/xact.h"
#include "access/sysattr.h"
+#include "access/table.h"
#include "catalog/indexing.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
@@ -13,22 +15,20 @@
#include "utils/fmgroids.h"
#include "utils/rel.h"
-#if PG_VERSION_NUM == 110000
struct shm_mq_alt
{
slock_t mq_mutex;
- PGPROC *mq_receiver; /* we need this one */
- PGPROC *mq_sender; /* this one */
- uint64 mq_bytes_read;
- uint64 mq_bytes_written;
+ PGPROC *mq_receiver;
+ PGPROC *mq_sender;
+ pg_atomic_uint64 mq_bytes_read;
+ pg_atomic_uint64 mq_bytes_written;
Size mq_ring_size;
- bool mq_detached; /* and this one */
+ bool mq_detached;
+ uint8 mq_ring_offset;
+ char mq_ring[FLEXIBLE_ARRAY_MEMBER];
/* in postgres version there are more attributes, but we don't need them */
};
-#else
-#error "shm_mq struct in jsonbd is copied from PostgreSQL 11, please correct it according to your version"
-#endif
/**
* Get 32-bit Murmur3 hash. Ported from qLibc library.
@@ -153,7 +153,7 @@ get_jsonbd_schema(void)
return InvalidOid; /* exit if pg_pathman does not exist */
ScanKeyInit(&entry[0],
- ObjectIdAttributeNumber,
+ Anum_pg_extension_oid,
BTEqualStrategyNumber, F_OIDEQ,
ObjectIdGetDatum(ext_oid));
diff --git a/jsonbd_worker.c b/jsonbd_worker.c
index e9cb5b6..c59e102 100644
--- a/jsonbd_worker.c
+++ b/jsonbd_worker.c
@@ -13,11 +13,11 @@
#include "access/xact.h"
#include "catalog/indexing.h"
#include "catalog/namespace.h"
-#include "catalog/pg_compression_opt.h"
#include "catalog/pg_extension.h"
#include "commands/extension.h"
#include "commands/dbcommands.h"
#include "executor/spi.h"
+#include "access/tableam.h"
#include "port/atomics.h"
#include "storage/ipc.h"
#include "storage/latch.h"
@@ -32,7 +32,6 @@
#include "utils/rel.h"
#include "utils/resowner.h"
#include "utils/snapmgr.h"
-#include "utils/tqual.h"
#include "utils/syscache.h"
static bool xact_started = false;
@@ -125,7 +124,7 @@ init_worker(dsm_segment *seg)
worker_args = (jsonbd_worker_args *) dsm_segment_address(seg);
/* Connect to our database */
- BackgroundWorkerInitializeConnectionByOid(worker_args->dboid, InvalidOid);
+ BackgroundWorkerInitializeConnectionByOid(worker_args->dboid, InvalidOid, 0);
worker_state = shm_toc_lookup(toc, worker_args->worker_num, false);
worker_state->proc = MyProc;
@@ -205,8 +204,8 @@ jsonbd_get_key(Relation rel, Relation indrel, Oid cmoptoid, uint32 key_id)
IndexScanDesc scan;
ScanKeyData skey[2];
Datum key_datum;
- HeapTuple tup;
bool isNull;
+ TupleTableSlot *slot;
MemoryContext old_mcxt;
char *res;
@@ -224,19 +223,22 @@ jsonbd_get_key(Relation rel, Relation indrel, Oid cmoptoid, uint32 key_id)
scan = index_beginscan(rel, indrel, SnapshotAny, 2, 0);
index_rescan(scan, skey, 2, NULL, 0);
- tup = index_getnext(scan, ForwardScanDirection);
- if (tup == NULL)
+ slot = table_slot_create(rel, NULL);
+
+ if (!index_getnext_slot(scan, ForwardScanDirection, slot))
elog(ERROR, "key not found for cmopt=%d and id=%d", cmoptoid, key_id);
- key_datum = heap_getattr(tup, JSONBD_DICTIONARY_REL_ATT_KEY,
- RelationGetDescr(rel), &isNull);
+ key_datum = slot_getattr(slot, JSONBD_DICTIONARY_REL_ATT_KEY, &isNull);
Assert(!isNull);
old_mcxt = MemoryContextSwitchTo(worker_context);
res = TextDatumGetCString(key_datum);
MemoryContextSwitchTo(old_mcxt);
+ ExecDropSingleTupleTableSlot(slot);
+
index_endscan(scan);
+
return res;
}
@@ -308,7 +310,7 @@ jsonbd_get_key_id(Relation rel, Relation indrel, Oid cmoptoid, char *key)
{
IndexScanDesc scan;
ScanKeyData skey[2];
- HeapTuple tup;
+ TupleTableSlot *slot;
bool isNull;
uint32 result = 0;
@@ -326,16 +328,18 @@ jsonbd_get_key_id(Relation rel, Relation indrel, Oid cmoptoid, char *key)
scan = index_beginscan(rel, indrel, SnapshotAny, 2, 0);
index_rescan(scan, skey, 2, NULL, 0);
- tup = index_getnext(scan, ForwardScanDirection);
- if (tup != NULL)
+ slot = table_slot_create(rel, NULL);
+
+ if (index_getnext_slot(scan, ForwardScanDirection, slot))
{
- Datum dat = heap_getattr(tup, JSONBD_DICTIONARY_REL_ATT_ID,
- RelationGetDescr(rel), &isNull);
+ Datum dat = slot_getattr(slot, JSONBD_DICTIONARY_REL_ATT_ID, &isNull);
Assert(!isNull);
result = DatumGetInt32(dat);
}
index_endscan(scan);
+ ExecDropSingleTupleTableSlot(slot);
+
return result;
}
@@ -547,7 +551,7 @@ jsonbd_launcher_main(Datum arg)
BackgroundWorkerUnblockSignals();
/* Init this launcher as backend so workers can notify it */
- InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL);
+ InitPostgres(NULL, InvalidOid, NULL, InvalidOid, NULL, false);
/* Create resource owner */
CurrentResourceOwner = ResourceOwnerCreate(NULL, "jsonbd_launcher_main");
@@ -821,10 +825,10 @@ jsonbd_register_worker(int worker_num, Oid dboid, int database_num)
worker.bgw_start_time = BgWorkerStart_ConsistentState;
worker.bgw_restart_time = 0;
worker.bgw_notify_pid = MyProcPid;
- memcpy(worker.bgw_library_name, "jsonbd", BGW_MAXLEN);
- memcpy(worker.bgw_function_name, CppAsString(jsonbd_worker_main), BGW_MAXLEN);
- snprintf(worker.bgw_name, BGW_MAXLEN, "jsonbd, worker %d, db: %d",
- worker_num, dboid);
+ strcpy(worker.bgw_library_name, "jsonbd");
+ strcpy(worker.bgw_function_name, "jsonbd_worker_main");
+ strcpy(worker.bgw_name, "jsonbd worker");
+ strcpy(worker.bgw_type, "jsonbd worker");
worker.bgw_main_arg = UInt32GetDatum(dsm_segment_handle(seg));
/* Start dynamic worker */
@@ -835,11 +839,7 @@ jsonbd_register_worker(int worker_num, Oid dboid, int database_num)
}
/* Wait to be signalled. */
-#if PG_VERSION_NUM >= 100000
- WaitLatch(&hdr->launcher_latch, WL_LATCH_SET, 0, PG_WAIT_EXTENSION);
-#else
- WaitLatch(&hdr->launcher_latch, WL_LATCH_SET, 0);
-#endif
+ WaitLatch(&hdr->launcher_latch, WL_LATCH_SET | WL_POSTMASTER_DEATH, 0, PG_WAIT_EXTENSION);
ResetLatch(&hdr->launcher_latch);
@@ -892,14 +892,14 @@ jsonbd_get_dictionary_relid(void)
Assert(relid != InvalidOid);
- rel = relation_open(relid, NoLock);
+ rel = relation_open(relid, AccessShareLock);
indexes = RelationGetIndexList(rel);
Assert(list_length(indexes) == 2);
foreach(lc, indexes)
{
Oid indOid = lfirst_oid(lc);
- Relation indRel = index_open(indOid, NoLock);
+ Relation indRel = index_open(indOid, AccessShareLock);
int attnum = indRel->rd_index->indkey.values[1];
if (attnum == JSONBD_DICTIONARY_REL_ATT_ID)
@@ -910,9 +910,9 @@ jsonbd_get_dictionary_relid(void)
jsonbd_keys_indoid = indOid;
}
- index_close(indRel, NoLock);
+ index_close(indRel, AccessShareLock);
}
- relation_close(rel, NoLock);
+ relation_close(rel, AccessShareLock);
}
finish_xact_command();
Re: [HACKERS] Custom compression methods
On Sat, Mar 16, 2019 at 12:52 AM Ildus Kurbangaliev
<i.kurbangaliev@gmail.com> wrote:
in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.
Hi Ildus,
To maximise the chances of more review in the new Commitfest that is
about to begin, could you please send a fresh rebase? This doesn't
apply anymore.
Thanks,
--
Thomas Munro
https://enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi, Thomas!
On Mon, Jul 1, 2019 at 1:22 PM Thomas Munro <thomas.munro@gmail.com> wrote:
On Sat, Mar 16, 2019 at 12:52 AM Ildus Kurbangaliev
<i.kurbangaliev@gmail.com> wrote:in my opinion this patch is usually skipped not because it is not
needed, but because of its size. It is not hard to maintain it until
commiters will have time for it or I will get actual response that
nobody is going to commit it.To maximise the chances of more review in the new Commitfest that is
about to begin, could you please send a fresh rebase? This doesn't
apply anymore.
As I get we're currently need to make high-level decision of whether
we need this [1]. I was going to bring this topic up at last PGCon,
but I didn't manage to attend. Does it worth bothering Ildus with
continuous rebasing assuming we don't have this high-level decision
yet?
Links
1. /messages/by-id/20190216054526.zss2cufdxfeudr4i@alap3.anarazel.de
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On 2019-Jul-01, Alexander Korotkov wrote:
As I get we're currently need to make high-level decision of whether
we need this [1]. I was going to bring this topic up at last PGCon,
but I didn't manage to attend. Does it worth bothering Ildus with
continuous rebasing assuming we don't have this high-level decision
yet?
I agree that having to constantly rebase a patch that doesn't get acted
upon is a bit pointless. I see a bit of a process problem here: if the
patch doesn't apply, it gets punted out of commitfest and reviewers
don't look at it. This means the discussion goes unseen and no
decisions are made. My immediate suggestion is to rebase even if other
changes are needed. Longer-term I think it'd be useful to have patches
marked as needing "high-level decisions" that may lag behind current
master; maybe we have them provide a git commit-ID on top of which the
patch applies cleanly.
I recently found git-imerge which can make rebasing of large patch
series easier, by letting you deal with smaller conflicts one step at a
time rather than one giant conflict; it may prove useful.
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Jul 1, 2019 at 5:51 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
On 2019-Jul-01, Alexander Korotkov wrote:
As I get we're currently need to make high-level decision of whether
we need this [1]. I was going to bring this topic up at last PGCon,
but I didn't manage to attend. Does it worth bothering Ildus with
continuous rebasing assuming we don't have this high-level decision
yet?I agree that having to constantly rebase a patch that doesn't get acted
upon is a bit pointless. I see a bit of a process problem here: if the
patch doesn't apply, it gets punted out of commitfest and reviewers
don't look at it. This means the discussion goes unseen and no
decisions are made. My immediate suggestion is to rebase even if other
changes are needed.
OK, let's do this assuming Ildus didn't give up yet :)
Longer-term I think it'd be useful to have patches
marked as needing "high-level decisions" that may lag behind current
master; maybe we have them provide a git commit-ID on top of which the
patch applies cleanly.
+1,
Sounds like good approach for me.
I recently found git-imerge which can make rebasing of large patch
series easier, by letting you deal with smaller conflicts one step at a
time rather than one giant conflict; it may prove useful.
Thank you for pointing, will try.
------
Alexander Korotkov
Postgres Professional: http://www.postgrespro.com
The Russian Postgres Company
Re: [HACKERS] Custom compression methods
On Mon, 1 Jul 2019 at 17:28, Alexander Korotkov <a.korotkov@postgrespro.ru>
wrote:
On Mon, Jul 1, 2019 at 5:51 PM Alvaro Herrera <alvherre@2ndquadrant.com>
wrote:On 2019-Jul-01, Alexander Korotkov wrote:
As I get we're currently need to make high-level decision of whether
we need this [1]. I was going to bring this topic up at last PGCon,
but I didn't manage to attend. Does it worth bothering Ildus with
continuous rebasing assuming we don't have this high-level decision
yet?I agree that having to constantly rebase a patch that doesn't get acted
upon is a bit pointless. I see a bit of a process problem here: if the
patch doesn't apply, it gets punted out of commitfest and reviewers
don't look at it. This means the discussion goes unseen and no
decisions are made. My immediate suggestion is to rebase even if other
changes are needed.OK, let's do this assuming Ildus didn't give up yet :)
No, I still didn't give up :)
I'm going to post rebased version in few days. I found that are new
conflicts with
a slice decompression, not sure how to figure out them for now.
Also I was thinking maybe there is a point to add possibility to compress
any data
that goes to some column despite toast threshold size. In our company we
have
types that could benefit from compression even on smallest blocks.
Since pluggable storages were committed I think I should notice that
compression
methods also can be used by them and are not supposed to work only with
toast tables.
Basically it's just an interface to call compression functions which are
related with some column.
Best regards,
Ildus Kurbangaliev
Re: [HACKERS] Custom compression methods
Attached latest version of the patch.
Added slice decompression function to the compression handler.
Based on: 6b8548964bccd0f2e65c687d591b7345d5146bfa
Best regards,
Ildus Kurbangaliev
Best regards,
Ildus Kurbangaliev
Show quoted text
On Tue, 2 Jul 2019 at 15:05, Ildus K <i.kurbangaliev@gmail.com> wrote:
On Mon, 1 Jul 2019 at 17:28, Alexander Korotkov <a.korotkov@postgrespro.ru> wrote:
On Mon, Jul 1, 2019 at 5:51 PM Alvaro Herrera <alvherre@2ndquadrant.com> wrote:
On 2019-Jul-01, Alexander Korotkov wrote:
As I get we're currently need to make high-level decision of whether
we need this [1]. I was going to bring this topic up at last PGCon,
but I didn't manage to attend. Does it worth bothering Ildus with
continuous rebasing assuming we don't have this high-level decision
yet?I agree that having to constantly rebase a patch that doesn't get acted
upon is a bit pointless. I see a bit of a process problem here: if the
patch doesn't apply, it gets punted out of commitfest and reviewers
don't look at it. This means the discussion goes unseen and no
decisions are made. My immediate suggestion is to rebase even if other
changes are needed.OK, let's do this assuming Ildus didn't give up yet :)
No, I still didn't give up :)
I'm going to post rebased version in few days. I found that are new conflicts with
a slice decompression, not sure how to figure out them for now.Also I was thinking maybe there is a point to add possibility to compress any data
that goes to some column despite toast threshold size. In our company we have
types that could benefit from compression even on smallest blocks.Since pluggable storages were committed I think I should notice that compression
methods also can be used by them and are not supposed to work only with toast tables.
Basically it's just an interface to call compression functions which are related with some column.Best regards,
Ildus Kurbangaliev
Attachments:
custom_compression_methods_v23.patchtext/x-patch; charset=US-ASCII; name=custom_compression_methods_v23.patchDownload
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 2c999fd3eb..9613558920 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 3428a7c0fa..4f310f7b65 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -592,9 +597,10 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only tables and indexes have access methods. The requirements for table
- and index access methods are discussed in detail in <xref linkend="tableam"/> and
- <xref linkend="indexam"/> respectively.
+ Currently, there are tables, indexes and compression access methods.
+ The requirements for table, index and compression access methods
+ are discussed in detail in <xref linkend="tableam"/>,
+ <xref linkend="indexam"/> and <xref linkend="compression-am"/> respectively.
</para>
<table>
@@ -898,6 +904,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 3da2365ea9..3ec32dca4c 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY tableam SYSTEM "tableam.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index dd54c68802..f49f16fdb8 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -55,7 +55,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the index
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index 3e115f1c76..37141e9559 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -252,6 +252,7 @@
&geqo;
&tableam;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 90bf19564c..1484e42967 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -53,6 +53,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -360,6 +361,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 9009addb9c..d945b3eba9 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -953,6 +954,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 1047c77a63..7709c7e2f6 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -393,10 +393,12 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/Makefile b/src/backend/Makefile
index b03d5e510f..cafd178f16 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -45,7 +45,7 @@ OBJS = $(SUBDIROBJS) $(LOCALOBJS) $(top_builddir)/src/port/libpgport_srv.a \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ca00737c53 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- table tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 2b3861710c..4e3994533e 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -96,6 +96,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -187,6 +188,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index a48a6cd757..35acc28481 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -177,6 +177,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -207,6 +208,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -245,6 +249,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -268,6 +281,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -304,7 +320,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -328,6 +344,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -338,6 +355,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -752,6 +770,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -864,6 +883,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -882,6 +902,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -934,6 +955,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -944,6 +966,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1093,6 +1116,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1415,6 +1439,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index cb23be859d..dc399e0335 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -50,6 +50,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -77,13 +78,30 @@ index_form_tuple(TupleDesc tupleDescriptor,
/*
* If value is stored EXTERNAL, must fetch it so we are not depending
- * on outside storage. This should be improved someday.
+ * on outside storage. This should be improved someday. If value also
+ * was compressed by custom compression method then we should
+ * decompress it too.
*/
if (VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, DatumGetPointer(values[i]));
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ else
+ untoasted_values[i] =
+ PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
+ DatumGetPointer(values[i])));
+ untoasted_free[i] = true;
+ }
+ else if (VARATT_IS_CUSTOM_COMPRESSED(DatumGetPointer(values[i])))
{
untoasted_values[i] =
- PointerGetDatum(heap_tuple_fetch_attr((struct varlena *)
- DatumGetPointer(values[i])));
+ PointerGetDatum(heap_tuple_untoast_attr((struct varlena *)
+ DatumGetPointer(values[i])));
untoasted_free[i] = true;
}
@@ -95,7 +113,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
VARSIZE(DatumGetPointer(untoasted_values[i])) > TOAST_INDEX_TARGET &&
(att->attstorage == 'x' || att->attstorage == 'm'))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -145,6 +163,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 5773021499..643558751a 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -988,11 +988,100 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const void *a, const void *b)
+{
+ DefElem *da = (DefElem *) lfirst(*(ListCell **) a);
+ DefElem *db = (DefElem *) lfirst(*(ListCell **) b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ resoptions = list_qsort(options, compare_options);
+ else
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 6bc4e4c036..14d7715a7f 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts)
desc->constr = NULL;
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -137,6 +148,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -304,6 +317,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
dstAtt->attgenerated = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -470,6 +486,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -557,6 +575,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -663,6 +682,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..7ea5ee2e43
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..03c323f2c2
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,190 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid, List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ return result;
+}
+
+static struct varlena *
+pglz_cmdecompress_slice(CompressionAmOptions *cmoptions, const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..e9bf2772e2
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,250 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(Oid acoid, List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..504da0638f
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,96 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index d768b9b061..e10536363b 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -71,7 +71,7 @@
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options, BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tup,
@@ -1817,13 +1817,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -1885,7 +1886,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2052,7 +2053,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2089,8 +2090,12 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return toast_insert_or_update(relation, tup, NULL, options);
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ return toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2138,7 +2143,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
slots[i]->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slots[i]->tts_tableOid;
heaptuples[i] = heap_prepare_insert(relation, tuple, xid, cid,
- options);
+ options, bistate);
}
/*
@@ -3406,6 +3411,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
@@ -3508,7 +3515,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 72a448ad31..fa94066685 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -665,7 +665,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
options |= HEAP_INSERT_NO_LOGICAL;
heaptup = toast_insert_or_update(state->rs_new_rel, tup, NULL,
- options);
+ options, NULL);
}
else
heaptup = tup;
diff --git a/src/backend/access/heap/tuptoaster.c b/src/backend/access/heap/tuptoaster.c
index 55d6e91d1c..513e74f5d7 100644
--- a/src/backend/access/heap/tuptoaster.c
+++ b/src/backend/access/heap/tuptoaster.c
@@ -30,17 +30,25 @@
#include <unistd.h>
#include <fcntl.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/tuptoaster.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/expandeddatum.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
#include "utils/typcache.h"
@@ -52,19 +60,51 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_attr_compression */
+} toast_compress_header_custom;
+
+static HTAB *amoptions_cache = NULL;
+static MemoryContext amoptions_cache_mcxt = NULL;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
static void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
static Datum toast_save_datum(Relation rel, Datum value,
@@ -83,7 +123,8 @@ static int toast_open_indexes(Relation toastrel,
static void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
static void init_toast_snapshot(Snapshot toast_snapshot);
-
+static void init_amoptions_cache(void);
+static bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/* ----------
* heap_tuple_fetch_attr -
@@ -425,7 +466,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
@@ -515,6 +556,98 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
}
}
+/* ----------
+ * toast_prepare_varlena
+ *
+ * Untoast or fetch varlena if needed. This function could return
+ * compressed varlena.
+ */
+static struct varlena *
+toast_prepare_varlena(Form_pg_attribute att, struct varlena *value,
+ List *preserved_amoids, bool *modified)
+{
+ struct varlena *tmpval = NULL;
+
+ *modified = false;
+
+ /*
+ * We took care of UPDATE, so any external value we find still in the
+ * tuple must be someone else's that we cannot reuse (this includes the
+ * case of an out-of-line in-memory datum). Fetch it back (without
+ * decompression, unless we are forcing PLAIN storage or it has a custom
+ * compression). If necessary, we'll push it out as a new external value
+ * below.
+ */
+ if (VARATT_IS_EXTERNAL(value))
+ {
+ *modified = true;
+ if (att->attstorage == 'p')
+ {
+ value = heap_tuple_untoast_attr(value);
+
+ /* untoasted value does not need any further work */
+ return value;
+ }
+ else
+ value = heap_tuple_fetch_attr(value);
+ }
+
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ if (VARATT_IS_CUSTOM_COMPRESSED(value))
+ {
+ bool storage_ok = (att->attstorage == 'm' || att->attstorage == 'x');
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) value;
+
+ /* nothing to do */
+ if (storage_ok && hdr->cmid == att->attcompression)
+ return value;
+
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression, hdr->cmid))
+ {
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(value));
+ memcpy(tmpval, value, VARSIZE(value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ tmpval = heap_tuple_untoast_attr(value);
+ }
+ else
+ /* just decompress the value */
+ tmpval = heap_tuple_untoast_attr(value);
+
+ }
+ else if (OidIsValid(att->attcompression))
+ tmpval = heap_tuple_untoast_attr(value);
+
+ if (tmpval && tmpval != value)
+ {
+ /* if value was external we need to free fetched data */
+ if (*modified)
+ pfree(value);
+
+ value = tmpval;
+ *modified = true;
+ }
+
+ return value;
+}
/* ----------
* toast_insert_or_update -
@@ -526,6 +659,8 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
* newtup: the candidate new tuple to be inserted
* oldtup: the old row version for UPDATE, or NULL for INSERT
* options: options to be passed to heap_insert() for toast rows
+ * preserved_am_info: hash table with attnum as key, for lists of compression
+ * methods which should be preserved on decompression.
* Result:
* either newtup if no toasting is needed, or a palloc'd modified tuple
* that is what should actually get stored
@@ -536,7 +671,7 @@ toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -671,27 +806,37 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
if (att->attlen == -1)
{
+ bool modified = false;
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == 'p')
toast_action[i] = 'p';
+ if (VARATT_IS_EXTERNAL(new_value))
+ toast_oldexternal[i] = new_value;
+
/*
- * We took care of UPDATE above, so any external value we find
- * still in the tuple must be someone else's that we cannot reuse
- * (this includes the case of an out-of-line in-memory datum).
- * Fetch it back (without decompression, unless we are forcing
- * PLAIN storage). If necessary, we'll push it out as a new
- * external value below.
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
*/
- if (VARATT_IS_EXTERNAL(new_value))
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
+ new_value = toast_prepare_varlena(att,
+ new_value, preserved_amoids, &modified);
+ if (modified)
{
- toast_oldexternal[i] = new_value;
- if (att->attstorage == 'p')
- new_value = heap_tuple_untoast_attr(new_value);
- else
- new_value = heap_tuple_fetch_attr(new_value);
toast_values[i] = PointerGetDatum(new_value);
toast_free[i] = true;
need_change = true;
@@ -743,12 +888,14 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
Datum old_value;
Datum new_value;
+ Form_pg_attribute att;
+
/*
* Search for the biggest yet unprocessed internal attribute
*/
for (i = 0; i < numAttrs; i++)
{
- Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
+ char attstorage;
if (toast_action[i] != ' ')
continue;
@@ -756,7 +903,9 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
continue; /* can't happen, toast_action would be 'p' */
if (VARATT_IS_COMPRESSED(DatumGetPointer(toast_values[i])))
continue;
- if (att->attstorage != 'x' && att->attstorage != 'e')
+
+ attstorage = (TupleDescAttr(tupleDesc, i))->attstorage;
+ if (attstorage != 'x' && attstorage != 'e')
continue;
if (toast_sizes[i] > biggest_size)
{
@@ -772,10 +921,11 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
* Attempt to compress it inline, if it has attstorage 'x'
*/
i = biggest_attno;
- if (TupleDescAttr(tupleDesc, i)->attstorage == 'x')
+ att = TupleDescAttr(tupleDesc, i);
+ if (att->attstorage == 'x')
{
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value, att->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -916,7 +1066,8 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
*/
i = biggest_attno;
old_value = toast_values[i];
- new_value = toast_compress_datum(old_value);
+ new_value = toast_compress_datum(old_value,
+ TupleDescAttr(tupleDesc, i)->attcompression);
if (DatumGetPointer(new_value) != NULL)
{
@@ -1044,6 +1195,7 @@ toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
}
else
@@ -1265,6 +1417,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
@@ -1344,6 +1497,18 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
return new_tuple;
}
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
/* ----------
* toast_compress_datum -
@@ -1359,54 +1524,47 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid acoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
@@ -1501,19 +1659,20 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo contains the actual size of the data payload in the toast
+ * records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -1521,7 +1680,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -1530,7 +1692,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -1890,7 +2052,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ressize = toast_pointer.va_extsize;
+ ressize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
numchunks = ((ressize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
result = (struct varlena *) palloc(ressize + VARHDRSZ);
@@ -2077,7 +2239,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset, int32 length)
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
totalchunks = ((attrsize - 1) / TOAST_MAX_CHUNK_SIZE) + 1;
if (sliceoffset >= attrsize)
@@ -2270,20 +2432,36 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ int rawsize;
+
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr), true);
+ if (rawsize < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ }
return result;
}
-
/* ----------
* toast_decompress_datum_slice -
*
@@ -2299,16 +2477,32 @@ toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions *cmoptions;
+ toast_compress_header_custom *hdr;
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ hdr = (toast_compress_header_custom *) attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ if (cmoptions->amroutine->cmdecompress_slice)
+ result = cmoptions->amroutine->cmdecompress_slice(cmoptions, attr, slicelength);
+ else
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ }
- SET_VARSIZE(result, rawsize + VARHDRSZ);
return result;
}
@@ -2409,3 +2603,159 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *) &entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *) hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+static void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum) 0);
+}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Return or generate cache record for attribute compression.
+ */
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
+{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
+ regproc amhandler;
+
+ /*
+ * This call could invalidate system cache so we need to call it before
+ * we're putting something to our cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreate(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+static bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 43627ab8f4..7f93ddff41 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -763,6 +763,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 8bece078dd..20edc2aa42 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -34,7 +34,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h pg_statistic_ext_data.h \
+ pg_statistic_ext.h pg_statistic_ext_data.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h pg_pltemplate.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6315fc4b2f..0a5f8c7946 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -25,6 +25,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -181,7 +182,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1480,6 +1482,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropTransformById(object->objectId);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2775,6 +2781,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 3b8c8b193a..7a7100d662 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -725,6 +725,7 @@ InsertPgAttributeTuple(Relation pg_attribute_rel,
values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(new_attribute->attislocal);
values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(new_attribute->attinhcount);
values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(new_attribute->attcollation);
+ values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(new_attribute->attcompression);
/* start out with empty permissions and empty options */
nulls[Anum_pg_attribute_attacl - 1] = true;
@@ -1636,6 +1637,9 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ /* Unset attribute compression Oid */
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index bb60b23093..0e185f6d00 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -367,6 +367,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -453,6 +454,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index caf48cefa9..e709a61ac3 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -26,6 +26,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -525,6 +526,18 @@ static const ObjectPropertyType ObjectProperty[] =
InvalidAttrNumber, /* no ACL (same as relation) */
OBJECT_STATISTIC_EXT,
true
+ },
+ {
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
}
};
@@ -3618,6 +3631,37 @@ getObjectDescription(const ObjectAddress *object)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4147,6 +4191,10 @@ getObjectTypeDescription(const ObjectAddress *object)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -4693,6 +4741,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d64628566d..ce7ef52e7a 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -13,8 +13,8 @@ top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
OBJS = amcmds.o aggregatecmds.o alter.o analyze.o async.o cluster.o comment.o \
- collationcmds.o constraint.o conversioncmds.o copy.o createas.o \
- dbcommands.o define.o discard.o dropcmds.o \
+ collationcmds.o compressioncmds.o constraint.o conversioncmds.o copy.o \
+ createas.o dbcommands.o define.o discard.o dropcmds.o \
event_trigger.o explain.o extension.o foreigncmds.o functioncmds.o \
indexcmds.o lockcmds.o matview.o operatorcmds.o opclasscmds.o \
policy.o portalcmds.o prepare.o proclang.o publicationcmds.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index 70dbcb0756..e537361f26 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -638,6 +638,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index c0e40980d5..fe898b7289 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -179,6 +179,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -199,6 +263,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -241,6 +315,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -278,6 +354,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..39ee3cf38e
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,672 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attr_compression_d.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_collation_d.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc_d.h"
+#include "catalog/pg_type_d.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
+/*
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = heap_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+ char *amname;
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else if (DatumGetPointer(acoptions) != NULL)
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq, &arrayeq_info, DEFAULT_COLLATION_OID, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
+ */
+Oid
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
+{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == 'p')
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ *need_rewrite = true;
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+add_tuple:
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is built-in attribute compression */
+ heap_close(rel, RowExclusiveLock);
+ return acoid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ heap_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
+}
+
+/*
+ * Remove the attribute compression record from pg_attr_compression.
+ */
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(((Form_pg_attr_compression) GETSTRUCT(tup))->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ heap_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+ Assert(!IsBinaryUpgrade);
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ heap_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = heap_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
+{
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
+
+ if (!OidIsValid(acoid))
+ return NULL;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
+ ReleaseSysCache(tuple);
+
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = heap_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ heap_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
+}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index f1161f0fee..af88740946 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2354,7 +2354,7 @@ CopyMultiInsertBufferInit(ResultRelInfo *rri)
buffer = (CopyMultiInsertBuffer *) palloc(sizeof(CopyMultiInsertBuffer));
memset(buffer->slots, 0, sizeof(TupleTableSlot *) * MAX_BUFFERED_TUPLES);
buffer->resultRelInfo = rri;
- buffer->bistate = GetBulkInsertState();
+ buffer->bistate = GetBulkInsertState(NULL);
buffer->nused = 0;
return buffer;
@@ -2975,7 +2975,7 @@ CopyFrom(CopyState cstate)
{
singleslot = table_slot_create(resultRelInfo->ri_RelationDesc,
&estate->es_tupleTable);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
}
has_before_insert_row_trig = (resultRelInfo->ri_TrigDesc &&
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 4c1d909d38..134c1c125c 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -560,7 +560,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
*/
myState->ti_options = TABLE_INSERT_SKIP_FSM |
(XLogIsNeeded() ? 0 : TABLE_INSERT_SKIP_WAL);
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(intoRelationDesc) == InvalidBlockNumber);
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index efef120c03..a99e018b0b 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1216,6 +1216,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index d7bc6e35f0..0c4c71909e 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -179,7 +135,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 537d0e8cef..ee63c09afe 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -465,7 +465,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->ti_options = TABLE_INSERT_SKIP_FSM | TABLE_INSERT_FROZEN;
if (!XLogIsNeeded())
myState->ti_options |= TABLE_INSERT_SKIP_WAL;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/* Not using WAL requires smgr_targblock be initially invalid */
Assert(RelationGetTargetBlock(transientrel) == InvalidBlockNumber);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3aee2d82ce..d55f574c4d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -24,6 +24,7 @@
#include "access/sysattr.h"
#include "access/tableam.h"
#include "access/tupconvert.h"
+#include "access/tuptoaster.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -35,6 +36,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -42,6 +44,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -88,6 +91,7 @@
#include "storage/smgr.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -162,6 +166,9 @@ typedef struct AlteredTableInfo
List *constraints; /* List of NewConstraint */
List *newvals; /* List of NewColumnValue */
bool verify_new_notnull; /* T if we should recheck NOT NULL */
+ bool new_notnull; /* T if we added new NOT NULL constraints */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
bool chgPersistence; /* T if SET LOGGED/UNLOGGED is used */
@@ -368,6 +375,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -501,6 +510,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -554,6 +565,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -766,6 +778,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -816,6 +830,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression, NULL, NULL);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -883,6 +904,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2307,6 +2345,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2341,6 +2392,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattno[parent_attno - 1] = ++child_attno;
}
@@ -2550,6 +2602,13 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3635,6 +3694,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4110,6 +4170,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4473,6 +4534,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -4533,8 +4599,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode)
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, we are adding/removing the OID column, or we are
- * changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -4768,7 +4835,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
ti_options = TABLE_INSERT_SKIP_FSM;
if (!XLogIsNeeded())
@@ -5052,6 +5119,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
table_close(oldrel, NoLock);
if (newrel)
{
@@ -5702,6 +5787,16 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuple */
ReleaseSysCache(typeTuple);
@@ -5866,6 +5961,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -5988,6 +6084,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = heap_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ heap_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -6010,6 +6130,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = heap_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ heap_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -6930,6 +7150,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -9959,6 +10183,45 @@ createForeignKeyCheckTriggers(Oid myRelOid, Oid refRelOid,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -10746,6 +11009,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -10856,7 +11125,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AttrCompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject));
@@ -10948,6 +11218,34 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
+ attTup->attcompression = InvalidOid;
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
+ attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -10956,6 +11254,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14023,6 +14326,86 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = heap_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ heap_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 78deade89b..313d812b53 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2877,6 +2877,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2896,6 +2897,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5582,6 +5595,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 4f2ebe5118..bfaecaac69 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2557,6 +2557,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2576,6 +2577,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3642,6 +3653,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 05ae73f7db..9555c6bbeb 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3688,6 +3688,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
@@ -3695,6 +3697,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8400dd319e..aa97a20bd4 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2782,6 +2782,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2799,6 +2800,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4136,6 +4147,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 208b4a1f28..7189924757 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -405,6 +405,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
%type <list> group_by_list
@@ -590,6 +591,10 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -622,9 +627,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2219,6 +2224,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3326,11 +3340,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3339,8 +3354,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3385,6 +3400,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
+optColumnCompression:
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3614,6 +3666,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5312,6 +5365,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
@@ -15055,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 7450d74b7a..f7f27c025f 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1097,6 +1097,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = MakeColumnCompression(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 591377d2cd..34a6e5ad5e 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -3122,7 +3122,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 166c863026..600a8700e3 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -808,6 +808,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 99db5ba389..0e81e70e09 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -116,6 +116,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 5c886cfe96..db18724625 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -419,3 +419,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(opaque);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index 2b992d7832..d7f9b8be72 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -585,6 +585,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 16297a52a1..fbb5a9b0b1 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -188,6 +189,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index db30b54a92..6153a4e6fb 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -150,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 806fc78f04..763e768c36 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -41,11 +41,13 @@
#include "getopt_long.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -388,6 +390,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -882,6 +885,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8227,9 +8232,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
int i_attcollation;
int i_attfdwoptions;
int i_attmissingval;
+ int i_attcmoptions;
+ int i_attcmname;
PGresult *res;
int ntups;
bool hasdefaults;
+ bool createWithCompression;
for (i = 0; i < numTables; i++)
{
@@ -8278,6 +8286,23 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attgenerated,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8330,7 +8355,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8359,6 +8390,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
i_attcollation = PQfnumber(res, "attcollation");
i_attfdwoptions = PQfnumber(res, "attfdwoptions");
i_attmissingval = PQfnumber(res, "attmissingval");
+ i_attcmname = PQfnumber(res, "attcmname");
+ i_attcmoptions = PQfnumber(res, "attcmoptions");
tbinfo->numatts = ntups;
tbinfo->attnames = (char **) pg_malloc(ntups * sizeof(char *));
@@ -8377,9 +8410,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (j = 0; j < ntups; j++)
@@ -8405,6 +8441,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, i_attcollation));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, i_attfdwoptions));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, i_attmissingval));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, i_attcmoptions));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, i_attcmname));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, i_atthasdef)[0] == 't')
hasdefaults = true;
@@ -8619,6 +8657,103 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -12728,6 +12863,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
break;
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
+ break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
@@ -15615,6 +15753,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_custom_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15639,6 +15778,14 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15674,6 +15821,25 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
@@ -16121,6 +16287,34 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "OPTIONS (\n %s\n);\n",
tbinfo->attfdwoptions[j]);
}
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index c3c2ea1473..e273ee30f3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,10 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmoptions; /* per-attribute current compression options */
+ char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
+
/*
* Stuff computed only for dumpable tables.
*/
@@ -348,6 +352,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 158c0c74b2..0822d549c0 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -77,6 +77,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -143,6 +144,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -428,6 +430,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -649,6 +653,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index f9b1ae6809..4b6d930219 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -75,6 +75,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -124,6 +125,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -380,6 +382,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index c56bf00e4b..fdde896477 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -651,6 +651,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1373,6 +1410,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2480,6 +2528,53 @@ my %tests = (
},
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 8b4cd53631..1a64927b97 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1472,6 +1472,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1889,6 +1890,24 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2015,6 +2034,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2090,6 +2111,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9009b05e12..ac26451da8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1958,11 +1958,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..86c5e7ba64
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,84 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_attr_compression.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_AC_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value,
+ int32 slicelength);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index 858bcb6bc9..7624c8e6e4 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -26,6 +26,7 @@
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -130,7 +131,7 @@ extern bool heap_hot_search_buffer(ItemPointer tid, Relation relation,
extern void heap_get_latest_tid(TableScanDesc scan, ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index dbaabcc073..7383a63f9d 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -31,6 +31,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 27f963e9e8..97dd38f111 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -274,7 +274,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -673,6 +675,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -779,7 +784,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index a1912f41e6..f06dc16fd1 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -269,6 +269,8 @@ extern void fillRelOptions(void *rdopts, Size basesize,
relopt_value *options, int numoptions,
bool validate,
const relopt_parse_elt *elems, int nelems);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index a06800555c..f357272b0a 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -45,6 +47,10 @@ typedef struct TupleConstr
bool has_generated_stored;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -81,6 +87,7 @@ typedef struct TupleDescData
int natts; /* number of attributes in the tuple */
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/access/tuptoaster.h b/src/include/access/tuptoaster.h
index f0aea2496b..113bc693c6 100644
--- a/src/include/access/tuptoaster.h
+++ b/src/include/access/tuptoaster.h
@@ -16,6 +16,7 @@
#include "access/htup_details.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
+#include "utils/hsearch.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -101,6 +102,15 @@
/* Size of an EXTERNAL datum that contains an indirection pointer */
#define INDIRECT_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_indirect))
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
/*
* Testing whether an externally-stored value is compressed now requires
* comparing extsize (the actual length of the external data) to rawsize
@@ -109,7 +119,7 @@
* saves space, so we expect either equality or less-than.
*/
#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -126,6 +136,16 @@ do { \
memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
} while (0)
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
/* ----------
* toast_insert_or_update -
*
@@ -134,7 +154,7 @@ do { \
*/
extern HeapTuple toast_insert_or_update(Relation rel,
HeapTuple newtup, HeapTuple oldtup,
- int options);
+ int options, HTAB *preserved_cmlist);
/* ----------
* toast_delete -
@@ -210,7 +230,7 @@ extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
* Create a compressed version of a varlena datum, if possible
* ----------
*/
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoptoid);
/* ----------
* toast_raw_datum_size -
@@ -236,4 +256,19 @@ extern Size toast_datum_size(Datum value);
*/
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TUPTOASTER_H */
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index 2927b7a4d3..00f91e4b90 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -25,6 +25,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index ef9c86864c..3a5c03ff6a 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index ef4445b017..bf98919d6f 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -90,6 +90,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 2030, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 2030
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 2121, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 393b41dd68..773b8597e1 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4191', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4192', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 706b5e81cb..8199f5c05d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -54,6 +54,7 @@ typedef FormData_pg_am *Form_pg_am;
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..cd716b4a5b
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,24 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ acoid => '4191', acname => 'pglz' },
+{ acoid => '4192', acname => 'zlib' },
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..d6550a0eec
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,5555,AttrCompressionRelationId)
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 04004b5703..e326df2180 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -163,6 +163,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,10 +190,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_class.dat b/src/include/catalog/pg_class.dat
index 9bcf28676d..82e75d70f9 100644
--- a/src/include/catalog/pg_class.dat
+++ b/src/include/catalog/pg_class.dat
@@ -34,7 +34,7 @@
relname => 'pg_attribute', reltype => 'pg_attribute', relam => 'heap',
relfilenode => '0', relpages => '0', reltuples => '0', relallvisible => '0',
reltoastrelid => '0', relhasindex => 'f', relisshared => 'f',
- relpersistence => 'p', relkind => 'r', relnatts => '25', relchecks => '0',
+ relpersistence => 'p', relkind => 'r', relnatts => '26', relchecks => '0',
relhasrules => 'f', relhastriggers => 'f', relhassubclass => 'f',
relrowsecurity => 'f', relforcerowsecurity => 'f', relispopulated => 't',
relreplident => 'n', relispartition => 'f', relfrozenxid => '3',
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 604470cb81..67e39ead3b 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -909,6 +909,16 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4193', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4194', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -6872,6 +6882,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2023', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7048,6 +7061,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '270', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '271', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
# tablesample method handlers
{ oid => '3313', descr => 'BERNOULLI tablesample method handler',
@@ -10080,6 +10100,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4035', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index be49e00114..0be6ba0306 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -578,6 +578,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index cc5dfed0bf..22f1b18cee 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -75,6 +75,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 5556, 5558);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index b4e7db67c3..7101af08c0 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,7 @@ extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void RemoveUserMappingById(Oid umId);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -157,8 +158,21 @@ extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern void RemoveAccessMethodById(Oid amOid);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 575e9243e5..3b70907d0b 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -31,6 +31,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 4e2fb39105..3b50a2e1f5 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -475,6 +475,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
@@ -505,6 +506,7 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
+ T_CompressionAmRoutine, /* in access/cmapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 94ded3c135..e0675e507e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -622,6 +622,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -645,6 +659,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -683,6 +698,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1824,7 +1840,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 00ace8425e..f274434fee 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 057a3413ac..a8e818347b 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index 918765cc99..20b8788a90 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c53ed3ebf5..1753a9744f 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..d3b4095fea
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,404 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 'pg_catalog.pg_attr_compression'::REGCLASS OR
+ classid = 'pg_catalog.pg_attr_compression'::REGCLASS) AND
+ (objid = 'cmtest'::REGCLASS OR refobjid = 'cmtest'::REGCLASS)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1, pglz2
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1, pglz2
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS OR acrelid = 'cmtest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: invalid input syntax for type integer: "best"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '9');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '1');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..e38ae0cecf
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,407 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 'pg_catalog.pg_attr_compression'::REGCLASS OR
+ classid = 'pg_catalog.pg_attr_compression'::REGCLASS) AND
+ (objid = 'cmtest'::REGCLASS OR refobjid = 'cmtest'::REGCLASS)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1, pglz2
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1, pglz2
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS OR acrelid = 'cmtest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '9');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '1');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 262abf2bfb..7cfb8e3e4e 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -479,11 +479,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -492,11 +492,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -513,10 +513,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -919,21 +919,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -941,11 +941,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -974,46 +974,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1045,11 +1045,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1058,10 +1058,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1071,10 +1071,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index b153d6adb1..7ec153f5ed 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -204,32 +204,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -243,12 +243,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -258,12 +258,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -277,11 +277,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 4ff1b4af41..5255951060 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index c6abcfc3cb..01a16a532c 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1042,13 +1042,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1061,14 +1061,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1078,14 +1078,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1125,33 +1125,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1162,27 +1162,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1192,37 +1192,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index 75e25cdf48..1829626e37 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 85af36ee5b..87588e0f12 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1827,6 +1827,18 @@ WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND
-----+--------+-----+---------
(0 rows)
+-- Check for compression amhandler functions with the wrong signature
+SELECT p1.oid, p1.amname, p2.oid, p2.proname
+FROM pg_am AS p1, pg_proc AS p2
+WHERE p2.oid = p1.amhandler AND p1.amtype = 'c' AND
+ (p2.prorettype != 'compression_am_handler'::regtype
+ OR p2.proretset
+ OR p2.pronargs != 1
+ OR p2.proargtypes[0] != 'internal'::regtype);
+ oid | amname | oid | proname
+-----+--------+-----+---------
+(0 rows)
+
-- **************** pg_amop ****************
-- Look for illegal values in pg_amop fields
SELECT p1.amopfamily, p1.amopstrategy
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 9021c808dc..bcb4b1e13d 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2779,34 +2779,34 @@ CREATE ACCESS METHOD heap_psql TYPE TABLE HANDLER heap_tableam_handler;
CREATE TABLE tbl_heap_psql(f1 int, f2 char(100)) using heap_psql;
CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
\d+ tbl_heap_psql
- Table "public.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "public.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "public.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "public.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
\set HIDE_TABLEAM on
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 0e5e8f2b92..d8f4233d45 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -65,11 +65,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -141,22 +141,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -178,11 +178,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 739608aab4..7e38d015e4 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index aa3b083ee3..15585f3c0e 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 210e9cd146..7d8c0b32ab 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2927,11 +2927,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -2947,11 +2947,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 8ff0da185e..1eec5f84cc 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -103,6 +105,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index 2083345c8e..b245059e37 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -690,14 +690,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 8fb55f045e..4b766826f1 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index a39ca1012a..c03affb334 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -79,6 +79,7 @@ test: drop_if_exists
test: updatable_views
test: roleattributes
test: create_am
+test: create_cm
test: hash_func
test: errors
test: sanity_check
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..a598484253
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,202 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 'pg_catalog.pg_attr_compression'::REGCLASS OR
+ classid = 'pg_catalog.pg_attr_compression'::REGCLASS) AND
+ (objid = 'cmtest'::REGCLASS OR refobjid = 'cmtest'::REGCLASS)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS OR acrelid = 'cmtest'::REGCLASS;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '9');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '1');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index 1227ef79f0..fe65d44041 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1221,6 +1221,16 @@ WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
+-- Check for compression amhandler functions with the wrong signature
+
+SELECT p1.oid, p1.amname, p2.oid, p2.proname
+FROM pg_am AS p1, pg_proc AS p2
+WHERE p2.oid = p1.amhandler AND p1.amtype = 'c' AND
+ (p2.prorettype != 'compression_am_handler'::regtype
+ OR p2.proretset
+ OR p2.pronargs != 1
+ OR p2.proargtypes[0] != 'internal'::regtype);
+
-- **************** pg_amop ****************
-- Look for illegal values in pg_amop fields
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 432d2d812e..d4ef37e701 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -358,6 +358,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
@@ -383,6 +384,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ConditionVariable
ConditionalStack
Re: [HACKERS] Custom compression methods
The compile of this one has been broken for a long time. Is there a
rebase happening?
--
�lvaro Herrera https://www.2ndQuadrant.com/
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: Re: [HACKERS] Custom compression methods
On Thu, Mar 7, 2019 at 2:51 AM Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:
Yes. I took a look at code of this patch. I think it's in pretty good shape. But high level review/discussion is required.
I agree that the code of this patch is in pretty good shape, although
there is a lot of rebasing needed at this point. Here is an attempt at
some high level review and discussion:
- As far as I can see, there is broad agreement that we shouldn't
consider ourselves to be locked into 'pglz' forever. I believe
numerous people have reported that there are other methods of doing
compression that either compress better, or compress faster, or
decompress faster, or all of the above. This isn't surprising and nor
is it a knock on 'pglz'; Jan committed it in 1999, and it's not
surprising that in 20 years some people have come up with better
ideas. Not only that, but the quantity and quality of open source
software that is available for this kind of thing and for many other
kinds of things have improved dramatically in that time.
- I can see three possible ways of breaking our dependence on 'pglz'
for TOAST compression. Option #1 is to pick one new algorithm which we
think is better than 'pglz' in all relevant ways and use it as the
default for all new compressed datums. This would be dramatically
simpler than what this patch does, because there would be no user
interface. It would just be out with the old and in with the new.
Option #2 is to create a short list of new algorithms that have
different trade-offs; e.g. one that is very fast (like lz4) and one
that has an extremely high compression ratio, and provide an interface
for users to choose between them. This would be moderately simpler
than what this patch does, because we would expose to the user
anything about how a new compression method could be added, but it
would still require a UI for the user to choose between the available
(and hard-coded) options. It has the further advantage that every
PostgreSQL cluster will offer the same options (or a subset of them,
perhaps, depending on configure flags) and so you don't have to worry
that, say, a pg_am row gets lost and suddenly all of your toasted data
is inaccessible and uninterpretable. Option #3 is to do what this
patch actually does, which is to allow for the addition of any number
of compressors, including by extensions. It has the advantage that new
compressors can be added with core's permission, so, for example, if
it is unclear whether some excellent compressor is free of patent
problems, we can elect not to ship support for it in core, while at
the same time people who are willing to accept the associated legal
risk can add that functionality to their own copy as an extension
without having to patch core. The legal climate may even vary by
jurisdiction, so what might be questionable in country A might be
clearly just fine in country B. Aside from those issues, this approach
allows people to experiment and innovate outside of core relatively
quickly, instead of being bound by the somewhat cumbrous development
process which has left this patch in limbo for the last few years. My
view is that option #1 is likely to be impractical, because getting
people to agree is hard, and better things are likely to come along
later, and people like options. So I prefer either #2 or #3.
- The next question is how a datum compressed with some non-default
method should be represented on disk. The patch handles this first of
all by making the observation that the compressed size can't be >=1GB,
because the uncompressed size can't be >=1GB, and we wouldn't have
stored it compressed if it expanded. Therefore, the upper two bits of
the compressed size should always be zero on disk, and the patch
steals one of them to indicate whether "custom" compression is in use.
If it is, the 4-byte varlena header is followed not only by a 4-byte
size (now with the new flag bit also included) but also by a 4-byte
OID, indicating the compression AM in use. I don't think this is a
terrible approach, but I don't think it's amazing, either. 4 bytes is
quite a bit to use for this; if I guess correctly what will be a
typical cluster configuration, you probably would really only need
about 2 bits. For a datum that is both stored externally and
compressed, the overhead is likely negligible, because the length is
probably measured in kB or MB. But for a datum that is compressed but
not stored externally, it seems pretty expensive; the datum is
probably short, and having an extra 4 bytes of uncompressible data
kinda sucks. One possibility would be to allow only one byte here:
require each compression AM that is installed to advertise a one-byte
value that will denote its compressed datums. If more than one AM
tries to claim the same byte value, complain. Another possibility is
to abandon this approach and go with #2 from the previous paragraph.
Or maybe we add 1 or 2 "privileged" built-in compressors that get
dedicated bit-patterns in the upper 2 bits of the size field, with the
last bit pattern being reserved for future algorithms. (e.g. 0x00 =
pglz, 0x01 = lz4, 0x10 = zstd, 0x11 = something else - see within for
details).
- I don't really like the use of the phrase "custom compression". I
think the terminology needs to be rethought so that we just talk about
compression methods. Perhaps in certain contexts we need to specify
that we mean extensible compression methods or user-provided
compression methods or something like that, but I don't think the word
"custom" is very well-suited here. The main point of this shouldn't be
for every cluster in the universe to use a different approach to
compression, or to compress columns within a database in 47 different
ways, but rather to help us get out from under 'pglz'. Eventually we
probably want to change the default, but as the patch phrases things
now, that default would be a custom method, which is almost a
contradiction in terms.
- Yet another possible approach to the on-disk format is to leave
varatt_external.va_extsize and varattrib_4b.rawsize untouched and
instead add new compression methods by adding new vartag_external
values. There's quite a lot of bit-space available there: we have a
whole byte, and we're currently only using 4 values. We could easily
add a half-dozen new possibilities there for new compression methods
without sweating the bit-space consumption. The main thing I don't
like about this is that it only seems like a useful way to provide for
out-of-line compression. Perhaps it could be generalized to allow for
inline compression as well, but it seems like it would take some
hacking.
- One thing I really don't like about the patch is that it consumes a
bit from infomask2 for a new flag HEAP_HASCUSTOMCOMPRESSED. infomask
bits are at a premium, and there's been no real progress in the decade
plus that I've been hanging around here in clawing back any bit-space.
I think we really need to avoid burning our remaining bits for
anything other than a really critical need, and I don't think I
understand what the need is in this case. I might be missing
something, but I'd really strongly suggest looking for a way to get
rid of this. It also invents the concept of a TupleDesc flag, and the
flag invented is TD_ATTR_CUSTOM_COMPRESSED; I'm not sure I see why we
need that, either.
- It seems like this kind of approach has a sort of built-in
circularity problem. It means that every place that might need to
detoast a datum needs to be able to access the pg_am catalog. I wonder
if that's actually true. For instance, consider logical decoding. I
guess that can do catalog lookups in general, but can it do them from
the places where detoasting is happening? Moreover, can it do them
with the right snapshot? Suppose we rewrite a table to change the
compression method, then drop the old compression method, then try to
decode a transaction that modified that table before those operations
were performed. As an even more extreme example, suppose we need to
open pg_am, and to do that we have to build a relcache entry for it,
and suppose the relevant pg_class entry had a relacl or reloptions
field that happened to be custom-compressed. Or equally suppose that
any of the various other tables we use when building a relcache entry
had the same kind of problem, especially those that have TOAST tables.
We could just disallow the use of non-default compressors in the
system catalogs, but the benefits mentioned in
/messages/by-id/5541614A.5030208@2ndquadrant.com seem too large to
ignore.
- I think it would be awfully appealing if we could find some way of
dividing this great big patch into some somewhat smaller patches. For
example:
Patch #1. Add syntax allowing a compression method to be specified,
but the only possible choice is pglz, and the PRESERVE stuff isn't
supported, and changing the value associated with an existing column
isn't supported, but we can add tab-completion support and stuff.
Patch #2. Add a second built-in method, like gzip or lz4.
Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.
Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.
Patch #5. Add support for compression methods via the AM interface.
Perhaps methods added in this manner are prohibited in system
catalogs. (This could also go before #4 or even before #3, but with a
noticeable hit to usability.)
Patch #6 (new development). Add a contrib module using the facility
added in #5, perhaps with a slightly off-beat compressor like bzip2
that is more of a niche use case.
I think that if the patch set were broken up this way, it would be a
lot easier to review and get committed. I think you could commit each
bit separately. I don't think you'd want to commit #1 unless you had a
sense that #2 was pretty close to done, and similarly for #5 and #6,
but that would still make things a lot easier than having one giant
monolithic patch, at least IMHO.
There might be more to say here, but that's what I have got for now. I
hope it helps.
Thanks,
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
Hi,
On 2020-06-19 13:03:02 -0400, Robert Haas wrote:
- I can see three possible ways of breaking our dependence on 'pglz'
for TOAST compression. Option #1 is to pick one new algorithm which we
think is better than 'pglz' in all relevant ways and use it as the
default for all new compressed datums. This would be dramatically
simpler than what this patch does, because there would be no user
interface. It would just be out with the old and in with the new.
Option #2 is to create a short list of new algorithms that have
different trade-offs; e.g. one that is very fast (like lz4) and one
that has an extremely high compression ratio, and provide an interface
for users to choose between them. This would be moderately simpler
than what this patch does, because we would expose to the user
anything about how a new compression method could be added, but it
would still require a UI for the user to choose between the available
(and hard-coded) options. It has the further advantage that every
PostgreSQL cluster will offer the same options (or a subset of them,
perhaps, depending on configure flags) and so you don't have to worry
that, say, a pg_am row gets lost and suddenly all of your toasted data
is inaccessible and uninterpretable. Option #3 is to do what this
patch actually does, which is to allow for the addition of any number
of compressors, including by extensions. It has the advantage that new
compressors can be added with core's permission, so, for example, if
it is unclear whether some excellent compressor is free of patent
problems, we can elect not to ship support for it in core, while at
the same time people who are willing to accept the associated legal
risk can add that functionality to their own copy as an extension
without having to patch core. The legal climate may even vary by
jurisdiction, so what might be questionable in country A might be
clearly just fine in country B. Aside from those issues, this approach
allows people to experiment and innovate outside of core relatively
quickly, instead of being bound by the somewhat cumbrous development
process which has left this patch in limbo for the last few years. My
view is that option #1 is likely to be impractical, because getting
people to agree is hard, and better things are likely to come along
later, and people like options. So I prefer either #2 or #3.
I personally favor going for #2, at least initially. Then we can discuss
the runtime-extensibility of #3 separately.
- The next question is how a datum compressed with some non-default
method should be represented on disk. The patch handles this first of
all by making the observation that the compressed size can't be >=1GB,
because the uncompressed size can't be >=1GB, and we wouldn't have
stored it compressed if it expanded. Therefore, the upper two bits of
the compressed size should always be zero on disk, and the patch
steals one of them to indicate whether "custom" compression is in use.
If it is, the 4-byte varlena header is followed not only by a 4-byte
size (now with the new flag bit also included) but also by a 4-byte
OID, indicating the compression AM in use. I don't think this is a
terrible approach, but I don't think it's amazing, either. 4 bytes is
quite a bit to use for this; if I guess correctly what will be a
typical cluster configuration, you probably would really only need
about 2 bits. For a datum that is both stored externally and
compressed, the overhead is likely negligible, because the length is
probably measured in kB or MB. But for a datum that is compressed but
not stored externally, it seems pretty expensive; the datum is
probably short, and having an extra 4 bytes of uncompressible data
kinda sucks. One possibility would be to allow only one byte here:
require each compression AM that is installed to advertise a one-byte
value that will denote its compressed datums. If more than one AM
tries to claim the same byte value, complain. Another possibility is
to abandon this approach and go with #2 from the previous paragraph.
Or maybe we add 1 or 2 "privileged" built-in compressors that get
dedicated bit-patterns in the upper 2 bits of the size field, with the
last bit pattern being reserved for future algorithms. (e.g. 0x00 =
pglz, 0x01 = lz4, 0x10 = zstd, 0x11 = something else - see within for
details).
Agreed. I favor an approach roughly like I'd implemented below
/messages/by-id/20130605150144.GD28067@alap2.anarazel.de
I.e. leave the vartag etc as-is, but utilize the fact that pglz
compressed datums starts with a 4 byte length header, and that due to
the 1GB limit, the first two bits currently have to be 0. That allows to
indicate 2 compression methods without any space overhead, and
additional compression methods are supported by using an additional byte
(or some variable length encoded larger amount) if both bits are 1.
- Yet another possible approach to the on-disk format is to leave
varatt_external.va_extsize and varattrib_4b.rawsize untouched and
instead add new compression methods by adding new vartag_external
values. There's quite a lot of bit-space available there: we have a
whole byte, and we're currently only using 4 values. We could easily
add a half-dozen new possibilities there for new compression methods
without sweating the bit-space consumption. The main thing I don't
like about this is that it only seems like a useful way to provide for
out-of-line compression. Perhaps it could be generalized to allow for
inline compression as well, but it seems like it would take some
hacking.
One additional note: Adding additional vartag_external values does incur
some noticable cost, distributed across lots of places.
- One thing I really don't like about the patch is that it consumes a
bit from infomask2 for a new flag HEAP_HASCUSTOMCOMPRESSED. infomask
bits are at a premium, and there's been no real progress in the decade
plus that I've been hanging around here in clawing back any bit-space.
I think we really need to avoid burning our remaining bits for
anything other than a really critical need, and I don't think I
understand what the need is in this case. I might be missing
something, but I'd really strongly suggest looking for a way to get
rid of this. It also invents the concept of a TupleDesc flag, and the
flag invented is TD_ATTR_CUSTOM_COMPRESSED; I'm not sure I see why we
need that, either.
+many
Small note: The current patch adds #include "postgres.h" to a few
headers - it shouldn't do so.
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On Mon, Jun 22, 2020 at 4:53 PM Andres Freund <andres@anarazel.de> wrote:
Or maybe we add 1 or 2 "privileged" built-in compressors that get
dedicated bit-patterns in the upper 2 bits of the size field, with the
last bit pattern being reserved for future algorithms. (e.g. 0x00 =
pglz, 0x01 = lz4, 0x10 = zstd, 0x11 = something else - see within for
details).Agreed. I favor an approach roughly like I'd implemented below
/messages/by-id/20130605150144.GD28067@alap2.anarazel.de
I.e. leave the vartag etc as-is, but utilize the fact that pglz
compressed datums starts with a 4 byte length header, and that due to
the 1GB limit, the first two bits currently have to be 0. That allows to
indicate 2 compression methods without any space overhead, and
additional compression methods are supported by using an additional byte
(or some variable length encoded larger amount) if both bits are 1.
I think there's essentially no difference between these two ideas,
unless the two bits we're talking about stealing are not the same in
the two cases. Am I missing something?
One additional note: Adding additional vartag_external values does incur
some noticable cost, distributed across lots of places.
OK.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
Hi,
On 2020-06-23 14:27:47 -0400, Robert Haas wrote:
On Mon, Jun 22, 2020 at 4:53 PM Andres Freund <andres@anarazel.de> wrote:
Or maybe we add 1 or 2 "privileged" built-in compressors that get
dedicated bit-patterns in the upper 2 bits of the size field, with the
last bit pattern being reserved for future algorithms. (e.g. 0x00 =
pglz, 0x01 = lz4, 0x10 = zstd, 0x11 = something else - see within for
details).Agreed. I favor an approach roughly like I'd implemented below
/messages/by-id/20130605150144.GD28067@alap2.anarazel.de
I.e. leave the vartag etc as-is, but utilize the fact that pglz
compressed datums starts with a 4 byte length header, and that due to
the 1GB limit, the first two bits currently have to be 0. That allows to
indicate 2 compression methods without any space overhead, and
additional compression methods are supported by using an additional byte
(or some variable length encoded larger amount) if both bits are 1.
/messages/by-id/20130621000900.GA12425@alap2.anarazel.de is a
thread with more information / patches further along.
I think there's essentially no difference between these two ideas,
unless the two bits we're talking about stealing are not the same in
the two cases. Am I missing something?
I confused this patch with the approach in
/messages/by-id/d8576096-76ba-487d-515b-44fdedba8bb5@2ndquadrant.com
sorry for that. It obviously still differs by not having lower space
overhead (by virtue of not having a 4 byte 'va_cmid', but no additional
space for two methods, and then 1 byte overhead for 256 more), but
that's not that fundamental a difference.
I do think it's nicer to hide the details of the compression inside
toast specific code as the version in the "further along" thread above
did.
The varlena stuff feels so archaic, it's hard to keep it all in my head...
I think I've pondered that elsewhere before (but perhaps just on IM with
you?), but I do think we'll need a better toast pointer format at some
point. It's pretty fundamentally based on having the 1GB limit, which I
don't think we can justify for that much longer.
Using something like /messages/by-id/20191210015054.5otdfuftxrqb5gum@alap3.anarazel.de
I'd probably make it something roughly like:
1) signed varint indicating "in-place" length
1a) if positive, it's "plain" "in-place" data
1b) if negative, data type indicator follows. abs(length) includes size of metadata.
2) optional: unsigned varint metadata type indicator
3) data
Because 1) is the size of the data, toast datums can be skipped with a
relatively low amount of instructions during tuple deforming. Instead of
needing a fair number of branches, as the case right now.
So a small in-place uncompressed varlena2 would have an overhead of 1
byte up to 63 bytes, and 2 bytes otherwise (with 8 kb pages at least).
An in-place compressed datum could have an overhead as low as 3 bytes (1
byte length, 1 byte indicator for type of compression, 1 byte raw size),
although I suspect it's rarely going to be useful at that small sizes.
Anyway. I think it's probably reasonable to utilize those two bits
before going to a new toast format. But if somebody were more interested
in working on toastv2 I'd not push back either.
Regards,
Andres
Re: [HACKERS] Custom compression methods
On Tue, Jun 23, 2020 at 4:00 PM Andres Freund <andres@anarazel.de> wrote:
/messages/by-id/20130621000900.GA12425@alap2.anarazel.de is a
thread with more information / patches further along.I confused this patch with the approach in
/messages/by-id/d8576096-76ba-487d-515b-44fdedba8bb5@2ndquadrant.com
sorry for that. It obviously still differs by not having lower space
overhead (by virtue of not having a 4 byte 'va_cmid', but no additional
space for two methods, and then 1 byte overhead for 256 more), but
that's not that fundamental a difference.
Wait a minute. Are we saying there are three (3) dueling patches for
adding an alternate TOAST algorithm? It seems like there is:
This "custom compression methods" thread - vintage 2017 - Original
code by Nikita Glukhov, later work by Ildus Kurbangaliev
The "pluggable compression support" thread - vintage 2013 - Andres Freund
The "plgz performance" thread - vintage 2019 - Petr Jelinek
Anyone want to point to a FOURTH implementation of this feature?
I guess the next thing to do is figure out which one is the best basis
for further work.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Wed, Jun 24, 2020 at 5:30 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Jun 23, 2020 at 4:00 PM Andres Freund <andres@anarazel.de> wrote:
/messages/by-id/20130621000900.GA12425@alap2.anarazel.de is a
thread with more information / patches further along.I confused this patch with the approach in
/messages/by-id/d8576096-76ba-487d-515b-44fdedba8bb5@2ndquadrant.com
sorry for that. It obviously still differs by not having lower space
overhead (by virtue of not having a 4 byte 'va_cmid', but no additional
space for two methods, and then 1 byte overhead for 256 more), but
that's not that fundamental a difference.Wait a minute. Are we saying there are three (3) dueling patches for
adding an alternate TOAST algorithm? It seems like there is:This "custom compression methods" thread - vintage 2017 - Original
code by Nikita Glukhov, later work by Ildus Kurbangaliev
The "pluggable compression support" thread - vintage 2013 - Andres Freund
The "plgz performance" thread - vintage 2019 - Petr JelinekAnyone want to point to a FOURTH implementation of this feature?
I guess the next thing to do is figure out which one is the best basis
for further work.
I have gone through these 3 threads and here is a summary of what I
understand from them. Feel free to correct me if I have missed
something.
#1. Custom compression methods: Provide a mechanism to create/drop
compression methods by using external libraries, and it also provides
a way to set the compression method for the columns/types. There are
a few complexities with this approach those are listed below:
a. We need to maintain the dependencies between the column and the
compression method. And the bigger issue is, even if the compression
method is changed, we need to maintain the dependencies with the older
compression methods as we might have some older tuples that were
compressed with older methods.
b. Inside the compressed attribute, we need to maintain the
compression method so that we know how to decompress it. For this, we
use 2 bits from the raw_size of the compressed varlena header.
#2. pglz performance: Along with pglz this patch provides an
additional compression method using lz4. The new compression method
can be enabled/disabled during configure time or using SIGHUP. We use
1 bit from the raw_size of the compressed varlena header to identify
the compression method (pglz or lz4).
#3. pluggable compression: This proposal is to replace the existing
pglz algorithm, with the snappy or lz4 whichever is better. As per
the performance data[1]/messages/by-id/20130621000900.GA12425@alap2.anarazel.de, it appeared that the lz4 is the winner in
most of the cases.
- This also provides an additional patch to plugin any compression method.
- This will also use 2 bits from the raw_size of the compressed
attribute for identifying the compression method.
- Provide an option to select the compression method using GUC, but
the comments in the patch suggest to remove the GUC. So it seems that
GUC was used only for the POC.
- Honestly, I did not clearly understand from this patch set that
whether it proposes to replace the existing compression method with
the best method (and the plugin is just provided for performance
testing) or it actually proposes an option to have pluggable
compression methods.
IMHO, We can provide a solution based on #1 and #2, i.e. we can
provide a few best compression methods in the core, and on top of
that, we can also provide a mechanism to create/drop the external
compression methods.
[1]: /messages/by-id/20130621000900.GA12425@alap2.anarazel.de
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
On 2020-06-24 07:59:47 -0400, Robert Haas wrote:
On Tue, Jun 23, 2020 at 4:00 PM Andres Freund <andres@anarazel.de> wrote:
/messages/by-id/20130621000900.GA12425@alap2.anarazel.de is a
thread with more information / patches further along.I confused this patch with the approach in
/messages/by-id/d8576096-76ba-487d-515b-44fdedba8bb5@2ndquadrant.com
sorry for that. It obviously still differs by not having lower space
overhead (by virtue of not having a 4 byte 'va_cmid', but no additional
space for two methods, and then 1 byte overhead for 256 more), but
that's not that fundamental a difference.Wait a minute. Are we saying there are three (3) dueling patches for
adding an alternate TOAST algorithm? It seems like there is:This "custom compression methods" thread - vintage 2017 - Original
code by Nikita Glukhov, later work by Ildus Kurbangaliev
The "pluggable compression support" thread - vintage 2013 - Andres Freund
The "plgz performance" thread - vintage 2019 - Petr JelinekAnyone want to point to a FOURTH implementation of this feature?
To be clear, I don't think the 2003 patch should be considered as being
"in the running".
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On Mon, Jun 29, 2020 at 12:31 PM Andres Freund <andres@anarazel.de> wrote:
This "custom compression methods" thread - vintage 2017 - Original
code by Nikita Glukhov, later work by Ildus Kurbangaliev
The "pluggable compression support" thread - vintage 2013 - Andres Freund
The "plgz performance" thread - vintage 2019 - Petr JelinekAnyone want to point to a FOURTH implementation of this feature?
To be clear, I don't think the 2003 patch should be considered as being
"in the running".
I guess you mean 2013, not 2003?
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: Re: [HACKERS] Custom compression methods
On Fri, Jun 19, 2020 at 10:33 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Mar 7, 2019 at 2:51 AM Alexander Korotkov
<a.korotkov@postgrespro.ru> wrote:Yes. I took a look at code of this patch. I think it's in pretty good shape. But high level review/discussion is required.
I agree that the code of this patch is in pretty good shape, although
there is a lot of rebasing needed at this point. Here is an attempt at
some high level review and discussion:- As far as I can see, there is broad agreement that we shouldn't
consider ourselves to be locked into 'pglz' forever. I believe
numerous people have reported that there are other methods of doing
compression that either compress better, or compress faster, or
decompress faster, or all of the above. This isn't surprising and nor
is it a knock on 'pglz'; Jan committed it in 1999, and it's not
surprising that in 20 years some people have come up with better
ideas. Not only that, but the quantity and quality of open source
software that is available for this kind of thing and for many other
kinds of things have improved dramatically in that time.- I can see three possible ways of breaking our dependence on 'pglz'
for TOAST compression. Option #1 is to pick one new algorithm which we
think is better than 'pglz' in all relevant ways and use it as the
default for all new compressed datums. This would be dramatically
simpler than what this patch does, because there would be no user
interface. It would just be out with the old and in with the new.
Option #2 is to create a short list of new algorithms that have
different trade-offs; e.g. one that is very fast (like lz4) and one
that has an extremely high compression ratio, and provide an interface
for users to choose between them. This would be moderately simpler
than what this patch does, because we would expose to the user
anything about how a new compression method could be added, but it
would still require a UI for the user to choose between the available
(and hard-coded) options. It has the further advantage that every
PostgreSQL cluster will offer the same options (or a subset of them,
perhaps, depending on configure flags) and so you don't have to worry
that, say, a pg_am row gets lost and suddenly all of your toasted data
is inaccessible and uninterpretable. Option #3 is to do what this
patch actually does, which is to allow for the addition of any number
of compressors, including by extensions. It has the advantage that new
compressors can be added with core's permission, so, for example, if
it is unclear whether some excellent compressor is free of patent
problems, we can elect not to ship support for it in core, while at
the same time people who are willing to accept the associated legal
risk can add that functionality to their own copy as an extension
without having to patch core. The legal climate may even vary by
jurisdiction, so what might be questionable in country A might be
clearly just fine in country B. Aside from those issues, this approach
allows people to experiment and innovate outside of core relatively
quickly, instead of being bound by the somewhat cumbrous development
process which has left this patch in limbo for the last few years. My
view is that option #1 is likely to be impractical, because getting
people to agree is hard, and better things are likely to come along
later, and people like options. So I prefer either #2 or #3.- The next question is how a datum compressed with some non-default
method should be represented on disk. The patch handles this first of
all by making the observation that the compressed size can't be >=1GB,
because the uncompressed size can't be >=1GB, and we wouldn't have
stored it compressed if it expanded. Therefore, the upper two bits of
the compressed size should always be zero on disk, and the patch
steals one of them to indicate whether "custom" compression is in use.
If it is, the 4-byte varlena header is followed not only by a 4-byte
size (now with the new flag bit also included) but also by a 4-byte
OID, indicating the compression AM in use. I don't think this is a
terrible approach, but I don't think it's amazing, either. 4 bytes is
quite a bit to use for this; if I guess correctly what will be a
typical cluster configuration, you probably would really only need
about 2 bits. For a datum that is both stored externally and
compressed, the overhead is likely negligible, because the length is
probably measured in kB or MB. But for a datum that is compressed but
not stored externally, it seems pretty expensive; the datum is
probably short, and having an extra 4 bytes of uncompressible data
kinda sucks. One possibility would be to allow only one byte here:
require each compression AM that is installed to advertise a one-byte
value that will denote its compressed datums. If more than one AM
tries to claim the same byte value, complain. Another possibility is
to abandon this approach and go with #2 from the previous paragraph.
Or maybe we add 1 or 2 "privileged" built-in compressors that get
dedicated bit-patterns in the upper 2 bits of the size field, with the
last bit pattern being reserved for future algorithms. (e.g. 0x00 =
pglz, 0x01 = lz4, 0x10 = zstd, 0x11 = something else - see within for
details).- I don't really like the use of the phrase "custom compression". I
think the terminology needs to be rethought so that we just talk about
compression methods. Perhaps in certain contexts we need to specify
that we mean extensible compression methods or user-provided
compression methods or something like that, but I don't think the word
"custom" is very well-suited here. The main point of this shouldn't be
for every cluster in the universe to use a different approach to
compression, or to compress columns within a database in 47 different
ways, but rather to help us get out from under 'pglz'. Eventually we
probably want to change the default, but as the patch phrases things
now, that default would be a custom method, which is almost a
contradiction in terms.- Yet another possible approach to the on-disk format is to leave
varatt_external.va_extsize and varattrib_4b.rawsize untouched and
instead add new compression methods by adding new vartag_external
values. There's quite a lot of bit-space available there: we have a
whole byte, and we're currently only using 4 values. We could easily
add a half-dozen new possibilities there for new compression methods
without sweating the bit-space consumption. The main thing I don't
like about this is that it only seems like a useful way to provide for
out-of-line compression. Perhaps it could be generalized to allow for
inline compression as well, but it seems like it would take some
hacking.- One thing I really don't like about the patch is that it consumes a
bit from infomask2 for a new flag HEAP_HASCUSTOMCOMPRESSED. infomask
bits are at a premium, and there's been no real progress in the decade
plus that I've been hanging around here in clawing back any bit-space.
I think we really need to avoid burning our remaining bits for
anything other than a really critical need, and I don't think I
understand what the need is in this case. I might be missing
something, but I'd really strongly suggest looking for a way to get
rid of this. It also invents the concept of a TupleDesc flag, and the
flag invented is TD_ATTR_CUSTOM_COMPRESSED; I'm not sure I see why we
need that, either.- It seems like this kind of approach has a sort of built-in
circularity problem. It means that every place that might need to
detoast a datum needs to be able to access the pg_am catalog. I wonder
if that's actually true. For instance, consider logical decoding. I
guess that can do catalog lookups in general, but can it do them from
the places where detoasting is happening? Moreover, can it do them
with the right snapshot? Suppose we rewrite a table to change the
compression method, then drop the old compression method, then try to
decode a transaction that modified that table before those operations
were performed. As an even more extreme example, suppose we need to
open pg_am, and to do that we have to build a relcache entry for it,
and suppose the relevant pg_class entry had a relacl or reloptions
field that happened to be custom-compressed. Or equally suppose that
any of the various other tables we use when building a relcache entry
had the same kind of problem, especially those that have TOAST tables.
We could just disallow the use of non-default compressors in the
system catalogs, but the benefits mentioned in
/messages/by-id/5541614A.5030208@2ndquadrant.com seem too large to
ignore.- I think it would be awfully appealing if we could find some way of
dividing this great big patch into some somewhat smaller patches. For
example:Patch #1. Add syntax allowing a compression method to be specified,
but the only possible choice is pglz, and the PRESERVE stuff isn't
supported, and changing the value associated with an existing column
isn't supported, but we can add tab-completion support and stuff.Patch #2. Add a second built-in method, like gzip or lz4.
Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.Patch #5. Add support for compression methods via the AM interface.
Perhaps methods added in this manner are prohibited in system
catalogs. (This could also go before #4 or even before #3, but with a
noticeable hit to usability.)Patch #6 (new development). Add a contrib module using the facility
added in #5, perhaps with a slightly off-beat compressor like bzip2
that is more of a niche use case.I think that if the patch set were broken up this way, it would be a
lot easier to review and get committed. I think you could commit each
bit separately. I don't think you'd want to commit #1 unless you had a
sense that #2 was pretty close to done, and similarly for #5 and #6,
but that would still make things a lot easier than having one giant
monolithic patch, at least IMHO.There might be more to say here, but that's what I have got for now. I
hope it helps.
I have rebased the patch on the latest head and currently, broken into 3 parts.
v1-0001: As suggested by Robert, it provides the syntax support for
setting the compression method for a column while creating a table and
adding columns. However, we don't support changing the compression
method for the existing column. As part of this patch, there is only
one built-in compression method that can be set (pglz). In this, we
have one in-build am (pglz) and the compressed attributes will directly
store the oid of the AM. In this patch, I have removed the
pg_attr_compresion as we don't support changing the compression
for the existing column so we don't need to preserve the old
compressions.
v1-0002: Add another built-in compression method (zlib)
v1:0003: Remaining patch set (nothing is changed except rebase on the
current head, stabilizing check-world and 0001 and 0002 are pulled
out of this)
Next, I will be working on separating out the remaining patches as per
the suggestion by Robert.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v1-0001-Add-support-for-setting-the-compression-method.patchapplication/x-patch; name=v1-0001-Add-support-for-setting-the-compression-method.patchDownload
From 0d49553d0671f1b435ded46d1bb22088939e3633 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 4 Aug 2020 09:53:10 +0530
Subject: [PATCH v1 1/3] Add support for setting the compression method
Add syntax allowing a compression method to be specified. As of now there is only
one build-in compression method (pglz) which can be set while creating a table or
adding a new column. No option for altering the compression type for an existing
column.
---
.../expected/pg_stat_statements.out | 2 +-
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/catalogs.sgml | 6 +-
doc/src/sgml/ref/create_access_method.sgml | 10 +-
doc/src/sgml/ref/create_table.sgml | 11 +
doc/src/sgml/storage.sgml | 2 +-
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 4 +-
src/backend/access/brin/brin_tuple.c | 2 +
src/backend/access/common/detoast.c | 74 +++--
src/backend/access/common/heaptuple.c | 28 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 118 +++++---
src/backend/access/common/tupdesc.c | 20 ++
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/cm_pglz.c | 123 +++++++++
src/backend/access/compression/cmapi.c | 43 +++
src/backend/access/heap/heapam.c | 7 +-
src/backend/access/heap/heaptoast.c | 2 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 1 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/Makefile | 1 +
src/backend/commands/amcmds.c | 79 ++++++
src/backend/commands/compressioncmds.c | 85 ++++++
src/backend/commands/tablecmds.c | 69 ++++-
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 7 +
.../replication/logical/reorderbuffer.c | 3 +-
src/backend/utils/adt/expandedrecord.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/relcache.c | 4 +
src/bin/pg_dump/pg_backup.h | 2 +
src/bin/pg_dump/pg_dump.c | 53 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/pg_dump/pg_dumpall.c | 5 +
src/bin/pg_dump/pg_restore.c | 3 +
src/bin/psql/describe.c | 37 +++
src/bin/psql/tab-complete.c | 5 +-
src/include/access/cmapi.h | 77 ++++++
src/include/access/detoast.h | 26 +-
src/include/access/heaptoast.h | 1 +
src/include/access/htup_details.h | 9 +-
src/include/access/reloptions.h | 3 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 93 ++++++-
src/include/access/tupdesc.h | 7 +
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 7 +-
src/include/catalog/pg_proc.dat | 13 +
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 6 +
src/include/commands/event_trigger.h | 1 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 26 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 71 +++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 68 ++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/opr_sanity.out | 12 +
src/test/regress/expected/psql.out | 52 ++--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/create_cm.sql | 37 +++
src/test/regress/sql/opr_sanity.sql | 10 +
src/tools/pgindent/typedefs.list | 2 +
87 files changed, 1675 insertions(+), 627 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cm_pglz.c
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/backend/commands/compressioncmds.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/pg_stat_statements/expected/pg_stat_statements.out b/contrib/pg_stat_statements/expected/pg_stat_statements.out
index e0edb134f3..d15769f760 100644
--- a/contrib/pg_stat_statements/expected/pg_stat_statements.out
+++ b/contrib/pg_stat_statements/expected/pg_stat_statements.out
@@ -248,7 +248,7 @@ FROM pg_stat_statements ORDER BY query COLLATE "C";
wal_records = rows as wal_records_as_rows +| | | | |
FROM pg_stat_statements ORDER BY query COLLATE "C" | | | | |
SET pg_stat_statements.track_utility = FALSE | 1 | 0 | f | f | t
- UPDATE pgss_test SET b = $1 WHERE a > $2 | 1 | 3 | t | t | t
+ UPDATE pgss_test SET b = $1 WHERE a > $2 | 1 | 3 | t | t | f
(7 rows)
--
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index d79cd316b7..56b34319a3 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 26fda20d19..5ef5b69e81 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -659,9 +659,9 @@
The catalog <structname>pg_am</structname> stores information about
relation access methods. There is one row for each access method supported
by the system.
- Currently, only tables and indexes have access methods. The requirements for table
- and index access methods are discussed in detail in <xref linkend="tableam"/> and
- <xref linkend="indexam"/> respectively.
+ Currently, there are tables, indexes and compression access methods.
+ The requirements for table and index access methods are discussed in detail
+ in <xref linkend="tableam"/> and <xref linkend="indexam"/> respectively.
</para>
<table>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..3da27c8ce9 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,11 +77,13 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
+ is described in <xref linkend="tableam"/>, the index access method
API is described in <xref linkend="indexam"/>.
</para>
</listitem>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index dc688c415f..1a01c91b85 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,16 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Currently we can give the built-in
+ compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 3234adb639..ecc3359f64 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -393,7 +393,7 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
</para>
<para>
-The compression technique used for either in-line or out-of-line compressed
+The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
<filename>src/common/pg_lzcompress.c</filename> for the details.
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ca00737c53 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
- table tablesample transam
+SUBDIRS = brin common compression gin gist hash heap index nbtree \
+ rmgrdesc spgist table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 6cb7c26b39..801c009f1c 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -95,6 +95,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
int keyno;
int idxattno;
uint16 phony_infomask = 0;
+ uint16 phony_infomask2 = 0;
bits8 *phony_nullbitmap;
Size len,
hoff,
@@ -186,6 +187,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(char *) rettuple + hoff,
data_len,
&phony_infomask,
+ &phony_infomask2,
phony_nullbitmap);
/* done with these */
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..0a6592fbf9 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -232,7 +232,7 @@ detoast_attr_slice(struct varlena *attr,
* of a given length (after decompression).
*/
max_size = pglz_maximum_compressed_size(sliceoffset + slicelength,
- toast_pointer.va_extsize);
+ VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/*
* Fetch enough compressed slices (compressed marker will get set
@@ -333,7 +333,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
result = (struct varlena *) palloc(attrsize + VARHDRSZ);
@@ -394,7 +394,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) || 0 == sliceoffset);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
if (sliceoffset >= attrsize)
{
@@ -449,15 +449,32 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions cmoptions;
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ lookup_compression_am_options(hdr->cmid, &cmoptions);
+ result = cmoptions.amroutine->cmdecompress(&cmoptions, attr);
+ }
+ else
+ {
+ int rawsize;
+
+ result = (struct varlena *)
+ palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ TOAST_COMPRESS_SIZE(attr),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(attr), true);
+ if (rawsize < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ }
return result;
}
@@ -478,16 +495,33 @@ toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ if (VARATT_IS_CUSTOM_COMPRESSED(attr))
+ {
+ CompressionAmOptions cmoptions;
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ lookup_compression_am_options(hdr->cmid, &cmoptions);
+ if (cmoptions.amroutine->cmdecompress_slice)
+ result = cmoptions.amroutine->cmdecompress_slice(&cmoptions, attr,
+ slicelength);
+ else
+ result = cmoptions.amroutine->cmdecompress(&cmoptions, attr);
+ }
+ else
+ {
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
+ VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ elog(ERROR, "compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ }
- SET_VARSIZE(result, rawsize + VARHDRSZ);
return result;
}
@@ -570,7 +604,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index f89769f379..c03ede9b4e 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -59,6 +59,7 @@
#include "access/heaptoast.h"
#include "access/sysattr.h"
+#include "access/toast_internals.h"
#include "access/tupdesc_details.h"
#include "executor/tuptable.h"
#include "utils/expandeddatum.h"
@@ -177,6 +178,7 @@ fill_val(Form_pg_attribute att,
int *bitmask,
char **dataP,
uint16 *infomask,
+ uint16 *infomask2,
Datum datum,
bool isnull)
{
@@ -207,6 +209,9 @@ fill_val(Form_pg_attribute att,
**bit |= *bitmask;
}
+ if (OidIsValid(att->attcompression))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+
/*
* XXX we use the att_align macros on the pointer value itself, not on an
* offset. This is a bit of a hack.
@@ -245,6 +250,15 @@ fill_val(Form_pg_attribute att,
/* no alignment, since it's short by definition */
data_length = VARSIZE_EXTERNAL(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_EXTERNAL_ONDISK(val))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, val);
+ if (VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
+ }
}
}
else if (VARATT_IS_SHORT(val))
@@ -268,6 +282,9 @@ fill_val(Form_pg_attribute att,
att->attalign);
data_length = VARSIZE(val);
memcpy(data, val, data_length);
+
+ if (VARATT_IS_CUSTOM_COMPRESSED(val))
+ *infomask2 |= HEAP_HASCUSTOMCOMPRESSED;
}
}
else if (att->attlen == -2)
@@ -304,7 +321,7 @@ void
heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit)
+ uint16 *infomask, uint16 *infomask2, bits8 *bit)
{
bits8 *bitP;
int bitmask;
@@ -328,6 +345,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
}
*infomask &= ~(HEAP_HASNULL | HEAP_HASVARWIDTH | HEAP_HASEXTERNAL);
+ *infomask2 &= ~HEAP_HASCUSTOMCOMPRESSED;
for (i = 0; i < numberOfAttributes; i++)
{
@@ -338,6 +356,7 @@ heap_fill_tuple(TupleDesc tupleDesc,
&bitmask,
&data,
infomask,
+ infomask2,
values ? values[i] : PointerGetDatum(NULL),
isnull ? isnull[i] : true);
}
@@ -752,6 +771,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
int bitMask = 0;
char *targetData;
uint16 *infoMask;
+ uint16 *infoMask2;
Assert((targetHeapTuple && !targetMinimalTuple)
|| (!targetHeapTuple && targetMinimalTuple));
@@ -864,6 +884,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(HeapTupleHeaderData, t_bits));
targetData = (char *) (*targetHeapTuple)->t_data + hoff;
infoMask = &(targetTHeader->t_infomask);
+ infoMask2 = &(targetTHeader->t_infomask2);
}
else
{
@@ -882,6 +903,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
+ offsetof(MinimalTupleData, t_bits));
targetData = (char *) *targetMinimalTuple + hoff;
infoMask = &((*targetMinimalTuple)->t_infomask);
+ infoMask2 = &((*targetMinimalTuple)->t_infomask2);
}
if (targetNullLen > 0)
@@ -934,6 +956,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
attrmiss[attnum].am_value,
false);
}
@@ -944,6 +967,7 @@ expand_tuple(HeapTuple *targetHeapTuple,
&bitMask,
&targetData,
infoMask,
+ infoMask2,
(Datum) 0,
true);
}
@@ -1093,6 +1117,7 @@ heap_form_tuple(TupleDesc tupleDescriptor,
(char *) td + hoff,
data_len,
&td->t_infomask,
+ &td->t_infomask2,
(hasnull ? td->t_bits : NULL));
return tuple;
@@ -1415,6 +1440,7 @@ heap_form_minimal_tuple(TupleDesc tupleDescriptor,
(char *) tuple + hoff,
data_len,
&tuple->t_infomask,
+ &tuple->t_infomask2,
(hasnull ? tuple->t_bits : NULL));
return tuple;
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..07413532af 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -57,6 +57,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
unsigned short infomask = 0;
bool hasnull = false;
uint16 tupmask = 0;
+ uint16 tupmask2 = 0;
int numberOfAttributes = tupleDescriptor->natts;
#ifdef TOAST_INDEX_HACK
@@ -103,7 +104,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i], InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
@@ -153,6 +154,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
(char *) tp + hoff,
data_size,
&tupmask,
+ &tupmask2,
(hasnull ? (bits8 *) tp + sizeof(IndexTupleData) : NULL));
#ifdef TOAST_INDEX_HACK
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..38d273e8bd 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,23 +13,45 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/heaptoast.h"
#include "access/table.h"
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/fmgroids.h"
+#include "utils/hsearch.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_CMID(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,54 +66,47 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid amoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionAmOptions cmoptions;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!OidIsValid(amoid))
+ amoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ lookup_compression_am_options(amoid, &cmoptions);
+ tmp = cmoptions.amroutine->cmcompress(&cmoptions, (const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression function reports
+ * success, because it might be satisfied with having saved as little as
+ * one byte in the compressed data --- which could turn into a net loss
+ * once you consider header and alignment padding. Worst case, the
+ * compressed format might require three padding bytes (plus header, which
+ * is included in VARSIZE(tmp)), whereas the uncompressed format would
+ * take only one header byte and no padding if the value is short enough.
+ * So we insist on a savings of more than 2 bytes to ensure we have a
+ * gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, amoid, valsize);
return PointerGetDatum(tmp);
}
- else
- {
- /* incompressible data */
- pfree(tmp);
- return PointerGetDatum(NULL);
- }
+
+ /* incompressible data */
+ pfree(tmp);
+ return PointerGetDatum(NULL);
}
/* ----------
@@ -152,19 +167,19 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo is the actual size of the data payload in the toast records.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -172,7 +187,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
+ if (VARATT_IS_CUSTOM_COMPRESSED(dval))
+ VARATT_EXTERNAL_SET_CUSTOM(toast_pointer);
+
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -181,7 +199,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo; /* no flags */
}
/*
@@ -630,3 +648,23 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+
+/* ----------
+ * lookup_compression_am_options
+ *
+ * Get compression handler routine.
+ */
+void
+lookup_compression_am_options(Oid acoid, CompressionAmOptions *result)
+{
+ regproc amhandler;
+
+ amhandler = get_am_handler_oid(acoid, AMTYPE_COMPRESSION, false);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ result->acstate = result->amroutine->cminitstate ?
+ result->amroutine->cminitstate(acoid) : NULL;
+
+ if (!result->amroutine)
+ /* should not happen but need to check if something goes wrong */
+ elog(ERROR, "compression access method routine is NULL");
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..3b2e434ca5 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/reloptions.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -72,6 +73,7 @@ CreateTemplateTupleDesc(int natts)
desc->constr = NULL;
desc->tdtypeid = RECORDOID;
desc->tdtypmod = -1;
+ desc->tdflags = 0;
desc->tdrefcount = -1; /* assume not reference-counted */
return desc;
@@ -94,8 +96,17 @@ CreateTupleDesc(int natts, Form_pg_attribute *attrs)
desc = CreateTemplateTupleDesc(natts);
for (i = 0; i < natts; ++i)
+ {
memcpy(TupleDescAttr(desc, i), attrs[i], ATTRIBUTE_FIXED_PART_SIZE);
+ /*
+ * If even one of attributes is compressed we save information about
+ * it to TupleDesc flags
+ */
+ if (OidIsValid(attrs[i]->attcompression))
+ desc->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+ }
+
return desc;
}
@@ -137,6 +148,7 @@ CreateTupleDescCopy(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -217,6 +229,7 @@ CreateTupleDescCopyConstr(TupleDesc tupdesc)
/* We can copy the tuple type identification, too */
desc->tdtypeid = tupdesc->tdtypeid;
desc->tdtypmod = tupdesc->tdtypmod;
+ desc->tdflags = tupdesc->tdflags;
return desc;
}
@@ -304,6 +317,7 @@ TupleDescCopyEntry(TupleDesc dst, AttrNumber dstAttno,
dstAtt->atthasmissing = false;
dstAtt->attidentity = '\0';
dstAtt->attgenerated = '\0';
+ dstAtt->attcompression = InvalidOid;
}
/*
@@ -418,6 +432,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (tupdesc1->tdtypeid != tupdesc2->tdtypeid)
return false;
+ if (tupdesc1->tdflags != tupdesc2->tdflags)
+ return false;
for (i = 0; i < tupdesc1->natts; i++)
{
@@ -470,6 +486,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -557,6 +575,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
}
else if (tupdesc2->constr != NULL)
return false;
+
return true;
}
@@ -663,6 +682,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..14286920d3
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..7541b777ef
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(Oid acoid)
+{
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+
+ return (void *) strategy;
+}
+
+static struct varlena *
+pglz_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) cmoptions->acstate;
+
+ Assert(strategy != NULL);
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_CUSTOM_COMPRESSED);
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED,
+ strategy);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+pglz_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ return result;
+}
+
+static struct varlena *
+pglz_cmdecompress_slice(CompressionAmOptions *cmoptions, const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_CUSTOM_COMPRESSED,
+ VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cminitstate = pglz_cminitstate;
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..3cdce17276
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/*
+ * InvokeCompressionAmHandler - call the specified access method handler routine to get
+ * its CompressionAmRoutine struct, which will be palloc'd in the caller's context.
+ */
+CompressionAmRoutine *
+InvokeCompressionAmHandler(Oid amhandler)
+{
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index f75e1cf0e7..9d52fa1131 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2078,7 +2078,10 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
+ else if (HeapTupleHasExternal(tup)
+ || RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
+ || HeapTupleHasCustomCompressed(tup)
+ || tup->t_len > TOAST_TUPLE_THRESHOLD)
return heap_toast_insert_or_update(relation, tup, NULL, options);
else
return tup;
@@ -3404,6 +3407,8 @@ l2:
else
need_toast = (HeapTupleHasExternal(&oldtup) ||
HeapTupleHasExternal(newtup) ||
+ RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED ||
+ HeapTupleHasCustomCompressed(newtup) ||
newtup->t_len > TOAST_TUPLE_THRESHOLD);
pagefree = PageGetHeapFreeSpace(page);
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 584f101dd9..e7fa3f6683 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -324,6 +324,7 @@ heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
((ttc.ttc_flags & TOAST_HAS_NULLS) != 0) ?
new_data->t_bits : NULL);
}
@@ -533,6 +534,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
(char *) new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
+ &(new_data->t_infomask2),
has_nulls ? new_data->t_bits : NULL);
/*
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 45b7efbe46..b84f41b469 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -755,6 +755,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..b67d3e5666 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -180,7 +180,7 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
};
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index f2ca686397..a7d64b8356 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -786,6 +786,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = ObjectIdGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1712,6 +1713,9 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ /* Unset attribute compression Oid */
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1be27eec52..ed83e3d732 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -373,6 +373,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
@@ -451,6 +452,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 6f05ee715b..a084377a53 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -155,6 +155,70 @@ get_am_type_oid(const char *amname, char amtype, bool missing_ok)
return oid;
}
+/*
+ * get_am_handler_oid - return access method handler Oid with additional
+ * type checking.
+ *
+ * If noerror is false, throw an error if access method not found or its
+ * type not equal to specified type, or its handler is not valid.
+ *
+ * If amtype is not '\0', an error is raised if the AM found is not of the
+ * given type.
+ */
+regproc
+get_am_handler_oid(Oid amOid, char amtype, bool noerror)
+{
+ HeapTuple tup;
+ Form_pg_am amform;
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ tup = SearchSysCache1(AMOID, ObjectIdGetDatum(amOid));
+ if (!HeapTupleIsValid(tup))
+ {
+ if (noerror)
+ return InvalidOid;
+
+ elog(ERROR, "cache lookup failed for access method %u", amOid);
+ }
+
+ amform = (Form_pg_am) GETSTRUCT(tup);
+ amhandler = amform->amhandler;
+
+ if (amtype != '\0' && amform->amtype != amtype)
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname),
+ get_am_type_string(amtype))));
+ }
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ {
+ if (noerror)
+ {
+ ReleaseSysCache(tup);
+ return InvalidOid;
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+ }
+
+ ReleaseSysCache(tup);
+ return amhandler;
+}
+
/*
* get_index_am_oid - given an access method name, look up its OID
* and verify it corresponds to an index AM.
@@ -175,6 +239,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
@@ -217,6 +291,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -254,6 +330,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..3d56f7399f
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,85 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "miscadmin.h"
+
+#include "access/cmapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/xact.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_collation_d.h"
+#include "catalog/pg_depend.h"
+#include "catalog/pg_proc_d.h"
+#include "catalog/pg_type_d.h"
+#include "commands/defrem.h"
+#include "parser/parse_func.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+#include "utils/rel.h"
+#include "utils/syscache.h"
+#include "utils/snapmgr.h"
+
+/*
+ * Link compression with an attribute.
+ *
+ * When compression is not specified returns default attribute compression.
+ * It is possible case for CREATE TABLE and ADD COLUMN commands
+ * where COMPRESSION syntax is optional.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
+
+/*
+ * Get compression name by attribute compression Oid.
+ */
+char *
+GetCompressionName(Oid amoid)
+{
+ HeapTuple tuple;
+ Form_pg_am amform;
+ char *amname;
+
+ if (!OidIsValid(amoid))
+ return NULL;
+
+ /* Get handler function OID for the access method */
+ tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", amoid);
+
+ amform = (Form_pg_am) GETSTRUCT(tuple);
+ amname = pstrdup(NameStr(amform->amname));
+
+ ReleaseSysCache(tuple);
+
+ return amname;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cd989c95e5..d71bfdb7c6 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -41,6 +42,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -88,6 +90,7 @@
#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -164,6 +167,7 @@ typedef struct AlteredTableInfo
List *newvals; /* List of NewColumnValue */
List *afterStmts; /* List of utility command parsetrees */
bool verify_new_notnull; /* T if we should recheck NOT NULL */
+ bool new_notnull; /* T if we added new NOT NULL constraints */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
bool chgPersistence; /* T if SET LOGGED/UNLOGGED is used */
@@ -800,6 +804,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
* We can set the atthasdef flags now in the tuple descriptor; this just
* saves StoreAttrDefault from having to do an immediate update of the
* pg_attribute rows.
+ *
+ * Also create attribute compression records if needed.
*/
rawDefaults = NIL;
cookedDefaults = NIL;
@@ -850,6 +856,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2371,6 +2384,20 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = GetCompressionName(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2405,6 +2432,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionName(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2614,6 +2642,18 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -4852,7 +4892,9 @@ ATRewriteTables(AlterTableStmt *parsetree, List **wqueue, LOCKMODE lockmode,
/*
* We only need to rewrite the table if at least one column needs to
- * be recomputed, or we are changing its persistence.
+ * be recomputed, we are adding/removing the OID column, we are
+ * changing its persistence, or we are changing compression type of a
+ * column without preserving old ones.
*
* There are two reasons for requiring a rewrite when changing
* persistence: on one hand, we need to ensure that the buffers
@@ -6130,6 +6172,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7489,6 +7540,10 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) && (newstorage != 'p'))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11566,6 +11621,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 89c409de66..7d71f4fb99 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e3f33c40be..583a7414f0 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2589,6 +2589,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index d85ca9f7c5..92452c5fd6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3808,6 +3808,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..a880a95ac6 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2834,6 +2834,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dbb47d4982..39f7be1854 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -599,6 +599,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -634,9 +636,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3374,11 +3376,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3387,8 +3390,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3433,6 +3436,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3662,6 +3673,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -5204,6 +5216,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
@@ -15059,6 +15072,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 25abc544fc..0638e56cd4 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1127,6 +1127,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionName(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..fc74380469 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
@@ -4238,7 +4239,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index ec12ec54fc..4b783580af 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -809,6 +809,7 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
(char *) tuphdr + erh->hoff,
erh->data_len,
&tuphdr->t_infomask,
+ &tuphdr->t_infomask2,
(erh->hasnull ? tuphdr->t_bits : NULL));
}
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..42ccb3bff6 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
diff --git a/src/backend/utils/cache/relcache.c b/src/backend/utils/cache/relcache.c
index a2453cf1f4..f8ae47fd5a 100644
--- a/src/backend/utils/cache/relcache.c
+++ b/src/backend/utils/cache/relcache.c
@@ -584,6 +584,10 @@ RelationBuildTupleDesc(Relation relation)
ndef++;
}
+ /* mark tupledesc as it contains attributes with custom compression */
+ if (attp->attcompression)
+ relation->rd_att->tdflags |= TD_ATTR_CUSTOM_COMPRESSED;
+
/* Likewise for a missing value */
if (attp->atthasmissing)
{
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..1435290e55 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -78,6 +78,7 @@ typedef struct _restoreOptions
int no_publications; /* Skip publication entries */
int no_security_labels; /* Skip security label entries */
int no_subscriptions; /* Skip subscription entries */
+ int no_compression_methods; /* Skip compression methods */
int strict_names;
const char *filename;
@@ -150,6 +151,7 @@ typedef struct _dumpOptions
int no_security_labels;
int no_publications;
int no_subscriptions;
+ int no_compression_methods;
int no_synchronized_snapshots;
int no_unlogged_table_data;
int serializable_deferrable;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 9c8436dde6..2eeeea306f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -39,6 +39,7 @@
#endif
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
@@ -382,6 +383,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -887,6 +889,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8435,6 +8439,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8520,6 +8525,16 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8538,7 +8553,13 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am c "
+ "ON a.attcompression = c.oid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8565,6 +8586,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8593,6 +8615,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attrdefs[j] = NULL; /* fix below */
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -12840,6 +12864,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
@@ -15637,6 +15664,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_custom_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15661,6 +15689,13 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ has_custom_compression = (tbinfo->attcmnames[j] &&
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0)));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15696,6 +15731,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_custom_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index da97b731b1..f9c87e6c16 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method names */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/pg_dump/pg_dumpall.c b/src/bin/pg_dump/pg_dumpall.c
index 2c82b39af0..96d2c47359 100644
--- a/src/bin/pg_dump/pg_dumpall.c
+++ b/src/bin/pg_dump/pg_dumpall.c
@@ -76,6 +76,7 @@ static int no_comments = 0;
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+static int no_compression_methods = 0;
static int no_unlogged_table_data = 0;
static int no_role_passwords = 0;
static int server_version;
@@ -142,6 +143,7 @@ main(int argc, char *argv[])
{"no-role-passwords", no_argument, &no_role_passwords, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{"no-sync", no_argument, NULL, 4},
{"no-unlogged-table-data", no_argument, &no_unlogged_table_data, 1},
{"on-conflict-do-nothing", no_argument, &on_conflict_do_nothing, 1},
@@ -427,6 +429,8 @@ main(int argc, char *argv[])
appendPQExpBufferStr(pgdumpopts, " --no-security-labels");
if (no_subscriptions)
appendPQExpBufferStr(pgdumpopts, " --no-subscriptions");
+ if (no_compression_methods)
+ appendPQExpBufferStr(pgdumpopts, " --no-compression-methods");
if (no_unlogged_table_data)
appendPQExpBufferStr(pgdumpopts, " --no-unlogged-table-data");
if (on_conflict_do_nothing)
@@ -648,6 +652,7 @@ help(void)
printf(_(" --no-role-passwords do not dump passwords for roles\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-sync do not wait for changes to be written safely to disk\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
diff --git a/src/bin/pg_dump/pg_restore.c b/src/bin/pg_dump/pg_restore.c
index 544ae3bc5c..1e92091c2b 100644
--- a/src/bin/pg_dump/pg_restore.c
+++ b/src/bin/pg_dump/pg_restore.c
@@ -71,6 +71,7 @@ main(int argc, char **argv)
static int no_publications = 0;
static int no_security_labels = 0;
static int no_subscriptions = 0;
+ static int no_compression_methods = 0;
static int strict_names = 0;
struct option cmdopts[] = {
@@ -120,6 +121,7 @@ main(int argc, char **argv)
{"no-publications", no_argument, &no_publications, 1},
{"no-security-labels", no_argument, &no_security_labels, 1},
{"no-subscriptions", no_argument, &no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &no_compression_methods, 1},
{NULL, 0, NULL, 0}
};
@@ -376,6 +378,7 @@ main(int argc, char **argv)
opts->no_publications = no_publications;
opts->no_security_labels = no_security_labels;
opts->no_subscriptions = no_subscriptions;
+ opts->no_compression_methods = no_compression_methods;
if (if_exists && !opts->dropSchema)
{
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index d81f1575bf..5a511865ec 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1471,6 +1471,7 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
+ attcompression_col = -1,
attdescr_col = -1;
int numrows;
struct
@@ -1888,6 +1889,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.amname "
+ " FROM pg_catalog.pg_am c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcmname");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2014,6 +2028,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2089,6 +2105,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index f41785f11c..08bd604f12 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..c9b074deea
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,77 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression AM.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
+#define DefaultCompressionOid (PGLZ_COMPRESSION_AM_OID)
+
+typedef struct CompressionAmRoutine CompressionAmRoutine;
+
+/*
+ * CompressionAmOptions contains all information needed to compress varlena.
+ *
+ * For optimization purposes it will be created once for each attribute
+ * compression and stored in cache, until its renewal on global cache reset,
+ * or until deletion of related attribute compression.
+ */
+typedef struct CompressionAmOptions
+{
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+
+typedef struct varlena *(*cmcompress_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (CompressionAmOptions *cmoptions, const struct varlena *value,
+ int32 slicelength);
+typedef void *(*cminitstate_function) (Oid acoid);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/cmapi.c */
+extern CompressionAmRoutine *InvokeCompressionAmHandler(Oid amhandler);
+extern List *GetAttrCompressionOptions(Oid acoid);
+extern Oid GetAttrCompressionAmOid(Oid acoid);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..31fc6b30b3 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -12,30 +12,8 @@
#ifndef DETOAST_H
#define DETOAST_H
-/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing extsize (the actual length of the external data) to rawsize
- * (the original uncompressed datum's size). The latter includes VARHDRSZ
- * overhead, the former doesn't. We never use compression unless it actually
- * saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
-
-/*
- * Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
- * into a local "struct varatt_external" toast pointer. This should be
- * just a memcpy, but some versions of gcc seem to produce broken code
- * that assumes the datum contents are aligned. Introducing an explicit
- * intermediate "varattrib_1b_e *" variable seems to fix it.
- */
-#define VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr) \
-do { \
- varattrib_1b_e *attre = (varattrib_1b_e *) (attr); \
- Assert(VARATT_IS_EXTERNAL(attre)); \
- Assert(VARSIZE_EXTERNAL(attre) == sizeof(toast_pointer) + VARHDRSZ_EXTERNAL); \
- memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
-} while (0)
+#include "access/attnum.h"
+#include "nodes/pg_list.h"
/* Size of an EXTERNAL datum that contains a standard TOAST pointer */
#define TOAST_POINTER_SIZE (VARHDRSZ_EXTERNAL + sizeof(varatt_external))
diff --git a/src/include/access/heaptoast.h b/src/include/access/heaptoast.h
index 2635831910..7169254199 100644
--- a/src/include/access/heaptoast.h
+++ b/src/include/access/heaptoast.h
@@ -15,6 +15,7 @@
#include "access/htup_details.h"
#include "storage/lockdefs.h"
+#include "utils/hsearch.h"
#include "utils/relcache.h"
/*
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index aebb1082ca..4b46ea7073 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -274,7 +274,9 @@ struct HeapTupleHeaderData
* information stored in t_infomask2:
*/
#define HEAP_NATTS_MASK 0x07FF /* 11 bits for number of attributes */
-/* bits 0x1800 are available */
+/* bit 0x800 is available */
+#define HEAP_HASCUSTOMCOMPRESSED 0x1000 /* tuple contains custom compressed
+ * varlena(s) */
#define HEAP_KEYS_UPDATED 0x2000 /* tuple was updated and key cols
* modified, or tuple deleted */
#define HEAP_HOT_UPDATED 0x4000 /* tuple was HOT-updated */
@@ -673,6 +675,9 @@ struct MinimalTupleData
#define HeapTupleHasExternal(tuple) \
(((tuple)->t_data->t_infomask & HEAP_HASEXTERNAL) != 0)
+#define HeapTupleHasCustomCompressed(tuple) \
+ (((tuple)->t_data->t_infomask2 & HEAP_HASCUSTOMCOMPRESSED) != 0)
+
#define HeapTupleIsHotUpdated(tuple) \
HeapTupleHeaderIsHotUpdated((tuple)->t_data)
@@ -779,7 +784,7 @@ extern Size heap_compute_data_size(TupleDesc tupleDesc,
extern void heap_fill_tuple(TupleDesc tupleDesc,
Datum *values, bool *isnull,
char *data, Size data_size,
- uint16 *infomask, bits8 *bit);
+ uint16 *infomask, uint16 *infomask2, bits8 *bit);
extern bool heap_attisnull(HeapTuple tup, int attnum, TupleDesc tupleDesc);
extern Datum nocachegetattr(HeapTuple tup, int attnum,
TupleDesc att);
diff --git a/src/include/access/reloptions.h b/src/include/access/reloptions.h
index 5964438cb0..abd6ce146c 100644
--- a/src/include/access/reloptions.h
+++ b/src/include/access/reloptions.h
@@ -233,6 +233,9 @@ extern void *build_reloptions(Datum reloptions, bool validate,
extern void *build_local_reloptions(local_relopts *relopts, Datum options,
bool validate);
+extern char *formatRelOptions(List *options);
+extern List *untransformRelOptions(Datum options);
+
extern bytea *default_reloptions(Datum reloptions, bool validate,
relopt_kind kind);
extern bytea *heap_reloptions(char relkind, Datum reloptions, bool validate);
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..4e932bc067 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..94e11e583f 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -13,6 +13,7 @@
#define TOAST_INTERNALS_H
#include "storage/lockdefs.h"
+#include "utils/hsearch.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,86 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* flags (2 bits) and rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* flags (2 high bits) and rawsize */
+ Oid cmid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_COMPRESS_HDRSZ_CUSTOM ((int32) sizeof(toast_compress_header_custom))
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len) | (0x02 << 30); \
+} while (0)
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmid = (oid))
+
+#define VARATT_EXTERNAL_SET_CUSTOM(toast_pointer) \
+do { \
+ ((toast_pointer).va_extinfo |= (1 << 31)); \
+ ((toast_pointer).va_extinfo &= ~(1 << 30)); \
+} while (0)
+
+/*
+ * Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
+ * into a local "struct varatt_external" toast pointer. This should be
+ * just a memcpy, but some versions of gcc seem to produce broken code
+ * that assumes the datum contents are aligned. Introducing an explicit
+ * intermediate "varattrib_1b_e *" variable seems to fix it.
+ */
+#define VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr) \
+ do \
+ { \
+ varattrib_1b_e *attre = (varattrib_1b_e *)(attr); \
+ Assert(VARATT_IS_EXTERNAL(attre)); \
+ Assert(VARSIZE_EXTERNAL(attre) == sizeof(toast_pointer) + VARHDRSZ_EXTERNAL); \
+ memcpy(&(toast_pointer), VARDATA_EXTERNAL(attre), sizeof(toast_pointer)); \
+ } while (0)
+
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and optional flags
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & 0x3FFFFFFF)
+#define VARATT_EXTERNAL_IS_CUSTOM_COMPRESSED(toast_pointer) \
+ (((toast_pointer).va_extinfo >> 30) == 0x02)
+
+/*
+ * Testing whether an externally-stored value is compressed now requires
+ * comparing extsize (the actual length of the external data) to rawsize
+ * (the original uncompressed datum's size). The latter includes VARHDRSZ
+ * overhead, the former doesn't. We never use compression unless it actually
+ * saves space, so we expect either equality or less-than.
+ */
+#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < (toast_pointer).va_rawsize - VARHDRSZ)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid acoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +117,20 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * lookup_compression_am_options -
+ *
+ * Return cached CompressionAmOptions for specified attribute compression.
+ */
+extern void lookup_compression_am_options(Oid acoid,
+ CompressionAmOptions *result);
+
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, Oid cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/access/tupdesc.h b/src/include/access/tupdesc.h
index d17af13ee3..cc8536f105 100644
--- a/src/include/access/tupdesc.h
+++ b/src/include/access/tupdesc.h
@@ -14,7 +14,9 @@
#ifndef TUPDESC_H
#define TUPDESC_H
+#include "postgres.h"
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
@@ -45,6 +47,10 @@ typedef struct TupleConstr
bool has_generated_stored;
} TupleConstr;
+/* tupledesc flags */
+#define TD_ATTR_CUSTOM_COMPRESSED 0x01 /* is TupleDesc contain attributes
+ * with custom compression? */
+
/*
* This struct is passed around within the backend to describe the structure
* of tuples. For tuples coming from on-disk relations, the information is
@@ -81,6 +87,7 @@ typedef struct TupleDescData
int natts; /* number of attributes in the tuple */
Oid tdtypeid; /* composite type ID for tuple type */
int32 tdtypmod; /* typmod for tuple type */
+ char tdflags; /* tuple additional flags */
int tdrefcount; /* reference count, or -1 if not counting */
TupleConstr *constr; /* constraints, or NULL if none */
/* attrs[N] is the description of Attribute Number N+1 */
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 0f051277a6..877aa3d7e5 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,8 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 63c03c4324..4d9d965b8d 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -54,6 +54,7 @@ typedef FormData_pg_am *Form_pg_am;
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..fe6dc48a6e 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options or InvalidOid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -180,10 +183,10 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* ATTRIBUTE_FIXED_PART_SIZE is the size of the fixed-layout,
* guaranteed-not-null part of a pg_attribute row. This is in fact as much
* of the row as gets copied into tuple descriptors, so don't expect you
- * can access fields beyond attcollation except in a real tuple!
+ * can access fields beyond attcompression except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 082a11f270..d8848f58c9 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -945,6 +945,12 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+# Compression access method handlers
+{ oid => '4388', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7182,6 +7188,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1c05ebce20 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+{ oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index c26a102b17..8d0de0babe 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -142,8 +142,14 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
+
+/* commands/compressioncmds.c */
+extern char *GetCompressionName(Oid acoid);
+extern Oid GetAttributeCompression(Form_pg_attribute attr, char *compression);
/* support routines in commands/define.c */
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..1800fd8d0c 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 381d84b4e4..35df26f978 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -510,6 +510,7 @@ typedef enum NodeTag
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
+ T_CompressionAmRoutine, /* in access/cmapi.h */
T_TsmRoutine, /* in access/tsmapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 151bcdb7ef..8182f8edba 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -684,6 +685,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 08f22ce211..98c4a78e0a 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..eef7c1d880 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * flags */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,9 +146,18 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression options */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
@@ -280,8 +290,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
+
+#define VARHDRSZ_CUSTOM_COMPRESSED \
+ (offsetof(varattrib_4b, va_custom_compressed.va_data))
/* Externally visible macros */
@@ -310,6 +326,8 @@ typedef struct
#define VARDATA_EXTERNAL(PTR) VARDATA_1B_E(PTR)
#define VARATT_IS_COMPRESSED(PTR) VARATT_IS_4B_C(PTR)
+#define VARATT_IS_CUSTOM_COMPRESSED(PTR) (VARATT_IS_4B_C(PTR) && \
+ (VARFLAGS_4B_C(PTR) == 0x02))
#define VARATT_IS_EXTERNAL(PTR) VARATT_IS_1B_E(PTR)
#define VARATT_IS_EXTERNAL_ONDISK(PTR) \
(VARATT_IS_EXTERNAL(PTR) && VARTAG_EXTERNAL(PTR) == VARTAG_ONDISK)
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 6f90eae2f8..c3c58a5e9d 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e40287d25a..2d3656eec3 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..84adf0c459
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,71 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1c72f23bc9..f0c0d6e247 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1021,21 +1021,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1043,11 +1043,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1076,46 +1076,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1147,11 +1147,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1160,10 +1160,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1173,10 +1173,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 655e8e41dd..6a6b7ee198 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -279,32 +279,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -318,12 +318,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -333,12 +333,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -352,11 +352,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 7ac9df767f..326cfeaa2a 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/opr_sanity.out b/src/test/regress/expected/opr_sanity.out
index 1b3c146e4c..f7d062353b 100644
--- a/src/test/regress/expected/opr_sanity.out
+++ b/src/test/regress/expected/opr_sanity.out
@@ -1937,6 +1937,18 @@ WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND
-----+--------+-----+---------
(0 rows)
+-- Check for compression amhandler functions with the wrong signature
+SELECT p1.oid, p1.amname, p2.oid, p2.proname
+FROM pg_am AS p1, pg_proc AS p2
+WHERE p2.oid = p1.amhandler AND p1.amtype = 'c' AND
+ (p2.prorettype != 'compression_am_handler'::regtype
+ OR p2.proretset
+ OR p2.pronargs != 1
+ OR p2.proargtypes[0] != 'internal'::regtype);
+ oid | amname | oid | proname
+-----+--------+-----+---------
+(0 rows)
+
-- **************** pg_amop ****************
-- Look for illegal values in pg_amop fields
SELECT p1.amopfamily, p1.amopstrategy
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 555d464f91..2502b23793 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2801,34 +2801,34 @@ CREATE ACCESS METHOD heap_psql TYPE TABLE HANDLER heap_tableam_handler;
CREATE TABLE tbl_heap_psql(f1 int, f2 char(100)) using heap_psql;
CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
\d+ tbl_heap_psql
- Table "public.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "public.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "public.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "public.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "public.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
\set HIDE_TABLEAM on
@@ -4843,8 +4843,9 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ pglz |
spgist | Index
-(8 rows)
+(9 rows)
\dA *
List of access methods
@@ -4857,8 +4858,9 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ pglz |
spgist | Index
-(8 rows)
+(9 rows)
\dA h*
List of access methods
@@ -4893,8 +4895,9 @@ List of access methods
hash | Index | hashhandler | hash index access method
heap | Table | heap_tableam_handler | heap table access method
heap2 | Table | heap_tableam_handler |
+ pglz | | pglzhandler | pglz compression access method
spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+(9 rows)
\dA+ *
List of access methods
@@ -4907,8 +4910,9 @@ List of access methods
hash | Index | hashhandler | hash index access method
heap | Table | heap_tableam_handler | heap table access method
heap2 | Table | heap_tableam_handler |
+ pglz | | pglzhandler | pglz compression access method
spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+(9 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 601734a6f1..e6f39d72f8 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -2994,11 +2994,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3014,11 +3014,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3045,11 +3045,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 979d926119..e3469e562f 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -81,6 +81,7 @@ test: drop_if_exists
test: updatable_views
test: roleattributes
test: create_am
+test: create_cm
test: hash_func
test: errors
test: sanity_check
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..a9ddccdc8f
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,37 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+DROP TABLE cmmove1, cmmove2, cmmove3;
\ No newline at end of file
diff --git a/src/test/regress/sql/opr_sanity.sql b/src/test/regress/sql/opr_sanity.sql
index 7a9180b081..b1aed2e2d1 100644
--- a/src/test/regress/sql/opr_sanity.sql
+++ b/src/test/regress/sql/opr_sanity.sql
@@ -1250,6 +1250,16 @@ WHERE p2.oid = p1.amhandler AND p1.amtype = 's' AND
OR p2.pronargs != 1
OR p2.proargtypes[0] != 'internal'::regtype);
+-- Check for compression amhandler functions with the wrong signature
+
+SELECT p1.oid, p1.amname, p2.oid, p2.proname
+FROM pg_am AS p1, pg_proc AS p2
+WHERE p2.oid = p1.amhandler AND p1.amtype = 'c' AND
+ (p2.prorettype != 'compression_am_handler'::regtype
+ OR p2.proretset
+ OR p2.pronargs != 1
+ OR p2.proargtypes[0] != 'internal'::regtype);
+
-- **************** pg_amop ****************
-- Look for illegal values in pg_amop fields
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index b4948ac675..1953697af9 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -394,6 +394,8 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmOptions
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
v1-0003-Custom-compression-methods.patchapplication/x-patch; name=v1-0003-Custom-compression-methods.patchDownload
From 243f62308c18e6ca79d74244d00f8a81d512fc5d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 12 Aug 2020 14:35:38 +0530
Subject: [PATCH v1 3/3] Custom compression methods
Allow to create custom compression methods. This also allow to change
the compression methods for the existing column.
---
doc/src/sgml/catalogs.sgml | 16 +-
doc/src/sgml/compression-am.sgml | 178 ++++++
doc/src/sgml/filelist.sgml | 1 +
doc/src/sgml/indexam.sgml | 2 +-
doc/src/sgml/postgres.sgml | 1 +
doc/src/sgml/ref/alter_table.sgml | 18 +
doc/src/sgml/ref/create_access_method.sgml | 3 +-
doc/src/sgml/ref/create_table.sgml | 10 +-
doc/src/sgml/storage.sgml | 4 +-
src/backend/access/common/detoast.c | 21 +-
src/backend/access/common/reloptions.c | 88 +++
src/backend/access/common/toast_internals.c | 167 +++++-
src/backend/access/compression/cm_pglz.c | 71 ++-
src/backend/access/compression/cm_zlib.c | 89 ++-
src/backend/access/compression/cmapi.c | 53 ++
src/backend/access/heap/heapam.c | 16 +-
src/backend/access/heap/heaptoast.c | 6 +-
src/backend/access/heap/rewriteheap.c | 2 +-
src/backend/access/table/toast_helper.c | 78 ++-
src/backend/catalog/Makefile | 2 +-
src/backend/catalog/dependency.c | 9 +
src/backend/catalog/objectaddress.c | 58 ++
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 623 +++++++++++++++++++-
src/backend/commands/copy.c | 4 +-
src/backend/commands/createas.c | 2 +-
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/foreigncmds.c | 46 +-
src/backend/commands/matview.c | 2 +-
src/backend/commands/tablecmds.c | 369 +++++++++++-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/nodeFuncs.c | 8 +
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 57 +-
src/backend/parser/parse_utilcmd.c | 2 +-
src/backend/utils/adt/pg_upgrade_support.c | 10 +
src/backend/utils/cache/syscache.c | 12 +
src/bin/pg_dump/pg_dump.c | 148 ++++-
src/bin/pg_dump/pg_dump.h | 15 +
src/bin/pg_dump/t/002_pg_dump.pl | 95 +++
src/bin/psql/describe.c | 11 +-
src/include/access/cmapi.h | 13 +-
src/include/access/heapam.h | 3 +-
src/include/access/heaptoast.h | 3 +-
src/include/access/hio.h | 2 +
src/include/access/toast_helper.h | 4 +-
src/include/access/toast_internals.h | 21 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/dependency.h | 5 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attr_compression.dat | 24 +
src/include/catalog/pg_attr_compression.h | 53 ++
src/include/catalog/pg_proc.dat | 7 +
src/include/catalog/toasting.h | 1 +
src/include/commands/defrem.h | 12 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 19 +-
src/include/utils/syscache.h | 1 +
src/test/regress/expected/create_cm.out | 376 +++++++++++-
src/test/regress/expected/create_cm_1.out | 407 +++++++++++++
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/expected/psql.out | 12 +-
src/test/regress/expected/sanity_check.out | 3 +
src/test/regress/sql/create_cm.sql | 171 +++++-
src/tools/pgindent/typedefs.list | 1 +
66 files changed, 3277 insertions(+), 221 deletions(-)
create mode 100644 doc/src/sgml/compression-am.sgml
create mode 100644 src/include/catalog/pg_attr_compression.dat
create mode 100644 src/include/catalog/pg_attr_compression.h
create mode 100644 src/test/regress/expected/create_cm_1.out
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5ef5b69e81..642a182f2a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -70,6 +70,11 @@
<entry>access method support functions</entry>
</row>
+ <row>
+ <entry><link linkend="catalog-pg-attr-compression"><structname>pg_attr_compression</structname></link></entry>
+ <entry>table columns compression relationships and options</entry>
+ </row>
+
<row>
<entry><link linkend="catalog-pg-attrdef"><structname>pg_attrdef</structname></link></entry>
<entry>column default values</entry>
@@ -660,8 +665,9 @@
relation access methods. There is one row for each access method supported
by the system.
Currently, there are tables, indexes and compression access methods.
- The requirements for table and index access methods are discussed in detail
- in <xref linkend="tableam"/> and <xref linkend="indexam"/> respectively.
+ The requirements for table, index and compression access methods
+ are discussed in detail in <xref linkend="tableam"/>,
+ <xref linkend="indexam"/> and <xref linkend="compression-am"/> respectively.
</para>
<table>
@@ -1011,6 +1017,12 @@
</sect1>
+ <sect1 id="catalog-pg-attr-compression">
+ <title><structname>pg_attr_compression</structname></title>
+ <indexterm zone="catalog-pg-attr-compression">
+ <primary>pg_attr_compression</primary>
+ </indexterm>
+ </sect1>
<sect1 id="catalog-pg-attrdef">
<title><structname>pg_attrdef</structname></title>
diff --git a/doc/src/sgml/compression-am.sgml b/doc/src/sgml/compression-am.sgml
new file mode 100644
index 0000000000..e23d817910
--- /dev/null
+++ b/doc/src/sgml/compression-am.sgml
@@ -0,0 +1,178 @@
+<!-- doc/src/sgml/compression-am.sgml -->
+
+<chapter id="compression-am">
+ <title>Compression Access Methods</title>
+ <para>
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through compression access methods interface.
+ </para>
+
+ <sect1 id="builtin-compression-methods">
+ <title>Built-in Compression Access Methods</title>
+ <para>
+ These compression access methods are included in
+ <productname>PostgreSQL</productname> and don't need any external extensions.
+ </para>
+ <table id="builtin-compression-methods-table">
+ <title>Built-in Compression Access Methods</title>
+ <tgroup cols="2">
+ <thead>
+ <row>
+ <entry>Name</entry>
+ <entry>Options</entry>
+ </row>
+ </thead>
+ <tbody>
+ <row>
+ <entry><literal>pglz</literal></entry>
+ <entry>
+ <literal>min_input_size (int)</literal>,
+ <literal>max_input_size (int)</literal>,
+ <literal>min_comp_rate (int)</literal>,
+ <literal>first_success_by (int)</literal>,
+ <literal>match_size_good (int)</literal>,
+ <literal>match_size_drop (int)</literal>
+ </entry>
+ </row>
+ <row>
+ <entry><literal>zlib</literal></entry>
+ <entry><literal>level (text)</literal>, <literal>dict (text)</literal></entry>
+ </row>
+ </tbody>
+ </tgroup>
+ </table>
+ <para>
+ Note that for <literal>zlib</literal> to work it should be installed in the
+ system and <productname>PostgreSQL</productname> should be compiled without
+ <literal>--without-zlib</literal> flag.
+ </para>
+ </sect1>
+
+ <sect1 id="compression-api">
+ <title>Basic API for compression methods</title>
+
+ <para>
+ Each compression access method is described by a row in the
+ <link linkend="catalog-pg-am"><structname>pg_am</structname></link>
+ system catalog. The <structname>pg_am</structname> entry
+ specifies a name and a <firstterm>handler function</firstterm> for the access
+ method. These entries can be created and deleted using the
+ <xref linkend="sql-create-access-method"/> and
+ <xref linkend="sql-drop-access-method"/> SQL commands.
+ </para>
+
+ <para>
+ A compression access method handler function must be declared to accept a
+ single argument of type <type>internal</type> and to return the
+ pseudo-type <type>compression_am_handler</type>. The argument is a dummy value that
+ simply serves to prevent handler functions from being called directly from
+ SQL commands. The result of the function must be a palloc'd struct of
+ type <structname>CompressionAmRoutine</structname>, which contains everything
+ that the core code needs to know to make use of the compression access method.
+ The <structname>CompressionAmRoutine</structname> struct, also called the access
+ method's <firstterm>API struct</firstterm>, contains pointers to support
+ functions for the access method. These support functions are plain C
+ functions and are not visible or callable at the SQL level.
+ The support functions are described in <xref linkend="compression-am-functions"/>.
+ </para>
+
+ <para>
+ The structure <structname>CompressionAmRoutine</structname> is defined thus:
+<programlisting>
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcheck_function cmcheck; /* can be NULL */
+ cminitstate_function cminitstate; /* can be NULL */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+} CompressionAmRoutine;
+</programlisting>
+ </para>
+ </sect1>
+ <sect1 id="compression-am-functions">
+ <title>Compression Access Method Functions</title>
+
+ <para>
+ The compression and auxiliary functions that an compression access
+ method must provide in <structname>CompressionAmRoutine</structname> are:
+ </para>
+
+ <para>
+<programlisting>
+void
+cmcheck (Form_pg_attribute att, List *options);
+</programlisting>
+ Called when an attribute is linked with compression access method. Could
+ be used to check compatibility with the attribute and other additional
+ checks.
+ </para>
+
+ <para>
+ Compression functions take special struct
+ <structname>CompressionAmOptions</structname> as first
+ parameter. This struct contains per backend cached state for each
+ attribute compression record. CompressionAmOptions is defined thus:
+
+<programlisting>
+typedef struct CompressionAmOptions
+{
+ Oid acoid; /* Oid of attribute compression */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
+ CompressionAmRoutine *amroutine; /* compression access method routine */
+
+ /* result of cminitstate function will be put here */
+ void *acstate;
+} CompressionAmOptions;
+</programlisting>
+ </para>
+
+ <para>
+ The <structfield>acstate</structfield> field is used to keep temporary state
+ between compression functions calls and stores the result of
+ <structfield>cminitstate</structfield> function. It could be useful to store
+ the parsed view of the compression options.
+ </para>
+
+ <para>
+ Note that any invalidation of <structname>pg_attr_compression</structname> relation
+ will cause all the cached <structfield>acstate</structfield> options cleared.
+ They will be recreated on the next compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+void *
+cminitstate (Oid acoid, List *options);
+</programlisting>
+ Called when <structname>CompressionAmOptions</structname> is being
+ initialized. Can return a pointer to memory that will be passed between
+ compression functions calls.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmcompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to compress varlena. Could return NULL if data is
+ incompressible. If it returns varlena bigger than original the core will
+ not use it.
+ </para>
+
+ <para>
+<programlisting>
+struct varlena *
+cmdecompress (CompressionAmOptions *cmoptions,
+ const struct varlena *value);
+</programlisting>
+ Function is used to decompress varlena.
+ </para>
+
+ </sect1>
+</chapter>
diff --git a/doc/src/sgml/filelist.sgml b/doc/src/sgml/filelist.sgml
index 64b5da0070..ec7f2b5ca9 100644
--- a/doc/src/sgml/filelist.sgml
+++ b/doc/src/sgml/filelist.sgml
@@ -91,6 +91,7 @@
<!ENTITY planstats SYSTEM "planstats.sgml">
<!ENTITY tableam SYSTEM "tableam.sgml">
<!ENTITY indexam SYSTEM "indexam.sgml">
+<!ENTITY compression-am SYSTEM "compression-am.sgml">
<!ENTITY nls SYSTEM "nls.sgml">
<!ENTITY plhandler SYSTEM "plhandler.sgml">
<!ENTITY fdwhandler SYSTEM "fdwhandler.sgml">
diff --git a/doc/src/sgml/indexam.sgml b/doc/src/sgml/indexam.sgml
index 1aea4db707..8f63c4cac9 100644
--- a/doc/src/sgml/indexam.sgml
+++ b/doc/src/sgml/indexam.sgml
@@ -55,7 +55,7 @@
<title>Basic API Structure for Indexes</title>
<para>
- Each index access method is described by a row in the
+ Each index access method is described by a row with INDEX type in the
<link linkend="catalog-pg-am"><structname>pg_am</structname></link>
system catalog. The <structname>pg_am</structname> entry
specifies a name and a <firstterm>handler function</firstterm> for the index
diff --git a/doc/src/sgml/postgres.sgml b/doc/src/sgml/postgres.sgml
index c41ce9499b..2d14e32775 100644
--- a/doc/src/sgml/postgres.sgml
+++ b/doc/src/sgml/postgres.sgml
@@ -259,6 +259,7 @@ break is not needed in a wider output rendering.
&geqo;
&tableam;
&indexam;
+ &compression-am;
&generic-wal;
&btree;
&gist;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index b2eb7097a9..1668a007b5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_am</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,23 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method_name</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
+ </term>
+ <listitem>
+ <para>
+ This form adds compression to a column. Compression access method should be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression access methods
+ used on the column and determines which of them should be kept on the
+ column. Without PRESERVE or partial list of compression methods table
+ will be rewritten.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index 3da27c8ce9..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -84,7 +84,8 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
is described in <xref linkend="tableam"/>, the index access method
- API is described in <xref linkend="indexam"/>.
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 1a01c91b85..c47b184add 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -968,11 +968,13 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_access_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
- This clause adds compression to a column. Currently we can give the built-in
- compression method.
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index ecc3359f64..b1bec9f1b2 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -396,7 +396,9 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
The default compression technique used for either in-line or out-of-line compressed
data is a fairly simple and very fast member
of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+<filename>src/common/pg_lzcompress.c</filename> for the details. Also custom
+compressions could be used. Look at <xref linkend="compression-am"/> for
+more information.
</para>
<sect2 id="storage-toast-ondisk">
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 0a6592fbf9..aa77b1bddd 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -451,12 +451,12 @@ toast_decompress_datum(struct varlena *attr)
if (VARATT_IS_CUSTOM_COMPRESSED(attr))
{
- CompressionAmOptions cmoptions;
+ CompressionAmOptions *cmoptions;
toast_compress_header_custom *hdr;
- hdr = (toast_compress_header_custom *) attr;
- lookup_compression_am_options(hdr->cmid, &cmoptions);
- result = cmoptions.amroutine->cmdecompress(&cmoptions, attr);
+ hdr = (toast_compress_header_custom *)attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
}
else
{
@@ -497,16 +497,15 @@ toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
if (VARATT_IS_CUSTOM_COMPRESSED(attr))
{
- CompressionAmOptions cmoptions;
+ CompressionAmOptions *cmoptions;
toast_compress_header_custom *hdr;
- hdr = (toast_compress_header_custom *) attr;
- lookup_compression_am_options(hdr->cmid, &cmoptions);
- if (cmoptions.amroutine->cmdecompress_slice)
- result = cmoptions.amroutine->cmdecompress_slice(&cmoptions, attr,
- slicelength);
+ hdr = (toast_compress_header_custom *)attr;
+ cmoptions = lookup_compression_am_options(hdr->cmid);
+ if (cmoptions->amroutine->cmdecompress_slice)
+ result = cmoptions->amroutine->cmdecompress_slice(cmoptions, attr, slicelength);
else
- result = cmoptions.amroutine->cmdecompress(&cmoptions, attr);
+ result = cmoptions->amroutine->cmdecompress(cmoptions, attr);
}
else
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..faa5deb812 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -1336,11 +1336,99 @@ untransformRelOptions(Datum options)
val = (Node *) makeString(pstrdup(p));
}
result = lappend(result, makeDefElem(pstrdup(s), val, -1));
+ pfree(s);
}
return result;
}
+/*
+ * helper function for qsort to compare DefElem
+ */
+static int
+compare_options(const ListCell *a, const ListCell *b)
+{
+ DefElem *da = (DefElem *) lfirst(a);
+ DefElem *db = (DefElem *) lfirst(b);
+
+ return strcmp(da->defname, db->defname);
+}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping,
+ * pg_foreign_table and pg_attr_compression
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options, bool sorted)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+ List *resoptions = NIL;
+ int len = list_length(options);
+
+ /* sort by option name if needed */
+ if (sorted && len > 1)
+ list_sort(options, compare_options);
+ resoptions = options;
+
+ foreach(cell, resoptions)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ /* free if the the modified version of list was used */
+ if (resoptions != options)
+ list_free(resoptions);
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
+
/*
* Extract and parse reloptions from a pg_class tuple.
*
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 38d273e8bd..62a09903b8 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -35,6 +35,9 @@
#include "utils/snapmgr.h"
#include "utils/syscache.h"
+HTAB *amoptions_cache = NULL;
+MemoryContext amoptions_cache_mcxt = NULL;
+
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
@@ -45,8 +48,7 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* for decompression.
* --------
*/
-void
-toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
+void toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_CMID(val, cmid);
@@ -66,21 +68,21 @@ toast_set_compressed_datum_info(struct varlena *val, Oid cmid, int32 rawsize)
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid amoid)
+toast_compress_datum(Datum value, Oid acoid)
{
struct varlena *tmp = NULL;
int32 valsize;
- CompressionAmOptions cmoptions;
+ CompressionAmOptions *cmoptions = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
/* Fallback to default compression if not specified */
- if (!OidIsValid(amoid))
- amoid = DefaultCompressionOid;
+ if (!OidIsValid(acoid))
+ acoid = DefaultCompressionOid;
- lookup_compression_am_options(amoid, &cmoptions);
- tmp = cmoptions.amroutine->cmcompress(&cmoptions, (const struct varlena *) value);
+ cmoptions = lookup_compression_am_options(acoid);
+ tmp = cmoptions->amroutine->cmcompress(cmoptions, (const struct varlena *)value);
if (!tmp)
return PointerGetDatum(NULL);
@@ -100,7 +102,7 @@ toast_compress_datum(Datum value, Oid amoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, amoid, valsize);
+ toast_set_compressed_datum_info(tmp, cmoptions->acoid, valsize);
return PointerGetDatum(tmp);
}
@@ -649,22 +651,157 @@ init_toast_snapshot(Snapshot toast_snapshot)
InitToastSnapshot(*toast_snapshot, snapshot->lsn, snapshot->whenTaken);
}
+/* ----------
+ * remove_cached_amoptions
+ *
+ * Remove specified compression options from cache
+ */
+static void
+remove_cached_amoptions(CompressionAmOptions *entry)
+{
+ MemoryContextDelete(entry->mcxt);
+
+ if (hash_search(amoptions_cache, (void *)&entry->acoid,
+ HASH_REMOVE, NULL) == NULL)
+ elog(ERROR, "hash table corrupted");
+}
+
+/* ----------
+ * invalidate_amoptions_cache
+ *
+ * Flush cache entries when pg_attr_compression is updated.
+ */
+static void
+invalidate_amoptions_cache(Datum arg, int cacheid, uint32 hashvalue)
+{
+ HASH_SEQ_STATUS status;
+ CompressionAmOptions *entry;
+
+ hash_seq_init(&status, amoptions_cache);
+ while ((entry = (CompressionAmOptions *)hash_seq_search(&status)) != NULL)
+ remove_cached_amoptions(entry);
+}
+
+/* ----------
+ * init_amoptions_cache
+ *
+ * Initialize a local cache for attribute compression options.
+ */
+void
+init_amoptions_cache(void)
+{
+ HASHCTL ctl;
+
+ amoptions_cache_mcxt = AllocSetContextCreate(TopMemoryContext,
+ "attr compression cache",
+ ALLOCSET_DEFAULT_SIZES);
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(Oid);
+ ctl.entrysize = sizeof(CompressionAmOptions);
+ ctl.hcxt = amoptions_cache_mcxt;
+ amoptions_cache = hash_create("attr compression cache", 100, &ctl,
+ HASH_ELEM | HASH_BLOBS | HASH_CONTEXT);
+
+ /* Set up invalidation callback on pg_attr_compression. */
+ CacheRegisterSyscacheCallback(ATTCOMPRESSIONOID,
+ invalidate_amoptions_cache,
+ (Datum)0);
+}
+
/* ----------
* lookup_compression_am_options
*
* Get compression handler routine.
*/
-void
-lookup_compression_am_options(Oid acoid, CompressionAmOptions *result)
+CompressionAmOptions *
+lookup_compression_am_options(Oid acoid)
{
+ bool found,
+ optisnull;
+ CompressionAmOptions *result;
+ Datum acoptions;
+ Form_pg_attr_compression acform;
+ HeapTuple tuple;
+ Oid amoid;
regproc amhandler;
- amhandler = get_am_handler_oid(acoid, AMTYPE_COMPRESSION, false);
- result->amroutine = InvokeCompressionAmHandler(amhandler);
- result->acstate = result->amroutine->cminitstate ?
- result->amroutine->cminitstate(acoid) : NULL;
+ /*
+ * This call could invalidate system cache so we need to call it before
+ * we're putting something to our cache.
+ */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression)GETSTRUCT(tuple);
+ amoid = get_compression_am_oid(NameStr(acform->acname), false);
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &optisnull);
+
+ if (!amoptions_cache)
+ init_amoptions_cache();
+
+ result = hash_search(amoptions_cache, &acoid, HASH_ENTER, &found);
+ if (!found)
+ {
+ MemoryContext oldcxt = CurrentMemoryContext;
+
+ result->mcxt = AllocSetContextCreate(amoptions_cache_mcxt,
+ "compression am options",
+ ALLOCSET_DEFAULT_SIZES);
+
+ PG_TRY();
+ {
+ result->acoid = acoid;
+ result->amoid = amoid;
+
+ MemoryContextSwitchTo(result->mcxt);
+ result->amroutine = InvokeCompressionAmHandler(amhandler);
+ if (optisnull)
+ result->acoptions = NIL;
+ else
+ result->acoptions = untransformRelOptions(acoptions);
+ result->acstate = result->amroutine->cminitstate ? result->amroutine->cminitstate(acoid, result->acoptions) : NULL;
+ }
+ PG_CATCH();
+ {
+ MemoryContextSwitchTo(oldcxt);
+ remove_cached_amoptions(result);
+ PG_RE_THROW();
+ }
+ PG_END_TRY();
+ MemoryContextSwitchTo(oldcxt);
+ }
+
+ ReleaseSysCache(tuple);
if (!result->amroutine)
/* should not happen but need to check if something goes wrong */
elog(ERROR, "compression access method routine is NULL");
+
+ return result;
+}
+
+/* ----------
+ * attr_compression_options_are_equal
+ *
+ * Compare two attribute compression records
+ */
+bool
+attr_compression_options_are_equal(Oid acoid1, Oid acoid2)
+{
+ CompressionAmOptions *a;
+ CompressionAmOptions *b;
+
+ a = lookup_compression_am_options(acoid1);
+ b = lookup_compression_am_options(acoid2);
+
+ if (a->amoid != b->amoid)
+ return false;
+
+ if (list_length(a->acoptions) != list_length(b->acoptions))
+ return false;
+
+ return equal(a->acoptions, b->acoptions);
}
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
index 7541b777ef..8203605966 100644
--- a/src/backend/access/compression/cm_pglz.c
+++ b/src/backend/access/compression/cm_pglz.c
@@ -18,17 +18,83 @@
#include "nodes/parsenodes.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
/*
* Configure PGLZ_Strategy struct for compression function
*/
static void *
-pglz_cminitstate(Oid acoid)
+pglz_cminitstate(Oid acoid, List *options)
{
+ ListCell *lc;
PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
/* initialize with default strategy values */
memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
-
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
return (void *) strategy;
}
@@ -114,6 +180,7 @@ pglzhandler(PG_FUNCTION_ARGS)
{
CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+ routine->cmcheck = pglz_cmcheck;
routine->cminitstate = pglz_cminitstate;
routine->cmcompress = pglz_cmcompress;
routine->cmdecompress = pglz_cmdecompress;
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
index b1ba33e90b..e9bf2772e2 100644
--- a/src/backend/access/compression/cm_zlib.c
+++ b/src/backend/access/compression/cm_zlib.c
@@ -30,14 +30,100 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(Form_pg_attribute att, List *options)
+{
+ ListCell *lc;
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ int ntokens = 0;
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+ if (strlen(val) > (ZLIB_MAX_DICTIONARY_LENGTH - 1))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary length should be less than %d", ZLIB_MAX_DICTIONARY_LENGTH))));
+
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ ntokens++;
+ val = NULL;
+ }
+
+ if (ntokens < 2)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ (errmsg("zlib dictionary is too small"))));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
static void *
-zlib_cminitstate(Oid acoid)
+zlib_cminitstate(Oid acoid, List *options)
{
zlib_state *state = NULL;
state = palloc0(sizeof(zlib_state));
state->level = Z_DEFAULT_COMPRESSION;
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ else if (strcmp(def->defname, "dict") == 0)
+ {
+ char *val,
+ *tok;
+
+ val = pstrdup(defGetString(def));
+
+ /* Fill the zlib dictionary */
+ while ((tok = strtok(val, ZLIB_DICTIONARY_DELIM)) != NULL)
+ {
+ int len = strlen(tok);
+ memcpy((void *) (state->dict + state->dictlen), tok, len);
+ state->dictlen += len + 1;
+ Assert(state->dictlen <= ZLIB_MAX_DICTIONARY_LENGTH);
+
+ /* add space as dictionary delimiter */
+ state->dict[state->dictlen - 1] = ' ';
+ val = NULL;
+ }
+ }
+ }
+ }
+
return state;
}
@@ -153,6 +239,7 @@ zlibhandler(PG_FUNCTION_ARGS)
#else
CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+ routine->cmcheck = zlib_cmcheck;
routine->cminitstate = zlib_cminitstate;
routine->cmcompress = zlib_cmcompress;
routine->cmdecompress = zlib_cmdecompress;
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
index 3cdce17276..504da0638f 100644
--- a/src/backend/access/compression/cmapi.c
+++ b/src/backend/access/compression/cmapi.c
@@ -18,6 +18,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "commands/defrem.h"
#include "utils/syscache.h"
@@ -41,3 +42,55 @@ InvokeCompressionAmHandler(Oid amhandler)
return routine;
}
+
+/*
+ * GetAttrCompressionOptions
+ *
+ * Parse array of attribute compression options and return it as a list.
+ */
+List *
+GetAttrCompressionOptions(Oid acoid)
+{
+ HeapTuple tuple;
+ List *result = NIL;
+ bool isnull;
+ Datum acoptions;
+
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for an attribute compression %u", acoid);
+
+ /* options could be NULL, so we can't use form struct */
+ acoptions = SysCacheGetAttr(ATTCOMPRESSIONOID, tuple,
+ Anum_pg_attr_compression_acoptions, &isnull);
+
+ if (!isnull)
+ result = untransformRelOptions(acoptions);
+
+ ReleaseSysCache(tuple);
+ return result;
+}
+
+/*
+ * GetAttrCompressionAmOid
+ *
+ * Return access method Oid by attribute compression Oid
+ */
+Oid
+GetAttrCompressionAmOid(Oid acoid)
+{
+ Oid result;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ /* extract access method Oid */
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ result = get_compression_am_oid(NameStr(acform->acname), false);
+ ReleaseSysCache(tuple);
+
+ return result;
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 9d52fa1131..38ddc9f69e 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -72,7 +72,7 @@
static HeapTuple heap_prepare_insert(Relation relation, HeapTuple tup,
- TransactionId xid, CommandId cid, int options);
+ TransactionId xid, CommandId cid, int options, BulkInsertState bistate);
static XLogRecPtr log_heap_update(Relation reln, Buffer oldbuf,
Buffer newbuf, HeapTuple oldtup,
HeapTuple newtup, HeapTuple old_key_tuple,
@@ -1803,13 +1803,14 @@ UpdateXmaxHintBits(HeapTupleHeader tuple, Buffer buffer, TransactionId xid)
* GetBulkInsertState - prepare status object for a bulk insert
*/
BulkInsertState
-GetBulkInsertState(void)
+GetBulkInsertState(HTAB *preserved_am_info)
{
BulkInsertState bistate;
bistate = (BulkInsertState) palloc(sizeof(BulkInsertStateData));
bistate->strategy = GetAccessStrategy(BAS_BULKWRITE);
bistate->current_buf = InvalidBuffer;
+ bistate->preserved_am_info = preserved_am_info;
return bistate;
}
@@ -1871,7 +1872,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
* Note: below this point, heaptup is the data we actually intend to store
* into the relation; tup is the caller's original untoasted data.
*/
- heaptup = heap_prepare_insert(relation, tup, xid, cid, options);
+ heaptup = heap_prepare_insert(relation, tup, xid, cid, options, bistate);
/*
* Find buffer to insert this tuple into. If the page is all visible,
@@ -2041,7 +2042,7 @@ heap_insert(Relation relation, HeapTuple tup, CommandId cid,
*/
static HeapTuple
heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
- CommandId cid, int options)
+ CommandId cid, int options, BulkInsertState bistate)
{
/*
* Parallel operations are required to be strictly read-only in a parallel
@@ -2082,7 +2083,8 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
|| RelationGetDescr(relation)->tdflags & TD_ATTR_CUSTOM_COMPRESSED
|| HeapTupleHasCustomCompressed(tup)
|| tup->t_len > TOAST_TUPLE_THRESHOLD)
- return heap_toast_insert_or_update(relation, tup, NULL, options);
+ return heap_toast_insert_or_update(relation, tup, NULL, options,
+ bistate ? bistate->preserved_am_info : NULL);
else
return tup;
}
@@ -2130,7 +2132,7 @@ heap_multi_insert(Relation relation, TupleTableSlot **slots, int ntuples,
slots[i]->tts_tableOid = RelationGetRelid(relation);
tuple->t_tableOid = slots[i]->tts_tableOid;
heaptuples[i] = heap_prepare_insert(relation, tuple, xid, cid,
- options);
+ options, bistate);
}
/*
@@ -3511,7 +3513,7 @@ l2:
if (need_toast)
{
/* Note we always use WAL and FSM during updates */
- heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0);
+ heaptup = heap_toast_insert_or_update(relation, newtup, &oldtup, 0, NULL);
newtupsize = MAXALIGN(heaptup->t_len);
}
else
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index e7fa3f6683..2ccc9893ae 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -94,7 +94,7 @@ heap_toast_delete(Relation rel, HeapTuple oldtup, bool is_speculative)
*/
HeapTuple
heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
- int options)
+ int options, HTAB *preserved_am_info)
{
HeapTuple result_tuple;
TupleDesc tupleDesc;
@@ -154,7 +154,7 @@ heap_toast_insert_or_update(Relation rel, HeapTuple newtup, HeapTuple oldtup,
ttc.ttc_oldisnull = toast_oldisnull;
}
ttc.ttc_attr = toast_attr;
- toast_tuple_init(&ttc);
+ toast_tuple_init(&ttc, preserved_am_info);
/* ----------
* Compress and/or save external until data fits into target length
@@ -531,7 +531,7 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
heap_fill_tuple(tupleDesc,
toast_values,
toast_isnull,
- (char *) new_data + new_header_len,
+ (char *)new_data + new_header_len,
new_data_len,
&(new_data->t_infomask),
&(new_data->t_infomask2),
diff --git a/src/backend/access/heap/rewriteheap.c b/src/backend/access/heap/rewriteheap.c
index 39e33763df..ede9fda72f 100644
--- a/src/backend/access/heap/rewriteheap.c
+++ b/src/backend/access/heap/rewriteheap.c
@@ -646,7 +646,7 @@ raw_heap_insert(RewriteState state, HeapTuple tup)
options |= HEAP_INSERT_NO_LOGICAL;
heaptup = heap_toast_insert_or_update(state->rs_new_rel, tup, NULL,
- options);
+ options, NULL);
}
else
heaptup = tup;
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..a882a145fb 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -38,7 +38,7 @@
* perform any initialization of the array before calling this function.
*/
void
-toast_tuple_init(ToastTupleContext *ttc)
+toast_tuple_init(ToastTupleContext *ttc, HTAB *preserved_am_info)
{
TupleDesc tupleDesc = ttc->ttc_rel->rd_att;
int numAttrs = tupleDesc->natts;
@@ -120,12 +120,30 @@ toast_tuple_init(ToastTupleContext *ttc)
*/
if (att->attlen == -1)
{
+ List *preserved_amoids = NIL;
+
/*
* If the table's attribute says PLAIN always, force it so.
*/
if (att->attstorage == TYPSTORAGE_PLAIN)
ttc->ttc_attr[i].tai_colflags |= TOASTCOL_IGNORE;
+ /*
+ * If it's ALTER SET COMPRESSION command we should check that
+ * access method of compression in preserved list.
+ */
+ if (preserved_am_info != NULL)
+ {
+ bool found;
+
+ AttrCmPreservedInfo *pinfo;
+
+ pinfo = hash_search(preserved_am_info, &att->attnum,
+ HASH_FIND, &found);
+ if (found)
+ preserved_amoids = pinfo->preserved_amoids;
+ }
+
/*
* We took care of UPDATE above, so any external value we find
* still in the tuple must be someone else's that we cannot reuse
@@ -145,6 +163,64 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_FREE;
ttc->ttc_flags |= (TOAST_NEEDS_CHANGE | TOAST_NEEDS_FREE);
}
+ /*
+ * Process custom compressed datum.
+ *
+ * 1) If destination column has identical compression move the data as is
+ * and only change attribute compression Oid. 2) If it's rewrite from
+ * ALTER command check list of preserved compression access methods. 3) In
+ * other cases just untoast the datum.
+ */
+ else if (VARATT_IS_CUSTOM_COMPRESSED(new_value))
+ {
+ bool storage_ok;
+ toast_compress_header_custom *hdr;
+
+ storage_ok = (att->attstorage == TYPSTORAGE_MAIN ||
+ att->attstorage == TYPSTORAGE_PLAIN);
+ hdr = (toast_compress_header_custom *) new_value;
+
+ if (!storage_ok || hdr->cmid != att->attcompression)
+ {
+ if (storage_ok &&
+ OidIsValid(att->attcompression) &&
+ attr_compression_options_are_equal(att->attcompression,
+ hdr->cmid))
+ {
+ struct varlena *tmpval = NULL;
+
+ /* identical compression, just change Oid to new one */
+ tmpval = palloc(VARSIZE(new_value));
+ memcpy(tmpval, new_value, VARSIZE(new_value));
+ TOAST_COMPRESS_SET_CMID(tmpval, att->attcompression);
+ new_value = tmpval;
+ ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_FREE;
+ ttc->ttc_flags |= (TOAST_NEEDS_CHANGE | TOAST_NEEDS_FREE);
+ }
+ else if (preserved_amoids != NULL)
+ {
+ Oid amoid = GetAttrCompressionAmOid(hdr->cmid);
+
+ /* decompress the value if it's not in preserved list */
+ if (!list_member_oid(preserved_amoids, amoid))
+ {
+ new_value = detoast_attr(new_value);
+ ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_FREE;
+ ttc->ttc_flags |= (TOAST_NEEDS_CHANGE | TOAST_NEEDS_FREE);
+ }
+ }
+ else
+ {
+ /* just decompress the value */
+ new_value = detoast_attr(new_value);
+ ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_FREE;
+ ttc->ttc_flags |= (TOAST_NEEDS_CHANGE | TOAST_NEEDS_FREE);
+ }
+
+ /* FIXME: Check the ttc_flags */
+ ttc->ttc_values[i] = PointerGetDatum(new_value);
+ }
+ }
/*
* Remember the size of this attribute
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..7b3a0a0efb 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -56,7 +56,7 @@ CATALOG_HEADERS := \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
- pg_statistic_ext.h pg_statistic_ext_data.h \
+ pg_statistic_ext.h pg_statistic_ext_data.h pg_attr_compression.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
pg_cast.h pg_enum.h pg_namespace.h pg_conversion.h pg_depend.h \
pg_database.h pg_db_role_setting.h pg_tablespace.h \
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b67d3e5666..99bac7839e 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -25,6 +25,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -181,6 +182,7 @@ static const Oid object_classes[] = {
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
TransformRelationId, /* OCLASS_TRANSFORM */
+ AttrCompressionRelationId /* OCLASS_ATTR_COMPRESSION */
};
@@ -1499,6 +1501,10 @@ doDeletion(const ObjectAddress *object, int flags)
DropObjectById(object);
break;
+ case OCLASS_ATTR_COMPRESSION:
+ RemoveAttributeCompression(object->objectId);
+ break;
+
/*
* These global object types are not supported here.
*/
@@ -2843,6 +2849,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case AttrCompressionRelationId:
+ return OCLASS_ATTR_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6dfe1be2cc..b9e22beb92 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -26,6 +26,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attrdef.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -616,6 +617,19 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_USER_MAPPING,
false
},
+ {
+ "compression",
+ AttrCompressionRelationId,
+ AttrCompressionIndexId,
+ ATTCOMPRESSIONOID,
+ -1,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ false
+ }
};
/*
@@ -3907,6 +3921,37 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ char *attname;
+ HeapTuple tup;
+ Form_pg_attr_compression acform;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(object->objectId));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", object->objectId);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tup);
+ if (OidIsValid(acform->acrelid))
+ {
+ appendStringInfo(&buffer, _("compression on "));
+ getRelationDescription(&buffer, acform->acrelid, false);
+
+ attname = get_attname(acform->acrelid, acform->acattnum, true);
+ if (attname)
+ {
+ appendStringInfo(&buffer, _(" column %s"), attname);
+ pfree(attname);
+ }
+ }
+ else
+ appendStringInfo(&buffer, _("attribute compression %u"), object->objectId);
+
+ ReleaseSysCache(tup);
+
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4524,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_ATTR_COMPRESSION:
+ appendStringInfoString(&buffer, "attribute compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5155,6 +5204,15 @@ getObjectIdentityParts(const ObjectAddress *object,
break;
}
+ case OCLASS_ATTR_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
case OCLASS_REWRITE:
{
Relation ruleDesc;
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..5c30758197 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 3d56f7399f..9cf57ae0cd 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -22,7 +22,7 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
-#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_collation_d.h"
#include "catalog/pg_depend.h"
@@ -37,49 +37,636 @@
#include "utils/syscache.h"
#include "utils/snapmgr.h"
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_attr_compression_oid = InvalidOid;
+
/*
- * Link compression with an attribute.
+ * When conditions of compression satisfies one if builtin attribute
+ * compresssion tuples the compressed attribute will be linked to
+ * builtin compression without new record in pg_attr_compression.
+ * So the fact that the column has a builtin compression we only can find out
+ * by its dependency.
+ */
+static void
+lookup_builtin_dependencies(Oid attrelid, AttrNumber attnum,
+ List **amoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ Oid amoid = InvalidOid;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AttrCompressionRelationId)
+ {
+ Assert(IsBuiltinCompression(depform->refobjid));
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ *amoids = list_append_unique_oid(*amoids, amoid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Find identical attribute compression for reuse and fill the list with
+ * used compression access methods.
+ */
+static Oid
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum,
+ Oid amoid, Datum acoptions, List **previous_amoids)
+{
+ Relation rel;
+ HeapTuple tuple;
+ SysScanDesc scan;
+ FmgrInfo arrayeq_info;
+ Oid result = InvalidOid;
+ ScanKeyData key[2];
+
+ /* fill FmgrInfo for array_eq function */
+ fmgr_info(F_ARRAY_EQ, &arrayeq_info);
+
+ Assert((attrelid > 0 && attnum > 0) || (attrelid == 0 && attnum == 0));
+ if (previous_amoids)
+ lookup_builtin_dependencies(attrelid, attnum, previous_amoids);
+
+ rel = table_open(AttrCompressionRelationId, AccessShareLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Oid acoid,
+ tup_amoid;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+ char *amname;
+
+ heap_deform_tuple(tuple, RelationGetDescr(rel), values, nulls);
+ acoid = DatumGetObjectId(values[Anum_pg_attr_compression_acoid - 1]);
+ amname = NameStr(*DatumGetName(values[Anum_pg_attr_compression_acname - 1]));
+ tup_amoid = get_am_oid(amname, false);
+
+ if (previous_amoids)
+ *previous_amoids = list_append_unique_oid(*previous_amoids, tup_amoid);
+
+ if (tup_amoid != amoid)
+ continue;
+
+ /*
+ * even if we found the match, we still need to acquire all previous
+ * access methods so don't break cycle if previous_amoids is not NULL
+ */
+ if (nulls[Anum_pg_attr_compression_acoptions - 1])
+ {
+ if (DatumGetPointer(acoptions) == NULL)
+ result = acoid;
+ }
+ else if (DatumGetPointer(acoptions) != NULL)
+ {
+ bool equal;
+
+ /* check if arrays for WITH options are equal */
+ equal = DatumGetBool(CallerFInfoFunctionCall2(
+ array_eq, &arrayeq_info, DEFAULT_COLLATION_OID, acoptions,
+ values[Anum_pg_attr_compression_acoptions - 1]));
+
+ if (equal)
+ result = acoid;
+ }
+
+ if (previous_amoids == NULL && OidIsValid(result))
+ break;
+ }
+
+ systable_endscan(scan);
+ table_close(rel, AccessShareLock);
+ return result;
+}
+
+/*
+ * Link compression with an attribute. Creates a row in pg_attr_compression
+ * if needed.
*
* When compression is not specified returns default attribute compression.
* It is possible case for CREATE TABLE and ADD COLUMN commands
* where COMPRESSION syntax is optional.
+ *
+ * If any of builtin attribute compression tuples satisfies conditions
+ * returns it.
+ *
+ * For ALTER command check for previous attribute compression record with
+ * identical compression options and reuse it if found any.
+ *
+ * Note we create attribute compression for EXTERNAL storage too, so when
+ * storage is changed we can start compression on future tuples right away.
*/
Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
+CreateAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite, List **preserved_amoids)
{
+ Relation rel;
+ HeapTuple newtup;
+ Oid acoid = InvalidOid,
+ amoid;
+ regproc amhandler;
+ Datum arropt;
+ Datum values[Natts_pg_attr_compression];
+ bool nulls[Natts_pg_attr_compression];
+
+ ObjectAddress myself,
+ amref;
+
+ CompressionAmRoutine *routine;
+
/* No compression for PLAIN storage. */
- if (att->attstorage == TYPSTORAGE_PLAIN)
+ if (att->attstorage == 'p')
return InvalidOid;
/* Fallback to default compression if it's not specified */
if (compression == NULL)
return DefaultCompressionOid;
- return get_compression_am_oid(compression, false);
+ amoid = get_compression_am_oid(compression->amname, false);
+ if (compression->options)
+ arropt = optionListToArray(compression->options, true);
+ else
+ arropt = PointerGetDatum(NULL);
+
+ /* Try to find builtin compression first */
+ acoid = lookup_attribute_compression(0, 0, amoid, arropt, NULL);
+
+ /* no rewrite by default */
+ if (need_rewrite != NULL)
+ *need_rewrite = false;
+
+ if (IsBinaryUpgrade)
+ {
+ /* Skip the rewrite checks and searching of identical compression */
+ goto add_tuple;
+ }
+
+ /*
+ * attrelid will be invalid on CREATE TABLE, no need for table rewrite
+ * check.
+ */
+ if (OidIsValid(att->attrelid))
+ {
+ Oid attacoid;
+ List *previous_amoids = NIL;
+
+ /*
+ * Try to find identical compression from previous tuples, and fill
+ * the list of previous compresssion methods
+ */
+ attacoid = lookup_attribute_compression(att->attrelid, att->attnum,
+ amoid, arropt, &previous_amoids);
+ if (!OidIsValid(acoid))
+ acoid = attacoid;
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: -
+ * SET COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE
+ * but not with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ Assert(preserved_amoids != NULL);
+
+ if (compression->preserve == NIL)
+ *need_rewrite = true;
+ else
+ {
+ ListCell *cell;
+
+ foreach(cell, compression->preserve)
+ {
+ char *amname_p = strVal(lfirst(cell));
+ Oid amoid_p = get_am_oid(amname_p, false);
+
+ if (!list_member_oid(previous_amoids, amoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", amname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")
+ ));
+
+ *preserved_amoids = list_append_unique_oid(*preserved_amoids, amoid_p);
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_amoids = list_delete_oid(previous_amoids, amoid_p);
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions
+ * then we need to rewrite tuples in the table.
+ *
+ * In binary upgrade list will not be free since it contains
+ * Oid of builtin compression access method.
+ */
+ if (list_length(previous_amoids) != 0)
+ *need_rewrite = true;
+ }
+ }
+
+ /* Cleanup */
+ list_free(previous_amoids);
+ }
+
+ /* Return Oid if we already found identical compression on this column */
+ if (OidIsValid(acoid))
+ {
+ if (DatumGetPointer(arropt) != NULL)
+ pfree(DatumGetPointer(arropt));
+
+ return acoid;
+ }
+
+add_tuple:
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ amhandler = get_am_handler_oid(amoid, AMTYPE_COMPRESSION, false);
+
+ rel = table_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ if (IsBinaryUpgrade)
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_attr_compression_oid < FirstNormalObjectId &&
+ (!OidIsValid(acoid) || binary_upgrade_next_attr_compression_oid != acoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ acoid = binary_upgrade_next_attr_compression_oid;
+ }
+ else
+ {
+ acoid = GetNewOidWithIndex(rel, AttrCompressionIndexId,
+ Anum_pg_attr_compression_acoid);
+
+ }
+
+ if (acoid < FirstNormalObjectId)
+ {
+ /* this is built-in attribute compression */
+ table_close(rel, RowExclusiveLock);
+ return acoid;
+ }
+
+ /* we need routine only to call cmcheck function */
+ routine = InvokeCompressionAmHandler(amhandler);
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(att, compression->options);
+ pfree(routine);
+
+ values[Anum_pg_attr_compression_acoid - 1] = ObjectIdGetDatum(acoid);
+ values[Anum_pg_attr_compression_acname - 1] = CStringGetDatum(compression->amname);
+ values[Anum_pg_attr_compression_acrelid - 1] = ObjectIdGetDatum(att->attrelid);
+ values[Anum_pg_attr_compression_acattnum - 1] = Int32GetDatum(att->attnum);
+
+ if (compression->options)
+ values[Anum_pg_attr_compression_acoptions - 1] = arropt;
+ else
+ nulls[Anum_pg_attr_compression_acoptions - 1] = true;
+
+ newtup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+ CatalogTupleInsert(rel, newtup);
+ heap_freetuple(newtup);
+ table_close(rel, RowExclusiveLock);
+
+ ObjectAddressSet(myself, AttrCompressionRelationId, acoid);
+ ObjectAddressSet(amref, AccessMethodRelationId, amoid);
+
+ recordDependencyOn(&myself, &amref, DEPENDENCY_NORMAL);
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ /* Make the changes visible */
+ CommandCounterIncrement();
+ return acoid;
}
/*
- * Get compression name by attribute compression Oid.
+ * Remove the attribute compression record from pg_attr_compression.
*/
-char *
-GetCompressionName(Oid amoid)
+void
+RemoveAttributeCompression(Oid acoid)
+{
+ Relation relation;
+ HeapTuple tup;
+
+ tup = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tup))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ /* check we're not trying to remove builtin attribute compression */
+ Assert(((Form_pg_attr_compression) GETSTRUCT(tup))->acrelid != 0);
+
+ /* delete the record from catalogs */
+ relation = table_open(AttrCompressionRelationId, RowExclusiveLock);
+ CatalogTupleDelete(relation, &tup->t_self);
+ table_close(relation, RowExclusiveLock);
+ ReleaseSysCache(tup);
+}
+
+/*
+ * CleanupAttributeCompression
+ *
+ * Remove entries in pg_attr_compression of the column except current
+ * attribute compression and related with specified list of access methods.
+ */
+void
+CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids)
+{
+ Oid acoid,
+ amoid;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ HeapTuple tuple,
+ attrtuple;
+ Form_pg_attribute attform;
+ List *removed = NIL;
+ ListCell *lc;
+
+ attrtuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(relid),
+ Int16GetDatum(attnum));
+
+ if (!HeapTupleIsValid(attrtuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ attnum, relid);
+ attform = (Form_pg_attribute) GETSTRUCT(attrtuple);
+ acoid = attform->attcompression;
+ ReleaseSysCache(attrtuple);
+
+ Assert(relid > 0 && attnum > 0);
+ Assert(!IsBinaryUpgrade);
+
+ rel = table_open(AttrCompressionRelationId, RowExclusiveLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+
+ /*
+ * Remove attribute compression tuples and collect removed Oids
+ * to list.
+ */
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+
+ /* skip current compression */
+ if (acform->acoid == acoid)
+ continue;
+
+ if (!list_member_oid(keepAmOids, amoid))
+ {
+ removed = lappend_oid(removed, acform->acoid);
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, RowExclusiveLock);
+
+ /*
+ * Now remove dependencies between attribute compression (dependent)
+ * and column.
+ */
+ rel = table_open(DependRelationId, RowExclusiveLock);
+ foreach(lc, removed)
+ {
+ Oid tup_acoid = lfirst_oid(lc);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(AttrCompressionRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(tup_acoid));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 2, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ CatalogTupleDelete(rel, &tuple->t_self);
+
+ systable_endscan(scan);
+ }
+ table_close(rel, RowExclusiveLock);
+
+ /* Now remove dependencies with builtin compressions */
+ rel = table_open(DependRelationId, RowExclusiveLock);
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tuple);
+
+ if (depform->refclassid != AttrCompressionRelationId)
+ continue;
+
+ /* skip current compression */
+ if (depform->refobjid == acoid)
+ continue;
+
+ amoid = GetAttrCompressionAmOid(depform->refobjid);
+ if (!list_member_oid(keepAmOids, amoid))
+ CatalogTupleDelete(rel, &tuple->t_self);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, RowExclusiveLock);
+}
+
+/*
+ * Construct ColumnCompression node by attribute compression Oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid acoid)
{
HeapTuple tuple;
- Form_pg_am amform;
- char *amname;
+ Form_pg_attr_compression acform;
+ ColumnCompression *node;
- if (!OidIsValid(amoid))
+ if (!OidIsValid(acoid))
return NULL;
- /* Get handler function OID for the access method */
- tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
if (!HeapTupleIsValid(tuple))
- elog(ERROR, "cache lookup failed for attribute compression %u", amoid);
-
- amform = (Form_pg_am) GETSTRUCT(tuple);
- amname = pstrdup(NameStr(amform->amname));
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ node = makeNode(ColumnCompression);
+ node->amname = pstrdup(NameStr(acform->acname));
ReleaseSysCache(tuple);
- return amname;
+ /*
+ * fill attribute compression options too. We could've do it above but
+ * it's easier to call this helper.
+ */
+ node->options = GetAttrCompressionOptions(acoid);
+
+ return node;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->amname, c2->amname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->amname, c2->amname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
+
+/*
+ * Return list of compression methods used in specified column.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Oid relOid = PG_GETARG_OID(0);
+ char *attname = TextDatumGetCString(PG_GETARG_TEXT_P(1));
+ Relation rel;
+ HeapTuple tuple;
+ AttrNumber attnum;
+ List *amoids = NIL;
+ Oid amoid;
+ ListCell *lc;
+
+ ScanKeyData key[2];
+ SysScanDesc scan;
+ StringInfoData result;
+
+ attnum = get_attnum(relOid, attname);
+ if (attnum == InvalidAttrNumber)
+ PG_RETURN_NULL();
+
+ /* Collect related builtin compression access methods */
+ lookup_builtin_dependencies(relOid, attnum, &amoids);
+
+ /* Collect other related access methods */
+ rel = table_open(AttrCompressionRelationId, AccessShareLock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_attr_compression_acrelid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relOid));
+ ScanKeyInit(&key[1],
+ Anum_pg_attr_compression_acattnum,
+ BTEqualStrategyNumber, F_INT2EQ,
+ Int16GetDatum(attnum));
+
+ scan = systable_beginscan(rel, AttrCompressionRelidAttnumIndexId,
+ true, NULL, 2, key);
+ while (HeapTupleIsValid(tuple = systable_getnext(scan)))
+ {
+ Form_pg_attr_compression acform;
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ amoid = get_am_oid(NameStr(acform->acname), false);
+ amoids = list_append_unique_oid(amoids, amoid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, AccessShareLock);
+
+ if (!list_length(amoids))
+ PG_RETURN_NULL();
+
+ /* Construct the list separated by comma */
+ amoid = InvalidOid;
+ initStringInfo(&result);
+ foreach(lc, amoids)
+ {
+ if (OidIsValid(amoid))
+ appendStringInfoString(&result, ", ");
+
+ amoid = lfirst_oid(lc);
+ appendStringInfoString(&result, get_am_name(amoid));
+ }
+
+ PG_RETURN_TEXT_P(CStringGetTextDatum(result.data));
}
diff --git a/src/backend/commands/copy.c b/src/backend/commands/copy.c
index db7d24a511..c1fcadebe2 100644
--- a/src/backend/commands/copy.c
+++ b/src/backend/commands/copy.c
@@ -2394,7 +2394,7 @@ CopyMultiInsertBufferInit(ResultRelInfo *rri)
buffer = (CopyMultiInsertBuffer *) palloc(sizeof(CopyMultiInsertBuffer));
memset(buffer->slots, 0, sizeof(TupleTableSlot *) * MAX_BUFFERED_TUPLES);
buffer->resultRelInfo = rri;
- buffer->bistate = GetBulkInsertState();
+ buffer->bistate = GetBulkInsertState(NULL);
buffer->nused = 0;
return buffer;
@@ -2974,7 +2974,7 @@ CopyFrom(CopyState cstate)
{
singleslot = table_slot_create(resultRelInfo->ri_RelationDesc,
&estate->es_tupleTable);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(NULL);
}
has_before_insert_row_trig = (resultRelInfo->ri_TrigDesc &&
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index d53ec952d0..c593f236a5 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -552,7 +552,7 @@ intorel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->reladdr = intoRelationAddr;
myState->output_cid = GetCurrentCommandId(true);
myState->ti_options = TABLE_INSERT_SKIP_FSM;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/*
* Valid smgr_targblock implies something already wrote to the relation.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 7844880170..8d6e837274 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_ATTR_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..78935294b6 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
@@ -176,7 +132,7 @@ transformGenericOptions(Oid catalogId,
}
}
- result = optionListToArray(resultOptions);
+ result = optionListToArray(resultOptions, false);
if (OidIsValid(fdwvalidator))
{
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index f80a9e96a9..37115e546b 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -458,7 +458,7 @@ transientrel_startup(DestReceiver *self, int operation, TupleDesc typeinfo)
myState->transientrel = transientrel;
myState->output_cid = GetCurrentCommandId(true);
myState->ti_options = TABLE_INSERT_SKIP_FSM | TABLE_INSERT_FROZEN;
- myState->bistate = GetBulkInsertState();
+ myState->bistate = GetBulkInsertState(NULL);
/*
* Valid smgr_targblock implies something already wrote to the relation.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d71bfdb7c6..8ef0434ebb 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -35,6 +35,7 @@
#include "catalog/objectaccess.h"
#include "catalog/partition.h"
#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_depend.h"
@@ -168,6 +169,8 @@ typedef struct AlteredTableInfo
List *afterStmts; /* List of utility command parsetrees */
bool verify_new_notnull; /* T if we should recheck NOT NULL */
bool new_notnull; /* T if we added new NOT NULL constraints */
+ HTAB *preservedAmInfo; /* Hash table for preserved compression
+ * methods */
int rewrite; /* Reason for forced rewrite, if any */
Oid newTableSpace; /* new tablespace; 0 means no change */
bool chgPersistence; /* T if SET LOGGED/UNLOGGED is used */
@@ -394,6 +397,8 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid);
+static void set_column_compression_relid(Oid acoid, Oid relid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,6 +536,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, ColumnCompression *compression, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -585,6 +592,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -859,8 +867,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression = GetAttributeCompression(attr,
- colDef->compression);
+ attr->attcompression = CreateAttributeCompression(attr,
+ colDef->compression, NULL, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -930,6 +938,23 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ {
+ set_column_compression_relid(attr->attcompression, relationId);
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2386,16 +2411,15 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
- char *compression = GetCompressionName(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2432,7 +2456,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionName(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2645,14 +2669,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression, newdef->compression))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -3799,6 +3818,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4306,6 +4326,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4699,6 +4720,11 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5141,7 +5167,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (newrel)
{
mycid = GetCurrentCommandId(true);
- bistate = GetBulkInsertState();
+ bistate = GetBulkInsertState(tab->preservedAmInfo);
ti_options = TABLE_INSERT_SKIP_FSM;
}
else
@@ -5463,6 +5489,24 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
FreeExecutorState(estate);
+ /* Remove old compression options */
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ {
+ AttrCmPreservedInfo *pinfo;
+ HASH_SEQ_STATUS status;
+
+ Assert(tab->preservedAmInfo);
+ hash_seq_init(&status, tab->preservedAmInfo);
+ while ((pinfo = (AttrCmPreservedInfo *) hash_seq_search(&status)) != NULL)
+ {
+ CleanupAttributeCompression(tab->relid, pinfo->attnum,
+ pinfo->preserved_amoids);
+ list_free(pinfo->preserved_amoids);
+ }
+
+ hash_destroy(tab->preservedAmInfo);
+ }
+
table_close(oldrel, NoLock);
if (newrel)
{
@@ -6176,12 +6220,13 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* create attribute compresssion record */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ attribute.attcompression = CreateAttributeCompression(&attribute,
+ colDef->compression,
+ NULL, NULL);
else
attribute.attcompression = InvalidOid;
- /* attribute.attacl is handled by InsertPgAttributeTuples() */
+ /* attribute.attacl is handled by InsertPgAttributeTuple() */
ReleaseSysCache(typeTuple);
@@ -6352,6 +6397,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6477,6 +6523,30 @@ add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid)
recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
}
+/* Update acrelid in pg_attr_compression */
+static void
+set_column_compression_relid(Oid acoid, Oid relid)
+{
+ Relation rel;
+ HeapTuple tuple;
+ Form_pg_attr_compression acform;
+
+ Assert(OidIsValid(relid));
+ if (IsBuiltinCompression(acoid))
+ return;
+
+ rel = table_open(AttrCompressionRelationId, RowExclusiveLock);
+ tuple = SearchSysCache1(ATTCOMPRESSIONOID, ObjectIdGetDatum(acoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for attribute compression %u", acoid);
+
+ acform = (Form_pg_attr_compression) GETSTRUCT(tuple);
+ acform->acrelid = relid;
+ CatalogTupleUpdate(rel, &tuple->t_self, tuple);
+ ReleaseSysCache(tuple);
+ table_close(rel, RowExclusiveLock);
+}
+
/*
* Install a column's dependency on its collation.
*/
@@ -6499,6 +6569,106 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression options on its column.
+ *
+ * If it is builtin attribute compression then column depends on it. This
+ * is used to determine connection between column and builtin attribute
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid acoid)
+{
+ bool found = false;
+ Relation depRel;
+ ObjectAddress acref,
+ attref;
+ HeapTuple depTup;
+ ScanKeyData key[3];
+ SysScanDesc scan;
+
+ if (!OidIsValid(acoid))
+ return;
+
+ Assert(relid > 0 && attnum > 0);
+ ObjectAddressSet(acref, AttrCompressionRelationId, acoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ depRel = table_open(DependRelationId, AccessShareLock);
+
+ if (IsBuiltinCompression(acoid))
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->refclassid == AttrCompressionRelationId &&
+ foundDep->refobjid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+ }
+ else
+ {
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_refclassid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_refobjid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(relid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_refobjsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(depRel, DependReferenceIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(depTup = systable_getnext(scan)))
+ {
+ Form_pg_depend foundDep = (Form_pg_depend) GETSTRUCT(depTup);
+
+ if (foundDep->classid == AttrCompressionRelationId &&
+ foundDep->objid == acoid)
+ {
+ found = true;
+ break;
+ }
+ }
+ systable_endscan(scan);
+
+ if (!found)
+ recordDependencyOn(&acref, &attref, DEPENDENCY_INTERNAL);
+ }
+ table_close(depRel, AccessShareLock);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -10591,6 +10761,45 @@ createForeignKeyCheckTriggers(Oid myRelOid, Oid refRelOid,
indexOid, false);
}
+/*
+ * Initialize hash table used to keep rewrite rules for
+ * compression changes in ALTER commands.
+ */
+static void
+setupCompressionRewriteRules(AlteredTableInfo *tab, const char *column,
+ AttrNumber attnum, List *preserved_amoids)
+{
+ bool found;
+ AttrCmPreservedInfo *pinfo;
+
+ Assert(!IsBinaryUpgrade);
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ /* initialize hash for oids */
+ if (tab->preservedAmInfo == NULL)
+ {
+ HASHCTL ctl;
+
+ MemSet(&ctl, 0, sizeof(ctl));
+ ctl.keysize = sizeof(AttrNumber);
+ ctl.entrysize = sizeof(AttrCmPreservedInfo);
+ tab->preservedAmInfo =
+ hash_create("preserved access methods cache", 10, &ctl,
+ HASH_ELEM | HASH_BLOBS);
+ }
+ pinfo = (AttrCmPreservedInfo *) hash_search(tab->preservedAmInfo,
+ &attnum, HASH_ENTER, &found);
+
+ if (found)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter compression of column \"%s\" twice", column),
+ errhint("Remove one of statements from the command.")));
+
+ pinfo->attnum = attnum;
+ pinfo->preserved_amoids = preserved_amoids;
+}
+
/*
* ALTER TABLE DROP CONSTRAINT
*
@@ -11420,6 +11629,12 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
colName)));
break;
+ case OCLASS_ATTR_COMPRESSION:
+
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_INTERNAL);
+ break;
+
case OCLASS_DEFAULT:
/*
@@ -11530,7 +11745,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AttrCompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11624,11 +11840,27 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
- /* Setup attribute compression */
- if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ /* Setup attribute compression for new attribute */
+ if (OidIsValid(attTup->attcompression) && attTup->attstorage == 'p')
+ {
+ /* Set up rewrite of table */
attTup->attcompression = InvalidOid;
- else if (!OidIsValid(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+ }
+ else if (OidIsValid(attTup->attcompression))
+ {
+ /* create new record */
+ ColumnCompression *compression = MakeColumnCompression(attTup->attcompression);
+
+ if (!IsBuiltinCompression(attTup->attcompression))
+ setupCompressionRewriteRules(tab, colName, attOldTup->attnum, NIL);
+
+ attTup->attcompression = CreateAttributeCompression(attTup, compression, NULL, NULL);
+ }
+ else if (attTup->attstorage != 'p')
attTup->attcompression = DefaultCompressionOid;
+ else
+ attTup->attcompression = InvalidOid;
}
else
attTup->attcompression = InvalidOid;
@@ -11641,6 +11873,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14810,6 +15047,86 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode)
+{
+ Oid acoid;
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ bool need_rewrite;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ List *preserved_amoids = NIL;
+ ObjectAddress address;
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* create pg_attr_compression record and its dependencies */
+ acoid = CreateAttributeCompression(atttableform, compression,
+ &need_rewrite, &preserved_amoids);
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, acoid);
+
+ /*
+ * Save list of preserved access methods to cache which will be passed to
+ * toast_insert_or_update
+ */
+ if (need_rewrite)
+ setupCompressionRewriteRules(tab, column, atttableform->attnum,
+ preserved_amoids);
+
+ atttableform->attcompression = acoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSet(address, AttrCompressionRelationId, acoid);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7d71f4fb99..9dcd088e37 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,7 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2950,6 +2950,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(amname);
+ COPY_NODE_FIELD(options);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5632,6 +5644,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 583a7414f0..880ff9cdaf 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2589,7 +2589,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2609,6 +2609,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(amname);
+ COMPARE_NODE_FIELD(options);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3684,6 +3694,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 92452c5fd6..b347341160 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3817,6 +3817,14 @@ raw_expression_tree_walker(Node *node,
/* for now, constraints are ignored */
}
break;
+ case T_ColumnCompression:
+ {
+ ColumnCompression *colcmp = (ColumnCompression *) node;
+
+ if (walker(colcmp->options, context))
+ return true;
+ }
+ break;
case T_IndexElem:
{
IndexElem *indelem = (IndexElem *) node;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index a880a95ac6..2e4ac1375c 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2834,7 +2834,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2852,6 +2852,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(amname);
+ WRITE_NODE_FIELD(options);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4200,6 +4210,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 39f7be1854..54d311db49 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -599,7 +600,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2269,6 +2272,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm> [WITH (<options>)]) */
+ | ALTER opt_column ColId SET alterColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = $5;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3381,7 +3393,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3436,13 +3448,42 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause optCompressionParameters
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionParameters optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->amname = $1;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0638e56cd4..1bdb3ed1b1 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1130,7 +1130,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionName(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index 14d9eb2b5b..0ba8a14a56 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -105,6 +105,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_attr_compression_oid(PG_FUNCTION_ARGS)
+{
+ Oid acoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_attr_compression_oid = acoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..dc4c672250 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -27,6 +27,7 @@
#include "catalog/pg_am.h"
#include "catalog/pg_amop.h"
#include "catalog/pg_amproc.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_auth_members.h"
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
@@ -188,6 +189,17 @@ static const struct cachedesc cacheinfo[] = {
},
16
},
+ {AttrCompressionRelationId, /* ATTCOMPRESSIONOID */
+ AttrCompressionIndexId,
+ 1,
+ {
+ Anum_pg_attr_compression_acoid,
+ 0,
+ 0,
+ 0
+ },
+ 8,
+ },
{AttributeRelationId, /* ATTNAME */
AttributeRelidNameIndexId,
2,
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2eeeea306f..6610a340bf 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -45,6 +45,7 @@
#include "catalog/pg_aggregate_d.h"
#include "catalog/pg_am_d.h"
#include "catalog/pg_attribute_d.h"
+#include "catalog/pg_attr_compression_d.h"
#include "catalog/pg_cast_d.h"
#include "catalog/pg_class_d.h"
#include "catalog/pg_default_acl_d.h"
@@ -8529,9 +8530,16 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "c.amname AS attcmname,\n");
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(c.acoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n"
+ "c.acname AS attcmname,\n");
else
appendPQExpBuffer(q,
+ "NULL AS attcmoptions,\n"
"NULL AS attcmname,\n");
@@ -8556,8 +8564,8 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
"ON a.atttypid = t.oid\n");
if (createWithCompression)
- appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am c "
- "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_attr_compression c "
+ "ON a.attcompression = c.acoid\n");
appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
@@ -8586,10 +8594,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
+ tbinfo->attcompression = NULL;
hasdefaults = false;
for (int j = 0; j < ntups; j++)
@@ -8616,6 +8626,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
@@ -8831,6 +8842,104 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 120000 && dopt->binary_upgrade)
+ {
+ int i_acname;
+ int i_acoid;
+ int i_parsedoptions;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ "SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " (CASE WHEN deptype = 'i' THEN refobjsubid ELSE objsubid END) AS curattnum,"
+ " (CASE WHEN deptype = 'n' THEN attcompression = refobjid"
+ " ELSE attcompression = objid END) AS iscurrent,"
+ " acname, acoid,"
+ " (CASE WHEN acoptions IS NOT NULL"
+ " THEN pg_catalog.array_to_string(ARRAY("
+ " SELECT pg_catalog.quote_ident(option_name) || "
+ " ' ' || pg_catalog.quote_literal(option_value) "
+ " FROM pg_catalog.pg_options_to_table(acoptions) "
+ " ORDER BY option_name"
+ " ), E',\n ')"
+ " ELSE NULL END) AS parsedoptions "
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " OR (d.refclassid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid"
+ " AND d.refobjid = a.attrelid"
+ " AND d.refobjsubid = a.attnum AND d.deptype = 'i'"
+ " AND d.classid = 'pg_attr_compression'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_attr_compression c ON"
+ " (d.deptype = 'i' AND d.objid = c.acoid AND a.attnum = c.acattnum"
+ " AND a.attrelid = c.acrelid) OR"
+ " (d.deptype = 'n' AND d.refobjid = c.acoid AND c.acattnum = 0"
+ " AND c.acrelid = 0)"
+ " WHERE (deptype = 'n' AND d.objid = %d) OR (deptype = 'i' AND d.refobjid = %d)"
+ " ORDER BY curattnum, iscurrent;",
+ tbinfo->dobj.catId.oid, tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_acname = PQfnumber(res, "acname");
+ i_acoid = PQfnumber(res, "acoid");
+ i_parsedoptions = PQfnumber(res, "parsedoptions");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->acname = pg_strdup(PQgetvalue(res, k, i_acname));
+ cmitem->acoid = atooid(PQgetvalue(res, k, i_acoid));
+
+ if (!PQgetisnull(res, k, i_parsedoptions))
+ cmitem->parsedoptions = pg_strdup(PQgetvalue(res, k, i_parsedoptions));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -15694,7 +15803,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
* pg_attr_compression
*/
has_custom_compression = (tbinfo->attcmnames[j] &&
- ((strcmp(tbinfo->attcmnames[j], "pglz") != 0)));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -15745,6 +15855,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
@@ -16186,6 +16299,33 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ if (cminfo->nitems)
+ appendPQExpBuffer(q, "\n-- For binary upgrade, recreate compression metadata on column %s\n",
+ fmtId(tbinfo->attnames[j]));
+
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ appendPQExpBuffer(q,
+ "SELECT binary_upgrade_set_next_attr_compression_oid('%d'::pg_catalog.oid);\n",
+ item->acoid);
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), item->acname);
+
+ if (item->parsedoptions)
+ appendPQExpBuffer(q, "\nWITH (%s);\n", item->parsedoptions);
+ else
+ appendPQExpBuffer(q, ";\n");
+ }
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index f9c87e6c16..a8891cb3f5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,7 +325,9 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmoptions; /* per-attribute current compression options */
char **attcmnames; /* per-attribute current compression method names */
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
@@ -348,6 +350,19 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid acoid; /* attribute compression oid */
+ char *acname; /* compression access method name */
+ char *parsedoptions; /* WITH options */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index ec63662060..be0c14883b 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -657,6 +657,43 @@ my %tests = (
},
},
+ # compression data in binary upgrade mode
+ 'ALTER TABLE test_table_compression ALTER COLUMN ... SET COMPRESSION' => {
+ all_runs => 1,
+ catch_all => 'ALTER TABLE ... commands',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text\E\n
+ \);
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col1\E\n
+ \QSET COMPRESSION pglz;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col2\E\n
+ \QSET COMPRESSION pglz2;\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col3\E\n
+ \QSET COMPRESSION pglz\E\n
+ \QWITH (min_input_size '1000');\E\n
+ .*
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '1000');\E\n
+ \QSELECT binary_upgrade_set_next_attr_compression_oid('\E\d+\Q'::pg_catalog.oid);\E\n
+ \QALTER TABLE dump_test.test_table_compression ALTER COLUMN col4\E\n
+ \QSET COMPRESSION pglz2\E\n
+ \QWITH (min_input_size '2000');\E\n
+ /xms,
+ like => { binary_upgrade => 1, },
+ },
+
'ALTER TABLE ONLY test_table ALTER COLUMN col1 SET STATISTICS 90' => {
create_order => 93,
create_sql =>
@@ -1379,6 +1416,17 @@ my %tests = (
like => { %full_runs, section_pre_data => 1, },
},
+ 'CREATE ACCESS METHOD pglz2' => {
+ all_runs => 1,
+ catch_all => 'CREATE ... commands',
+ create_order => 52,
+ create_sql =>
+ 'CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;',
+ regexp =>
+ qr/CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;/m,
+ like => { %full_runs, section_pre_data => 1, },
+ },
+
'CREATE COLLATION test0 FROM "C"' => {
create_order => 76,
create_sql => 'CREATE COLLATION test0 FROM "C";',
@@ -2553,6 +2601,53 @@ my %tests = (
},
},
+ 'CREATE TABLE test_table_compression' => {
+ create_order => 55,
+ create_sql => 'CREATE TABLE dump_test.test_table_compression (
+ col1 text,
+ col2 text COMPRESSION pglz2,
+ col3 text COMPRESSION pglz WITH (min_input_size \'1000\'),
+ col4 text COMPRESSION pglz2 WITH (min_input_size \'1000\')
+ );',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
+ 'ALTER TABLE test_table_compression' => {
+ create_order => 56,
+ create_sql => 'ALTER TABLE dump_test.test_table_compression
+ ALTER COLUMN col4
+ SET COMPRESSION pglz2
+ WITH (min_input_size \'2000\')
+ PRESERVE (pglz2);',
+ regexp => qr/^
+ \QCREATE TABLE dump_test.test_table_compression (\E\n
+ \s+\Qcol1 text,\E\n
+ \s+\Qcol2 text COMPRESSION pglz2,\E\n
+ \s+\Qcol3 text COMPRESSION pglz WITH (min_input_size '1000'),\E\n
+ \s+\Qcol4 text COMPRESSION pglz2 WITH (min_input_size '2000')\E\n
+ \);
+ /xm,
+ like =>
+ { %full_runs, %dump_test_schema_runs, section_pre_data => 1, },
+ unlike => {
+ binary_upgrade => 1,
+ exclude_dump_test_schema => 1,
+ },
+ },
+
'CREATE STATISTICS extended_stats_no_options' => {
create_order => 97,
create_sql => 'CREATE STATISTICS dump_test.test_ext_stats_no_options
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 5a511865ec..887d4d3491 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1895,9 +1895,14 @@ describeOneTableDetails(const char *schemaname,
tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
{
appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
- " (SELECT c.amname "
- " FROM pg_catalog.pg_am c "
- " WHERE c.oid = a.attcompression) "
+ " (SELECT c.acname || "
+ " (CASE WHEN acoptions IS NULL "
+ " THEN '' "
+ " ELSE '(' || array_to_string(ARRAY(SELECT quote_ident(option_name) || ' ' || quote_literal(option_value)"
+ " FROM pg_options_to_table(acoptions)), ', ') || ')'"
+ " END) "
+ " FROM pg_catalog.pg_attr_compression c "
+ " WHERE c.acoid = a.attcompression) "
" END AS attcmname");
attcompression_col = cols++;
}
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
index c9b074deea..d9f2871712 100644
--- a/src/include/access/cmapi.h
+++ b/src/include/access/cmapi.h
@@ -14,12 +14,12 @@
#define CMAPI_H
#include "postgres.h"
-#include "catalog/pg_am.h"
+#include "catalog/pg_attr_compression.h"
#include "catalog/pg_attribute.h"
#include "nodes/pg_list.h"
#define IsBuiltinCompression(cmid) ((cmid) < FirstBootstrapObjectId)
-#define DefaultCompressionOid (PGLZ_COMPRESSION_AM_OID)
+#define DefaultCompressionOid (PGLZ_AC_OID)
typedef struct CompressionAmRoutine CompressionAmRoutine;
@@ -32,18 +32,24 @@ typedef struct CompressionAmRoutine CompressionAmRoutine;
*/
typedef struct CompressionAmOptions
{
+ Oid acoid; /* Oid of attribute compression,
+ should go always first */
+ Oid amoid; /* Oid of compression access method */
+ List *acoptions; /* Parsed options, used for comparison */
CompressionAmRoutine *amroutine; /* compression access method routine */
+ MemoryContext mcxt;
/* result of cminitstate function will be put here */
void *acstate;
} CompressionAmOptions;
+typedef void (*cmcheck_function) (Form_pg_attribute att, List *options);
typedef struct varlena *(*cmcompress_function)
(CompressionAmOptions *cmoptions, const struct varlena *value);
typedef struct varlena *(*cmdecompress_slice_function)
(CompressionAmOptions *cmoptions, const struct varlena *value,
int32 slicelength);
-typedef void *(*cminitstate_function) (Oid acoid);
+typedef void *(*cminitstate_function) (Oid acoid, List *options);
/*
* API struct for a compression AM.
@@ -63,6 +69,7 @@ struct CompressionAmRoutine
{
NodeTag type;
+ cmcheck_function cmcheck; /* can be NULL */
cminitstate_function cminitstate; /* can be NULL */
cmcompress_function cmcompress;
cmcompress_function cmdecompress;
diff --git a/src/include/access/heapam.h b/src/include/access/heapam.h
index ba77013f64..4c89a2cfa9 100644
--- a/src/include/access/heapam.h
+++ b/src/include/access/heapam.h
@@ -28,6 +28,7 @@
#include "storage/shm_toc.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
+#include "utils/hsearch.h"
/* "options" flag bits for heap_insert */
@@ -131,7 +132,7 @@ extern bool heap_hot_search_buffer(ItemPointer tid, Relation relation,
extern void heap_get_latest_tid(TableScanDesc scan, ItemPointer tid);
extern void setLastTid(const ItemPointer tid);
-extern BulkInsertState GetBulkInsertState(void);
+extern BulkInsertState GetBulkInsertState(HTAB *);
extern void FreeBulkInsertState(BulkInsertState);
extern void ReleaseBulkInsertStatePin(BulkInsertState bistate);
diff --git a/src/include/access/heaptoast.h b/src/include/access/heaptoast.h
index 7169254199..6d35aafa2d 100644
--- a/src/include/access/heaptoast.h
+++ b/src/include/access/heaptoast.h
@@ -96,7 +96,8 @@
* ----------
*/
extern HeapTuple heap_toast_insert_or_update(Relation rel, HeapTuple newtup,
- HeapTuple oldtup, int options);
+ HeapTuple oldtup, int options,
+ HTAB *preserved_am_info);
/* ----------
* heap_toast_delete -
diff --git a/src/include/access/hio.h b/src/include/access/hio.h
index f69a92521b..ad82fa7755 100644
--- a/src/include/access/hio.h
+++ b/src/include/access/hio.h
@@ -30,6 +30,8 @@ typedef struct BulkInsertStateData
{
BufferAccessStrategy strategy; /* our BULKWRITE strategy object */
Buffer current_buf; /* current insertion target page */
+ HTAB *preserved_am_info; /* hash table with preserved compression
+ * methods for attributes */
} BulkInsertStateData;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 4e932bc067..7d1f941b95 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -14,8 +14,10 @@
#ifndef TOAST_HELPER_H
#define TOAST_HELPER_H
+#include "utils/hsearch.h"
#include "utils/rel.h"
+
/*
* Information about one column of a tuple being toasted.
*
@@ -101,7 +103,7 @@ typedef struct
#define TOASTCOL_IGNORE 0x0010
#define TOASTCOL_INCOMPRESSIBLE 0x0020
-extern void toast_tuple_init(ToastTupleContext *ttc);
+extern void toast_tuple_init(ToastTupleContext *ttc, HTAB *preserved_am_info);
extern int toast_tuple_find_biggest_attribute(ToastTupleContext *ttc,
bool for_compression,
bool check_main);
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 94e11e583f..ef16034699 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -34,9 +34,22 @@ typedef struct toast_compress_header_custom
{
int32 vl_len_; /* varlena header (do not touch directly!) */
uint32 info; /* flags (2 high bits) and rawsize */
- Oid cmid; /* Oid from pg_am */
+ Oid cmid; /* Oid from pg_attr_compression */
} toast_compress_header_custom;
+/*
+ * Used to pass to preserved compression access methods from
+ * ALTER TABLE SET COMPRESSION into toast_insert_or_update.
+ */
+typedef struct AttrCmPreservedInfo
+{
+ AttrNumber attnum;
+ List *preserved_amoids;
+} AttrCmPreservedInfo;
+
+HTAB *amoptions_cache;
+MemoryContext amoptions_cache_mcxt;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -116,14 +129,14 @@ extern int toast_open_indexes(Relation toastrel,
extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
-
+extern void init_amoptions_cache(void);
+extern bool attr_compression_options_are_equal(Oid acoid1, Oid acoid2);
/*
* lookup_compression_am_options -
*
* Return cached CompressionAmOptions for specified attribute compression.
*/
-extern void lookup_compression_am_options(Oid acoid,
- CompressionAmOptions *result);
+extern CompressionAmOptions *lookup_compression_am_options(Oid acoid);
/*
* toast_set_compressed_datum_info -
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index 02fecb90f7..8b80dd6850 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -24,6 +24,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_attr_compression_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..e35ddd64d8 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_ATTR_COMPRESSION /* pg_attr_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_ATTR_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index a7e2a9b26b..e1448dd7fd 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -95,6 +95,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_attr_compression_index, 2137, on pg_attr_compression using btree(acoid oid_ops));
+#define AttrCompressionIndexId 2137
+DECLARE_INDEX(pg_attr_compression_relid_attnum_index, 2121, on pg_attr_compression using btree(acrelid oid_ops, acattnum int2_ops));
+#define AttrCompressionRelidAttnumIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attr_compression.dat b/src/include/catalog/pg_attr_compression.dat
new file mode 100644
index 0000000000..f0683083ed
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.dat
@@ -0,0 +1,24 @@
+#----------------------------------------------------------------------
+#
+# pg_attr_compression.dat
+# Initial contents of the pg_attr_compression system relation.
+#
+# Predefined compression options for builtin compression access methods.
+# It is safe to use Oids that equal to Oids of access methods, since
+# these are just numbers and not system Oids.
+#
+# Note that predefined options should have 0 in acrelid and acattnum.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_attr_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ acoid => '4225', acname => 'pglz' },
+{ acoid => '4226', acname => 'zlib' },
+
+]
diff --git a/src/include/catalog/pg_attr_compression.h b/src/include/catalog/pg_attr_compression.h
new file mode 100644
index 0000000000..d6550a0eec
--- /dev/null
+++ b/src/include/catalog/pg_attr_compression.h
@@ -0,0 +1,53 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_attr_compression.h
+ *
+ * Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_attr_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_ATTR_COMPRESSION_H
+#define PG_ATTR_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_am_d.h"
+#include "catalog/pg_attr_compression_d.h"
+
+/* ----------------
+ * pg_attr_compression definition. cpp turns this into
+ * typedef struct FormData_pg_attr_compression
+ * ----------------
+ */
+CATALOG(pg_attr_compression,5555,AttrCompressionRelationId)
+{
+ Oid acoid; /* attribute compression oid */
+ NameData acname; /* name of compression AM */
+ Oid acrelid BKI_DEFAULT(0); /* attribute relation */
+ int16 acattnum BKI_DEFAULT(0); /* attribute number in the relation */
+
+#ifdef CATALOG_VARLEN /* variable-length fields start here */
+ text acoptions[1] BKI_DEFAULT(_null_); /* specific options from WITH */
+#endif
+} FormData_pg_attr_compression;
+
+/* ----------------
+ * Form_pg_attr_compresssion corresponds to a pointer to a tuple with
+ * the format of pg_attr_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_attr_compression *Form_pg_attr_compression;
+
+#ifdef EXPOSE_TO_CLIENT_CODE
+/* builtin attribute compression Oids */
+#define PGLZ_AC_OID (PGLZ_COMPRESSION_AM_OID)
+#define ZLIB_AC_OID (ZLIB_COMPRESSION_AM_OID)
+#endif
+
+#endif /* PG_ATTR_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 9404a24199..902a9fb78c 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7022,6 +7022,9 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2228', descr => 'list of compression methods used by the column',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'regclass text', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -10366,6 +10369,10 @@
proname => 'binary_upgrade_set_missing_value', provolatile => 'v',
proparallel => 'u', prorettype => 'void', proargtypes => 'oid text text',
prosrc => 'binary_upgrade_set_missing_value' },
+{ oid => '4035', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_attr_compression_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_attr_compression_oid' },
# conversion functions
{ oid => '4302',
diff --git a/src/include/catalog/toasting.h b/src/include/catalog/toasting.h
index 51491c4513..8370509822 100644
--- a/src/include/catalog/toasting.h
+++ b/src/include/catalog/toasting.h
@@ -75,6 +75,7 @@ DECLARE_TOAST(pg_trigger, 2336, 2337);
DECLARE_TOAST(pg_ts_dict, 4169, 4170);
DECLARE_TOAST(pg_type, 4171, 4172);
DECLARE_TOAST(pg_user_mapping, 4173, 4174);
+DECLARE_TOAST(pg_attr_compression, 5556, 5558);
/* shared catalogs */
DECLARE_TOAST(pg_authid, 4175, 4176);
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8d0de0babe..0f533314d9 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -133,6 +133,7 @@ extern ObjectAddress AlterUserMapping(AlterUserMappingStmt *stmt);
extern Oid RemoveUserMapping(DropUserMappingStmt *stmt);
extern void CreateForeignTable(CreateForeignTableStmt *stmt, Oid relid);
extern void ImportForeignSchema(ImportForeignSchemaStmt *stmt);
+extern Datum optionListToArray(List *options, bool sorted);
extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
@@ -148,8 +149,15 @@ extern char *get_am_name(Oid amOid);
extern regproc get_am_handler_oid(Oid amOid, char amtype, bool noerror);
/* commands/compressioncmds.c */
-extern char *GetCompressionName(Oid acoid);
-extern Oid GetAttributeCompression(Form_pg_attribute attr, char *compression);
+extern ColumnCompression *MakeColumnCompression(Oid acoid);
+extern Oid CreateAttributeCompression(Form_pg_attribute attr,
+ ColumnCompression *compression,
+ bool *need_rewrite,
+ List **preserved_amoids);
+extern void RemoveAttributeCompression(Oid acoid);
+extern void CheckCompressionMismatch(ColumnCompression *c1,
+ ColumnCompression *c2, const char *attributeName);
+void CleanupAttributeCompression(Oid relid, AttrNumber attnum, List *keepAmOids);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 35df26f978..205ae1210f 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -480,6 +480,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 8182f8edba..8a101466bd 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> WITH (<params>) PRESERVE <compression AMs>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *amname;
+ List *options;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression;
+ ColumnCompression *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -1849,7 +1863,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..b5c2f5da51 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -37,6 +37,7 @@ enum SysCacheIdentifier
AMOPOPID,
AMOPSTRATEGY,
AMPROCNUM,
+ ATTCOMPRESSIONOID,
ATTNAME,
ATTNUM,
AUTHMEMMEMROLE,
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..d3b4095fea 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -1,53 +1,273 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
-- test storages
CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
\d+ cmstoragetest
- Table "public.cmstoragetest"
- Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
- st1 | text | | | | external | pglz | |
- st2 | integer | | | | plain | | |
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
\d+ cmstoragetest
- Table "public.cmstoragetest"
- Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
---------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
- st1 | text | | | | main | pglz | |
- st2 | integer | | | | plain | | |
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
\d+ cmstoragetest
- Table "public.cmstoragetest"
- Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
---------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
- st1 | text | | | | plain | pglz | |
- st2 | integer | | | | plain | | |
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
\d+ cmstoragetest
- Table "public.cmstoragetest"
- Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
- st1 | text | | | | extended | pglz | |
- st2 | integer | | | | plain | | |
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
DROP TABLE cmstoragetest;
-CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 'pg_catalog.pg_attr_compression'::REGCLASS OR
+ classid = 'pg_catalog.pg_attr_compression'::REGCLASS) AND
+ (objid = 'cmtest'::REGCLASS OR refobjid = 'cmtest'::REGCLASS)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1, pglz2
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
INSERT INTO cmdata VALUES(repeat('1234567890',1001));
-- copy with table creation
SELECT * INTO cmmove1 FROM cmdata;
-- we update using datum from different table
-CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
-- copy to existing table
-CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
INSERT INTO cmmove3 SELECT * FROM cmdata;
-- drop original compression information
DROP TABLE cmdata;
--- check data is okdd
+-- check data is ok
SELECT length(f1) FROM cmmove1;
length
--------
@@ -68,13 +288,108 @@ SELECT length(f1) FROM cmmove3;
10010
(2 rows)
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1, pglz2
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS OR acrelid = 'cmtest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: unexpected parameter for zlib: "invalid"
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: invalid input syntax for type integer: "best"
CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
-CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
-ERROR: relation "zlibtest" already exists
-CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
-ERROR: relation "zlibtest" already exists
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '9');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '1');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: zlib dictionary is too small
INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
length
@@ -83,4 +398,7 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
-DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_cm_1.out b/src/test/regress/expected/create_cm_1.out
new file mode 100644
index 0000000000..e38ae0cecf
--- /dev/null
+++ b/src/test/regress/expected/create_cm_1.out
@@ -0,0 +1,407 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+ERROR: cannot drop access method pglz because it is required by the database system
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+ERROR: cannot drop access method pglz1 because other objects depend on it
+DETAIL: column d1 of table droptest depends on access method pglz1
+HINT: Use DROP ... CASCADE to drop the dependent objects too.
+DROP ACCESS METHOD pglz1 CASCADE;
+NOTICE: drop cascades to column d1 of table droptest
+\d+ droptest
+ Table "public.droptest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+---------+-------------+--------------+-------------
+
+DROP TABLE droptest;
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+ pglz2 | 1 |
+(1 row)
+
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+DROP TABLE cmaltertest;
+-- test drop
+DROP ACCESS METHOD pglz2;
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ at1 | integer | | | | plain | | |
+
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------
+(0 rows)
+
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+DROP TABLE cmaltertest;
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ERROR: column data type integer does not support compression
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | external | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | main | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | plain | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+------------------------------------------------+--------------+-------------
+ st1 | text | | | | extended | pglz(min_comp_rate '50', min_input_size '100') | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+ Table "public.cmtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 'pg_catalog.pg_attr_compression'::REGCLASS OR
+ classid = 'pg_catalog.pg_attr_compression'::REGCLASS) AND
+ (objid = 'cmtest'::REGCLASS OR refobjid = 'cmtest'::REGCLASS)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+(1 row)
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+(2 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+(3 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1, pglz2
+(1 row)
+
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(2 rows)
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+NOTICE: cmtest rewrite
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+ length
+--------
+ 10010
+ 10020
+ 10030
+ 10040
+(4 rows)
+
+SELECT pg_column_compression('cmtest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+(1 row)
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+ classid | objsubid | refclassid | refobjsubid | deptype
+---------+----------+------------+-------------+---------
+ 1259 | 1 | 5555 | 0 | n
+ 5555 | 0 | 1259 | 1 | i
+ 5555 | 0 | 1259 | 1 | i
+(3 rows)
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+NOTICE: drop cascades to event trigger notice_on_cmtest_rewrite
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is ok
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+\d+ cmtestlike1
+ Table "public.cmtestlike1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+
+\d+ cmtestlike2
+ Table "public.cmtestlike2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz1 | |
+ f2 | integer | | | | plain | | |
+Inherits: cmexample
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+ Table "public.cmaltertest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+ f2 | text | | | | extended | pglz1 | |
+
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: cannot alter compression of column "f1" twice
+HINT: Remove one of statements from the command.
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz, pglz1, pglz2
+(1 row)
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+ pg_column_compression
+-----------------------
+ pglz, pglz1
+(1 row)
+
+SELECT pg_column_compression('cmaltertest', 'f2');
+ pg_column_compression
+-----------------------
+ pglz1
+(1 row)
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS OR acrelid = 'cmtest'::REGCLASS;
+ acname | acattnum | acoptions
+--------+----------+-----------------------
+ pglz1 | 1 |
+ pglz | 1 | {min_input_size=1000}
+ pglz1 | 2 |
+ pglz1 | 1 |
+(4 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
+ERROR: not built with zlib support
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '9');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '1');
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ERROR: not built with zlib support
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
+ERROR: not built with zlib support
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+ERROR: not built with zlib support
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+(0 rows)
+
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index e3e6634d7e..2f3bc66435 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | attribute compression 4225 | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2073,7 +2074,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(10 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2092,6 +2093,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | attribute compression 4225 | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2101,7 +2103,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(9 rows)
+(10 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 531b298fe3..6fab6cddd8 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4844,9 +4844,10 @@ List of access methods
heap | Table
heap2 | Table
pglz |
+ pglz1 |
spgist | Index
zlib |
-(10 rows)
+(11 rows)
\dA *
List of access methods
@@ -4860,9 +4861,10 @@ List of access methods
heap | Table
heap2 | Table
pglz |
+ pglz1 |
spgist | Index
zlib |
-(10 rows)
+(11 rows)
\dA h*
List of access methods
@@ -4898,9 +4900,10 @@ List of access methods
heap | Table | heap_tableam_handler | heap table access method
heap2 | Table | heap_tableam_handler |
pglz | | pglzhandler | pglz compression access method
+ pglz1 | | pglzhandler |
spgist | Index | spghandler | SP-GiST index access method
zlib | | zlibhandler | zlib compression access method
-(10 rows)
+(11 rows)
\dA+ *
List of access methods
@@ -4914,9 +4917,10 @@ List of access methods
heap | Table | heap_tableam_handler | heap table access method
heap2 | Table | heap_tableam_handler |
pglz | | pglzhandler | pglz compression access method
+ pglz1 | | pglzhandler |
spgist | Index | spghandler | SP-GiST index access method
zlib | | zlibhandler | zlib compression access method
-(10 rows)
+(11 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..6b7f772c95 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -32,6 +32,8 @@ check2_tbl|f
check_tbl|f
circle_tbl|t
city|f
+cmaltertest|f
+cmtest|f
copy_tbl|f
d|f
d_star|f
@@ -104,6 +106,7 @@ pg_aggregate|t
pg_am|t
pg_amop|t
pg_amproc|t
+pg_attr_compression|t
pg_attrdef|t
pg_attribute|t
pg_auth_members|t
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..a598484253 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -1,5 +1,47 @@
+-- test drop
+DROP ACCESS METHOD pglz; --fail
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE droptest(d1 TEXT COMPRESSION pglz1);
+DROP ACCESS METHOD pglz1;
+DROP ACCESS METHOD pglz1 CASCADE;
+\d+ droptest
+DROP TABLE droptest;
+
+CREATE ACCESS METHOD pglz1 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test auto drop of related attribute compression
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+CREATE TABLE cmaltertest (f1 TEXT COMPRESSION pglz2);
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ALTER TABLE cmaltertest DROP COLUMN f1;
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+DROP TABLE cmaltertest;
+
+-- test drop
+DROP ACCESS METHOD pglz2;
+
+-- test alter data type
+CREATE TABLE cmaltertest(at1 TEXT);
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET COMPRESSION pglz1 PRESERVE (pglz);
+SELECT pg_column_compression('cmaltertest', 'at1');
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE INT USING at1::INTEGER;
+\d+ cmaltertest
+SELECT pg_column_compression('cmaltertest', 'at1');
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS;
+ALTER TABLE cmaltertest ALTER COLUMN at1 SET DATA TYPE TEXT;
+SELECT pg_column_compression('cmaltertest', 'at1');
+DROP TABLE cmaltertest;
+
-- test storages
CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st2
+ SET COMPRESSION pglz WITH (min_input_size '100'); -- fail
+ALTER TABLE cmstoragetest ALTER COLUMN st1
+ SET COMPRESSION pglz WITH (min_input_size '100', min_comp_rate '50');
ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
\d+ cmstoragetest
ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
@@ -10,7 +52,76 @@ ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
\d+ cmstoragetest
DROP TABLE cmstoragetest;
-CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+
+-- test PRESERVE
+CREATE TABLE cmtest(f1 TEXT);
+\d+ cmtest
+-- view to check dependencies
+CREATE VIEW cmtest_deps AS
+ SELECT classid, objsubid, refclassid, refobjsubid, deptype
+ FROM pg_depend
+ WHERE (refclassid = 'pg_catalog.pg_attr_compression'::REGCLASS OR
+ classid = 'pg_catalog.pg_attr_compression'::REGCLASS) AND
+ (objid = 'cmtest'::REGCLASS OR refobjid = 'cmtest'::REGCLASS)
+ ORDER by objid, refobjid;
+INSERT INTO cmtest VALUES(repeat('1234567890',1001));
+
+-- one normal dependency
+SELECT * FROM cmtest_deps;
+
+-- check decompression
+SELECT length(f1) FROM cmtest;
+SELECT length(f1) FROM cmtest;
+
+CREATE FUNCTION on_cmtest_rewrite()
+RETURNS event_trigger AS $$
+BEGIN
+ RAISE NOTICE 'cmtest rewrite';
+END;
+$$ LANGUAGE plpgsql;
+
+CREATE EVENT TRIGGER notice_on_cmtest_rewrite ON table_rewrite
+ EXECUTE PROCEDURE on_cmtest_rewrite();
+
+-- no rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1002));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one normal and one internal dependency
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE (pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1003));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- rewrite
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz;
+INSERT INTO cmtest VALUES(repeat('1234567890',1004));
+SELECT length(f1) FROM cmtest;
+SELECT pg_column_compression('cmtest', 'f1');
+-- one nornal dependency
+SELECT * FROM cmtest_deps;
+
+-- no rewrites
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz1 PRESERVE (pglz);
+INSERT INTO cmtest VALUES(repeat('1234567890',1005));
+ALTER TABLE cmtest ALTER COLUMN f1 SET COMPRESSION pglz
+ WITH (min_input_size '1000') PRESERVE (pglz, pglz1);
+INSERT INTO cmtest VALUES(repeat('1234567890',1006));
+-- one nornal dependency and two internal dependencies
+SELECT * FROM cmtest_deps;
+
+-- remove function and related event trigger
+DROP FUNCTION on_cmtest_rewrite CASCADE;
+
+-- test moving
+CREATE TABLE cmdata(f1 text COMPRESSION pglz WITH (first_success_by '10000'));
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
INSERT INTO cmdata VALUES(repeat('1234567890',1001));
@@ -18,28 +129,74 @@ INSERT INTO cmdata VALUES(repeat('1234567890',1001));
SELECT * INTO cmmove1 FROM cmdata;
-- we update using datum from different table
-CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz WITH (min_input_size '100'));
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
-- copy to existing table
-CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz WITH (min_input_size '100'));
INSERT INTO cmmove3 SELECT * FROM cmdata;
-- drop original compression information
DROP TABLE cmdata;
--- check data is okdd
+-- check data is ok
SELECT length(f1) FROM cmmove1;
SELECT length(f1) FROM cmmove2;
SELECT length(f1) FROM cmmove3;
+-- create different types of tables
+CREATE TABLE cmexample (f1 TEXT COMPRESSION pglz1);
+CREATE TABLE cmtestlike1 (LIKE cmexample INCLUDING COMPRESSION);
+CREATE TABLE cmtestlike2 (f2 INT) INHERITS (cmexample);
+
+\d+ cmtestlike1
+\d+ cmtestlike2
+
+-- test two columns
+CREATE TABLE cmaltertest(f1 TEXT, f2 TEXT COMPRESSION pglz1);
+\d+ cmaltertest;
+-- fail, changing one column twice
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz,
+ ALTER COLUMN f1 SET COMPRESSION pglz;
+
+-- with rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz1,
+ ALTER COLUMN f2 SET COMPRESSION pglz PRESERVE (pglz1);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- no rewrite
+ALTER TABLE cmaltertest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (pglz1),
+ ALTER COLUMN f2 SET COMPRESSION pglz2 PRESERVE (pglz1, pglz);
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+-- make pglz2 droppable
+ALTER TABLE cmaltertest ALTER COLUMN f2 SET COMPRESSION pglz1;
+SELECT pg_column_compression('cmaltertest', 'f1');
+SELECT pg_column_compression('cmaltertest', 'f2');
+
+SELECT acname, acattnum, acoptions FROM pg_attr_compression
+ WHERE acrelid = 'cmaltertest'::REGCLASS OR acrelid = 'cmtest'::REGCLASS;
+
-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (invalid 'param'));
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib WITH (level 'best'));
CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
-CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
-CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '9');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (level '1');
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one');
INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+ALTER TABLE zlibtest
+ ALTER COLUMN f1 SET COMPRESSION zlib WITH (dict 'one two') PRESERVE (zlib);
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
-DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
+DROP ACCESS METHOD pglz2;
+DROP VIEW cmtest_deps;
+DROP TABLE cmmove1, cmmove2, cmmove3;
+DROP TABLE cmexample, cmtestlike1, cmtestlike2, zlibtest;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1953697af9..de06e3993f 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -367,6 +367,7 @@ CollectedCommand
CollectedCommandType
ColorTrgm
ColorTrgmInfo
+ColumnCompression
ColumnCompareData
ColumnDef
ColumnIOData
--
2.23.0
v1-0002-Add-support-for-another-built-in-compression-meth.patchapplication/x-patch; name=v1-0002-Add-support-for-another-built-in-compression-meth.patchDownload
From b6e2dd66c5e3006b2ea7b079cab8675e968df801 Mon Sep 17 00:00:00 2001
From: dilip kumar <dilipbalaut@localhost.localdomain>
Date: Thu, 13 Aug 2020 15:57:07 +0530
Subject: [PATCH v1 2/3] Add support for another built-in compression method
(zlib)
---
src/backend/access/compression/Makefile | 2 +-
src/backend/access/compression/cm_zlib.c | 163 +++++++++++++++++++++++
src/include/catalog/pg_am.dat | 3 +
src/include/catalog/pg_proc.dat | 4 +
src/test/regress/expected/create_cm.out | 17 ++-
src/test/regress/expected/psql.out | 12 +-
src/test/regress/sql/create_cm.sql | 10 +-
7 files changed, 204 insertions(+), 7 deletions(-)
create mode 100644 src/backend/access/compression/cm_zlib.c
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
index 14286920d3..7ea5ee2e43 100644
--- a/src/backend/access/compression/Makefile
+++ b/src/backend/access/compression/Makefile
@@ -12,6 +12,6 @@ subdir = src/backend/access/compression
top_builddir = ../../../..
include $(top_builddir)/src/Makefile.global
-OBJS = cm_pglz.o cmapi.o
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..b1ba33e90b
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,163 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+static void *
+zlib_cminitstate(Oid acoid)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ return state;
+}
+
+static struct varlena *
+zlib_cmcompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state->level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ if (state->dictlen > 0)
+ {
+ res = deflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib: %s", zp->msg);
+ }
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *)((char *) tmp + VARHDRSZ_CUSTOM_COMPRESSED);
+
+ do {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_CUSTOM_COMPRESSED);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+static struct varlena *
+zlib_cmdecompress(CompressionAmOptions *cmoptions, const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+ zlib_state *state = (zlib_state *) cmoptions->acstate;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ Assert(VARATT_IS_CUSTOM_COMPRESSED(value));
+ zp->next_in = (void *) ((char *) value + VARHDRSZ_CUSTOM_COMPRESSED);
+ zp->avail_in = VARSIZE(value) - VARHDRSZ_CUSTOM_COMPRESSED;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (res == Z_NEED_DICT && state->dictlen > 0)
+ {
+ res = inflateSetDictionary(zp, state->dict, state->dictlen);
+ if (res != Z_OK)
+ elog(ERROR, "could not set dictionary for zlib");
+ continue;
+ }
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionAmRoutine *routine = makeNode(CompressionAmRoutine);
+
+ routine->cminitstate = zlib_cminitstate;
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 877aa3d7e5..53c2b498ab 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -36,5 +36,8 @@
{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4226', oid_symbol => 'ZLIB_COMPRESSION_AM_OID',
+ descr => 'zlib compression access method',
+ amname => 'zlib', amhandler => 'zlibhandler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d8848f58c9..9404a24199 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -950,6 +950,10 @@
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'zlib compression access method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 84adf0c459..ee091bb01f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -68,4 +68,19 @@ SELECT length(f1) FROM cmmove3;
10010
(2 rows)
-DROP TABLE cmmove1, cmmove2, cmmove3;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 2502b23793..531b298fe3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4845,7 +4845,8 @@ List of access methods
heap2 | Table
pglz |
spgist | Index
-(9 rows)
+ zlib |
+(10 rows)
\dA *
List of access methods
@@ -4860,7 +4861,8 @@ List of access methods
heap2 | Table
pglz |
spgist | Index
-(9 rows)
+ zlib |
+(10 rows)
\dA h*
List of access methods
@@ -4897,7 +4899,8 @@ List of access methods
heap2 | Table | heap_tableam_handler |
pglz | | pglzhandler | pglz compression access method
spgist | Index | spghandler | SP-GiST index access method
-(9 rows)
+ zlib | | zlibhandler | zlib compression access method
+(10 rows)
\dA+ *
List of access methods
@@ -4912,7 +4915,8 @@ List of access methods
heap2 | Table | heap_tableam_handler |
pglz | | pglzhandler | pglz compression access method
spgist | Index | spghandler | SP-GiST index access method
-(9 rows)
+ zlib | | zlibhandler | zlib compression access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index a9ddccdc8f..56501b45b0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -34,4 +34,12 @@ SELECT length(f1) FROM cmmove1;
SELECT length(f1) FROM cmmove2;
SELECT length(f1) FROM cmmove3;
-DROP TABLE cmmove1, cmmove2, cmmove3;
\ No newline at end of file
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: Re: [HACKERS] Custom compression methods
On Thu, Aug 13, 2020 at 5:18 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
There was some question which Robert has asked in this mail, please
find my answer inline. Also, I have a few questions regarding further
splitting up this patch.
On Fri, Jun 19, 2020 at 10:33 PM Robert Haas <robertmhaas@gmail.com> wrote:
- One thing I really don't like about the patch is that it consumes a
bit from infomask2 for a new flag HEAP_HASCUSTOMCOMPRESSED. infomask
bits are at a premium, and there's been no real progress in the decade
plus that I've been hanging around here in clawing back any bit-space.
I think we really need to avoid burning our remaining bits for
anything other than a really critical need, and I don't think I
understand what the need is in this case.
IIUC, the main reason for using this flag is for taking the decision
whether we need any detoasting for this tuple. For example, if we are
rewriting the table because the compression method is changed then if
HEAP_HASCUSTOMCOMPRESSED bit is not set in the tuple header and tuple
length, not tup->t_len > TOAST_TUPLE_THRESHOLD then we don't need to
call heap_toast_insert_or_update function for this tuple. Whereas if
this flag is set then we need to because we might need to uncompress
and compress back using a different compression method. The same is
the case with INSERT into SELECT * FROM.
I might be missing
something, but I'd really strongly suggest looking for a way to get
rid of this. It also invents the concept of a TupleDesc flag, and the
flag invented is TD_ATTR_CUSTOM_COMPRESSED; I'm not sure I see why we
need that, either.
This is also used in a similar way as the above but for the target
table, i.e. if the target table has the custom compressed attribute
then maybe we can not directly insert the tuple because it might have
compressed data which are compressed using the default compression
methods.
- It seems like this kind of approach has a sort of built-in
circularity problem. It means that every place that might need to
detoast a datum needs to be able to access the pg_am catalog. I wonder
if that's actually true. For instance, consider logical decoding. I
guess that can do catalog lookups in general, but can it do them from
the places where detoasting is happening? Moreover, can it do them
with the right snapshot? Suppose we rewrite a table to change the
compression method, then drop the old compression method, then try to
decode a transaction that modified that table before those operations
were performed. As an even more extreme example, suppose we need to
open pg_am, and to do that we have to build a relcache entry for it,
and suppose the relevant pg_class entry had a relacl or reloptions
field that happened to be custom-compressed. Or equally suppose that
any of the various other tables we use when building a relcache entry
had the same kind of problem, especially those that have TOAST tables.
We could just disallow the use of non-default compressors in the
system catalogs, but the benefits mentioned in
/messages/by-id/5541614A.5030208@2ndquadrant.com seem too large to
ignore.- I think it would be awfully appealing if we could find some way of
dividing this great big patch into some somewhat smaller patches. For
example:Patch #1. Add syntax allowing a compression method to be specified,
but the only possible choice is pglz, and the PRESERVE stuff isn't
supported, and changing the value associated with an existing column
isn't supported, but we can add tab-completion support and stuff.Patch #2. Add a second built-in method, like gzip or lz4.
I have already extracted these 2 patches from the main patch set.
But, in these patches, I am still storing the am_oid in the toast
header. I am not sure can we get rid of that at least for these 2
patches? But, then wherever we try to uncompress the tuple we need to
know the tuple descriptor to get the am_oid but I think that is not
possible in all the cases. Am I missing something here?
Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.
Does this make sense to have Patch #3 and Patch #4, without having
Patch #5? I mean why do we need to support rewrite or preserve unless
we have the customer compression methods right? because the build-in
compression method can not be dropped so why do we need to preserve?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: Re: [HACKERS] Custom compression methods
On Mon, Aug 24, 2020 at 2:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
IIUC, the main reason for using this flag is for taking the decision
whether we need any detoasting for this tuple. For example, if we are
rewriting the table because the compression method is changed then if
HEAP_HASCUSTOMCOMPRESSED bit is not set in the tuple header and tuple
length, not tup->t_len > TOAST_TUPLE_THRESHOLD then we don't need to
call heap_toast_insert_or_update function for this tuple. Whereas if
this flag is set then we need to because we might need to uncompress
and compress back using a different compression method. The same is
the case with INSERT into SELECT * FROM.
This doesn't really seem worth it to me. I don't see how we can
justify burning an on-disk bit just to save a little bit of overhead
during a rare maintenance operation. If there's a performance problem
here we need to look for another way of mitigating it. Slowing CLUSTER
and/or VACUUM FULL down by a large amount for this feature would be
unacceptable, but is that really a problem? And if so, can we solve it
without requiring this bit?
something, but I'd really strongly suggest looking for a way to get
rid of this. It also invents the concept of a TupleDesc flag, and the
flag invented is TD_ATTR_CUSTOM_COMPRESSED; I'm not sure I see why we
need that, either.This is also used in a similar way as the above but for the target
table, i.e. if the target table has the custom compressed attribute
then maybe we can not directly insert the tuple because it might have
compressed data which are compressed using the default compression
methods.
I think this is just an in-memory flag, which is much less precious
than an on-disk bit. However, I still wonder whether it's really the
right design. I think that if we offer lz4 we may well want to make it
the default eventually, or perhaps even right away. If that ends up
causing this flag to get set on every tuple all the time, then it
won't really optimize anything.
I have already extracted these 2 patches from the main patch set.
But, in these patches, I am still storing the am_oid in the toast
header. I am not sure can we get rid of that at least for these 2
patches? But, then wherever we try to uncompress the tuple we need to
know the tuple descriptor to get the am_oid but I think that is not
possible in all the cases. Am I missing something here?
I think we should instead use the high bits of the toast size word for
patches #1-#4, as discussed upthread.
Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.Does this make sense to have Patch #3 and Patch #4, without having
Patch #5? I mean why do we need to support rewrite or preserve unless
we have the customer compression methods right? because the build-in
compression method can not be dropped so why do we need to preserve?
I think that patch #3 makes sense because somebody might have a table
that is currently compressed with pglz and they want to switch to lz4,
and I think patch #4 also makes sense because they might want to start
using lz4 for future data but not force a rewrite to get rid of all
the pglz data they've already got. Those options are valuable as soon
as there is more than one possible compression algorithm, even if
they're all built in. Now, as I said upthread, it's also true that you
could do #5 before #3 and #4. I don't think that's insane. But I
prefer it in the other order, because I think having #5 without #3 and
#4 wouldn't be too much fun for users.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: Re: [HACKERS] Custom compression methods
On Thu, 13 Aug 2020 at 17:18, Dilip Kumar <dilipbalaut@gmail.com> wrote:
I have rebased the patch on the latest head and currently, broken into 3 parts.
v1-0001: As suggested by Robert, it provides the syntax support for
setting the compression method for a column while creating a table and
adding columns. However, we don't support changing the compression
method for the existing column. As part of this patch, there is only
one built-in compression method that can be set (pglz). In this, we
have one in-build am (pglz) and the compressed attributes will directly
store the oid of the AM. In this patch, I have removed the
pg_attr_compresion as we don't support changing the compression
for the existing column so we don't need to preserve the old
compressions.
v1-0002: Add another built-in compression method (zlib)
v1:0003: Remaining patch set (nothing is changed except rebase on the
current head, stabilizing check-world and 0001 and 0002 are pulled
out of this)Next, I will be working on separating out the remaining patches as per
the suggestion by Robert.
Thanks for this new feature. Looks promising and very useful, with so
many good compression libraries already available.
I see that with the patch-set, I would be able to create an extension
that defines a PostgreSQL C handler function which assigns all the
required hook function implementations for compressing, decompressing
and validating, etc. In short, I would be able to use a completely
different compression algorithm to compress toast data if I write such
an extension. Correct me if I am wrong with my interpretation.
Just a quick superficial set of review comments ....
A minor re-base is required due to a conflict in a regression test
-------------
In heap_toast_insert_or_update() and in other places, the comments for
new parameter preserved_am_info are missing.
-------------
+toast_compress_datum(Datum value, Oid acoid)
{
struct varlena *tmp = NULL;
int32 valsize;
- CompressionAmOptions cmoptions;
+ CompressionAmOptions *cmoptions = NULL;I think tmp and cmoptions need not be initialized to NULL
-------------
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, amoid, valsize);
return PointerGetDatum(tmp);Any particular reason why is this code put in a new extern function ?
Is there a plan to re-use it ? Otherwise, it's not necessary to do
this.
------------
Also, not sure why "HTAB *amoptions_cache" and "MemoryContext
amoptions_cache_mcxt" aren't static declarations. They are being used
only in toast_internals.c
-----------
The tab-completion doesn't show COMPRESSION :
postgres=# create access method my_method TYPE
INDEX TABLE
postgres=# create access method my_method TYPE
Also, the below syntax also would better be tab-completed so as to
display all the installed compression methods, in line with how we
show all the storage methods like plain,extended,etc:
postgres=# ALTER TABLE lztab ALTER COLUMN t SET COMPRESSION
------------
I could see the differences in compression ratio, and the compression
and decompression speed when I use lz versus zib :
CREATE TABLE zlibtab(t TEXT COMPRESSION zlib WITH (level '4'));
create table lztab(t text);
ALTER TABLE lztab ALTER COLUMN t SET COMPRESSION pglz;
pgg:s2:pg$ time psql -c "\copy zlibtab from text.data"
COPY 13050
real 0m1.344s
user 0m0.031s
sys 0m0.026s
pgg:s2:pg$ time psql -c "\copy lztab from text.data"
COPY 13050
real 0m2.088s
user 0m0.008s
sys 0m0.050s
pgg:s2:pg$ time psql -c "select pg_table_size('zlibtab'::regclass),
pg_table_size('lztab'::regclass)"
pg_table_size | pg_table_size
---------------+---------------
1261568 | 1687552
pgg:s2:pg$ time psql -c "select NULL from zlibtab where t like '0000'"
/dev/null
real 0m0.127s
user 0m0.000s
sys 0m0.002s
pgg:s2:pg$ time psql -c "select NULL from lztab where t like '0000'"
/dev/null
real 0m0.050s
user 0m0.002s
sys 0m0.000s
--
Thanks,
-Amit Khandekar
Huawei Technologies
Re: [HACKERS] Custom compression methods
On Aug 13, 2020, at 4:48 AM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
v1-0001: As suggested by Robert, it provides the syntax support for
setting the compression method for a column while creating a table and
adding columns. However, we don't support changing the compression
method for the existing column. As part of this patch, there is only
one built-in compression method that can be set (pglz). In this, we
have one in-build am (pglz) and the compressed attributes will directly
store the oid of the AM. In this patch, I have removed the
pg_attr_compresion as we don't support changing the compression
for the existing column so we don't need to preserve the old
compressions.
I do not like the way pglz compression is handled in this patch. After upgrading PostgreSQL to the first version with this patch included, pre-existing on-disk compressed data will not include any custom compression Oid in the header, and toast_decompress_datum will notice that and decompress the data directly using pglz_decompress. If the same data were then written back out, perhaps to another table, into a column with no custom compression method defined, it will get compressed by toast_compress_datum using DefaultCompressionOid, which is defined as PGLZ_COMPRESSION_AM_OID. That isn't a proper round-trip for the data, as when it gets re-compressed, the PGLZ_COMPRESSION_AM_OID gets written into the header, which makes the data a bit longer, but also means that it is not byte-for-byte the same as it was, which is counter-intuitive. Given that any given pglz compressed datum now has two totally different formats that might occur on disk, code may have to consider both of them, which increases code complexity, and regression tests will need to be written with coverage for both of them, which increases test complexity. It's also not easy to write the extra tests, as there isn't any way (that I see) to intentionally write out the traditional shorter form from a newer database server; you'd have to do something like a pg_upgrade test where you install an older server to write the older format, upgrade, and then check that the new server can handle it.
The cleanest solution to this would seem to be removal of the compression am's Oid from the header for all compression ams, so that pre-patch written data and post-patch written data look exactly the same. The other solution is to give pglz pride-of-place as the original compression mechanism, and just say that when pglz is the compression method, no Oid gets written to the header, and only when other compression methods are used does the Oid get written. This second option seems closer to the implementation that you already have, because you already handle the decompression of data where the Oid is lacking, so all you have to do is intentionally not write the Oid when compressing using pglz.
Or did I misunderstand your implementation?
—
Mark Dilger
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Wed, Sep 2, 2020 at 4:57 AM Mark Dilger <mark.dilger@enterprisedb.com> wrote:
On Aug 13, 2020, at 4:48 AM, Dilip Kumar <dilipbalaut@gmail.com> wrote:
v1-0001: As suggested by Robert, it provides the syntax support for
setting the compression method for a column while creating a table and
adding columns. However, we don't support changing the compression
method for the existing column. As part of this patch, there is only
one built-in compression method that can be set (pglz). In this, we
have one in-build am (pglz) and the compressed attributes will directly
store the oid of the AM. In this patch, I have removed the
pg_attr_compresion as we don't support changing the compression
for the existing column so we don't need to preserve the old
compressions.I do not like the way pglz compression is handled in this patch. After upgrading PostgreSQL to the first version with this patch included, pre-existing on-disk compressed data will not include any custom compression Oid in the header, and toast_decompress_datum will notice that and decompress the data directly using pglz_decompress. If the same data were then written back out, perhaps to another table, into a column with no custom compression method defined, it will get compressed by toast_compress_datum using DefaultCompressionOid, which is defined as PGLZ_COMPRESSION_AM_OID. That isn't a proper round-trip for the data, as when it gets re-compressed, the PGLZ_COMPRESSION_AM_OID gets written into the header, which makes the data a bit longer, but also means that it is not byte-for-byte the same as it was, which is counter-intuitive. Given that any given pglz compressed datum now has two totally different formats that might occur on disk, code may have to consider both of them, which increases code complexity, and regression tests will need to be written with coverage for both of them, which increases test complexity. It's also not easy to write the extra tests, as there isn't any way (that I see) to intentionally write out the traditional shorter form from a newer database server; you'd have to do something like a pg_upgrade test where you install an older server to write the older format, upgrade, and then check that the new server can handle it.
The cleanest solution to this would seem to be removal of the compression am's Oid from the header for all compression ams, so that pre-patch written data and post-patch written data look exactly the same. The other solution is to give pglz pride-of-place as the original compression mechanism, and just say that when pglz is the compression method, no Oid gets written to the header, and only when other compression methods are used does the Oid get written. This second option seems closer to the implementation that you already have, because you already handle the decompression of data where the Oid is lacking, so all you have to do is intentionally not write the Oid when compressing using pglz.
Or did I misunderstand your implementation?
Thanks for looking into it. Actually, I am planning to change this
patch such that we will use the upper 2 bits of the size field instead
of storing the amoid for the builtin compression methods.
e. g. 0x00 = pglz, 0x01 = zlib, 0x10 = other built-in, 0x11 -> custom
compression method. And when 0x11 is set then only we will store the
amoid in the toast header. I think after a week or two I will make
these changes and post my updated patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: Re: [HACKERS] Custom compression methods
On Mon, Aug 31, 2020 at 10:45 AM Amit Khandekar <amitdkhan.pg@gmail.com> wrote:
On Thu, 13 Aug 2020 at 17:18, Dilip Kumar <dilipbalaut@gmail.com> wrote:
I have rebased the patch on the latest head and currently, broken into 3 parts.
v1-0001: As suggested by Robert, it provides the syntax support for
setting the compression method for a column while creating a table and
adding columns. However, we don't support changing the compression
method for the existing column. As part of this patch, there is only
one built-in compression method that can be set (pglz). In this, we
have one in-build am (pglz) and the compressed attributes will directly
store the oid of the AM. In this patch, I have removed the
pg_attr_compresion as we don't support changing the compression
for the existing column so we don't need to preserve the old
compressions.
v1-0002: Add another built-in compression method (zlib)
v1:0003: Remaining patch set (nothing is changed except rebase on the
current head, stabilizing check-world and 0001 and 0002 are pulled
out of this)Next, I will be working on separating out the remaining patches as per
the suggestion by Robert.Thanks for this new feature. Looks promising and very useful, with so
many good compression libraries already available.
Thanks for looking into it.
I see that with the patch-set, I would be able to create an extension
that defines a PostgreSQL C handler function which assigns all the
required hook function implementations for compressing, decompressing
and validating, etc. In short, I would be able to use a completely
different compression algorithm to compress toast data if I write such
an extension. Correct me if I am wrong with my interpretation.Just a quick superficial set of review comments ....
A minor re-base is required due to a conflict in a regression test
Okay, I will do this.
-------------
In heap_toast_insert_or_update() and in other places, the comments for
new parameter preserved_am_info are missing.-------------
ok
+toast_compress_datum(Datum value, Oid acoid) { struct varlena *tmp = NULL; int32 valsize; - CompressionAmOptions cmoptions; + CompressionAmOptions *cmoptions = NULL;I think tmp and cmoptions need not be initialized to NULL
Right
-------------
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize); - SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ); /* successful compression */ + toast_set_compressed_datum_info(tmp, amoid, valsize); return PointerGetDatum(tmp);Any particular reason why is this code put in a new extern function ?
Is there a plan to re-use it ? Otherwise, it's not necessary to do
this.------------
Also, not sure why "HTAB *amoptions_cache" and "MemoryContext
amoptions_cache_mcxt" aren't static declarations. They are being used
only in toast_internals.c
-----------
The tab-completion doesn't show COMPRESSION :
postgres=# create access method my_method TYPE
INDEX TABLE
postgres=# create access method my_method TYPEAlso, the below syntax also would better be tab-completed so as to
display all the installed compression methods, in line with how we
show all the storage methods like plain,extended,etc:
postgres=# ALTER TABLE lztab ALTER COLUMN t SET COMPRESSION------------
I will fix these comments in the next version of the patch.
I could see the differences in compression ratio, and the compression
and decompression speed when I use lz versus zib :CREATE TABLE zlibtab(t TEXT COMPRESSION zlib WITH (level '4'));
create table lztab(t text);
ALTER TABLE lztab ALTER COLUMN t SET COMPRESSION pglz;pgg:s2:pg$ time psql -c "\copy zlibtab from text.data"
COPY 13050real 0m1.344s
user 0m0.031s
sys 0m0.026spgg:s2:pg$ time psql -c "\copy lztab from text.data"
COPY 13050real 0m2.088s
user 0m0.008s
sys 0m0.050spgg:s2:pg$ time psql -c "select pg_table_size('zlibtab'::regclass),
pg_table_size('lztab'::regclass)"
pg_table_size | pg_table_size
---------------+---------------
1261568 | 1687552pgg:s2:pg$ time psql -c "select NULL from zlibtab where t like '0000'"
/dev/null
real 0m0.127s
user 0m0.000s
sys 0m0.002spgg:s2:pg$ time psql -c "select NULL from lztab where t like '0000'"
/dev/null
real 0m0.050s
user 0m0.002s
sys 0m0.000s
Thanks for testing this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: Re: [HACKERS] Custom compression methods
On Tue, Aug 25, 2020 at 11:20 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 24, 2020 at 2:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
IIUC, the main reason for using this flag is for taking the decision
whether we need any detoasting for this tuple. For example, if we are
rewriting the table because the compression method is changed then if
HEAP_HASCUSTOMCOMPRESSED bit is not set in the tuple header and tuple
length, not tup->t_len > TOAST_TUPLE_THRESHOLD then we don't need to
call heap_toast_insert_or_update function for this tuple. Whereas if
this flag is set then we need to because we might need to uncompress
and compress back using a different compression method. The same is
the case with INSERT into SELECT * FROM.This doesn't really seem worth it to me. I don't see how we can
justify burning an on-disk bit just to save a little bit of overhead
during a rare maintenance operation. If there's a performance problem
here we need to look for another way of mitigating it. Slowing CLUSTER
and/or VACUUM FULL down by a large amount for this feature would be
unacceptable, but is that really a problem? And if so, can we solve it
without requiring this bit?
Okay, if we want to avoid keeping the bit then there are multiple ways
to handle this, but the only thing is none of that will be specific to
those scenarios.
approach1. In ExecModifyTable, we can process the source tuple and see
if any of the varlena attributes is compressed and its stored
compression method is not the same as the target table attribute then
we can decompress it.
approach2. In heap_prepare_insert, always call the
heap_toast_insert_or_update, therein we can check if any of the source
tuple attributes are compressed with different compression methods
then the target table then we can decompress it.
With either of the approach, we have to do this in a generic path
because the source of the tuple is not known, I mean it can be a
output from a function, or the join tuple or a subquery. So in the
attached patch, I have implemented with approach1.
For testing, I have implemented using approach1 as well as using
approach2 and I have checked the performance of the pg_bench to see
whether it impacts the performance of the generic paths or not, but I
did not see any impact.
I have already extracted these 2 patches from the main patch set.
But, in these patches, I am still storing the am_oid in the toast
header. I am not sure can we get rid of that at least for these 2
patches? But, then wherever we try to uncompress the tuple we need to
know the tuple descriptor to get the am_oid but I think that is not
possible in all the cases. Am I missing something here?I think we should instead use the high bits of the toast size word for
patches #1-#4, as discussed upthread.Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.Does this make sense to have Patch #3 and Patch #4, without having
Patch #5? I mean why do we need to support rewrite or preserve unless
we have the customer compression methods right? because the build-in
compression method can not be dropped so why do we need to preserve?I think that patch #3 makes sense because somebody might have a table
that is currently compressed with pglz and they want to switch to lz4,
and I think patch #4 also makes sense because they might want to start
using lz4 for future data but not force a rewrite to get rid of all
the pglz data they've already got. Those options are valuable as soon
as there is more than one possible compression algorithm, even if
they're all built in. Now, as I said upthread, it's also true that you
could do #5 before #3 and #4. I don't think that's insane. But I
prefer it in the other order, because I think having #5 without #3 and
#4 wouldn't be too much fun for users.
Details of the attached patch set
0001: This provides syntax to set the compression method from the
built-in compression method (pglz or zlib). pg_attribute stores the
compression method (char) and there are conversion functions to
convert that compression method to the built-in compression array
index. As discussed up thread the first 2 bits will be storing the
compression method index using that we can directly get the handler
routing using the built-in compression method array.
0002: This patch provides an option to changes the compression method
for an existing column and it will rewrite the table.
Next, I will be working on providing an option to alter the
compression method without rewriting the whole table, basically, we
can provide a preserve list to preserve old compression methods.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v2-0002-alter-table-set-compression.patchapplication/octet-stream; name=v2-0002-alter-table-set-compression.patchDownload
From a667b0d572740a189f8034001d2936b45b0227c0 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 12 Sep 2020 17:19:34 +0530
Subject: [PATCH v2 2/2] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/access/table/toast_helper.c | 25 ++++++++
src/backend/commands/tablecmds.c | 83 +++++++++++++++++++++++++
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/nodes/parsenodes.h | 3 +-
6 files changed, 133 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 35971cd723..7a459cc5b5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..58c13eb7ea 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -145,6 +145,31 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_FREE;
ttc->ttc_flags |= (TOAST_NEEDS_CHANGE | TOAST_NEEDS_FREE);
}
+ /*
+ * Process compressed datum.
+ *
+ * If destination column has different compression method then
+ * untoast the datum.
+ */
+ else if (VARATT_IS_COMPRESSED(new_value))
+ {
+ bool storage_ok;
+
+ storage_ok = (att->attstorage == TYPSTORAGE_MAIN ||
+ att->attstorage == TYPSTORAGE_PLAIN);
+
+ if (!storage_ok ||
+ TOAST_COMPRESS_METHOD(new_value) != att->attcompression)
+ {
+ /* just decompress the value */
+ new_value = detoast_attr(new_value);
+
+ ttc->ttc_values[i] = PointerGetDatum(new_value);
+ ttc->ttc_attr[i].tai_colflags |= TOASTCOL_NEEDS_FREE;
+ ttc->ttc_flags |= (TOAST_NEEDS_CHANGE | TOAST_NEEDS_FREE);
+
+ }
+ }
/*
* Remember the size of this attribute
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 40e5bae41c..d82803e6f8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -533,6 +533,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3855,6 +3857,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4370,6 +4373,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4772,6 +4776,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -14955,6 +14963,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char cmethod;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmethod = GetCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmethod)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmethod;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index ad65072ef2..b82781b9c3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2268,6 +2268,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 96dbe5878a..1863bdcb93 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
--
2.23.0
v2-0001-Built-in-compression-method.patchapplication/octet-stream; name=v2-0001-Built-in-compression-method.patchDownload
From 9f940365200bb0d06a304eb377425eff3bb5ae48 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v2 1/2] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
doc/src/sgml/ref/create_table.sgml | 12 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 39 ++-
src/backend/access/common/indextuple.c | 5 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 3 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/cm_pglz.c | 101 +++++++
src/backend/access/compression/cm_zlib.c | 122 +++++++++
src/backend/access/compression/cmapi.c | 116 ++++++++
src/backend/access/heap/heapam.c | 5 +-
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 3 +
src/backend/commands/tablecmds.c | 67 +++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 25 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 51 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 26 +-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/cmapi.h | 77 ++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/pg_attribute.h | 5 +-
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
56 files changed, 1363 insertions(+), 553 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cm_pglz.c
create mode 100644 src/backend/access/compression/cm_zlib.c
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 087cad184c..40d8f44ecf 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods. Default compression
+ method will be <literal>PGLZ</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..5c82d8c334 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,14 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +467,21 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..2f3a8accd7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,12 +16,14 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+
/*
* This enables de-toasting of index entries. Needed until VACUUM is
* smart enough to rebuild indexes from scratch.
@@ -103,7 +105,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionMethod);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..749113c0eb 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,23 +13,43 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
+#include "access/reloptions.h"
#include "access/heaptoast.h"
#include "access/table.h"
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "miscadmin.h"
#include "utils/fmgroids.h"
+#include "utils/inval.h"
+#include "utils/memutils.h"
#include "utils/rel.h"
#include "utils/snapmgr.h"
+#include "utils/syscache.h"
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,28 +64,31 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cm)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ uint32 cmid;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression if not specified */
+ if (!IsValidCompression(cm))
+ cm = DefaultCompressionMethod;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routine w.r.t the compression method */
+ cmid = GetCompressionMethodID(cm);
+ cmroutine = GetCompressionRoutine(cmid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
+ * We recheck the actual size even if compression reports success,
* because it might be satisfied with having saved as little as one byte
* in the compressed data --- which could turn into a net loss once you
* consider header and alignment padding. Worst case, the compressed
@@ -74,16 +97,12 @@ toast_compress_datum(Datum value)
* only one header byte and no padding if the value is short enough. So
* we insist on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..b5bedb5ad8 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -650,6 +651,8 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+ att->attcompression = InvalidCompressionMethod;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..7ea5ee2e43
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..6b1e35d081
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,101 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "access/toast_internals.h"
+#include "commands/defrem.h"
+#include "common/pg_lzcompress.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..bc942df5bc
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,122 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "access/toast_internals.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *)((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..3fdb3344b0
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,116 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression access methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "fmgr.h"
+#include "access/cmapi.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/pg_am.h"
+#include "commands/defrem.h"
+#include "utils/syscache.h"
+
+/* Built-in compression method routines */
+static struct CompressionRoutine BuiltInCMRoutine[] =
+{
+ {"pglz", pglz_cmcompress, pglz_cmdecompress, pglz_cmdecompress_slice},
+ {"zlib", zlib_cmcompress, zlib_cmdecompress, NULL}
+};
+
+/*
+ * GetCompressionMethod - Get compression method from compression name
+ *
+ * Search the array of built-in compression method and return the compression
+ * method.
+ */
+char
+GetCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ int i;
+
+ /* No compression for PLAIN storage. */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidCompressionMethod;
+
+ /* Fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ /* Search in the built-in compression method array */
+ for (i = 0; i < MAX_BUILTIN_COMPRESSION_METHOD; i++)
+ {
+ if (strcmp(BuiltInCMRoutine[i].cmname, compression) == 0)
+ return GetCompressionMethodFromCMID(i);
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * Get the compression method from id.
+ */
+char
+GetCompressionMethodFromCMID(PGCompressionID cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * Get the compression method id.
+ */
+PGCompressionID
+GetCompressionMethodID(char cmethod)
+{
+ switch(cmethod)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %c", cmethod);
+ }
+}
+
+/*
+ * GetCompressionName - Get the compression method name.
+ */
+char *
+GetCompressionName(char cmethod)
+{
+ if (!IsValidCompression(cmethod))
+ return NULL;
+
+ return BuiltInCMRoutine[GetCompressionMethodID(cmethod)].cmname;
+}
+
+/*
+ * GetCompressionRoutine - Get the compression method handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(PGCompressionID cmid)
+{
+ if (cmid >= MAX_BUILTIN_COMPRESSION_METHOD)
+ elog(ERROR, "Invalid compression method %d", cmid);
+
+ return &BuiltInCMRoutine[cmid];
+}
diff --git a/src/backend/access/heap/heapam.c b/src/backend/access/heap/heapam.c
index 1585861a02..f376047f39 100644
--- a/src/backend/access/heap/heapam.c
+++ b/src/backend/access/heap/heapam.c
@@ -2076,10 +2076,7 @@ heap_prepare_insert(Relation relation, HeapTuple tup, TransactionId xid,
Assert(!HeapTupleHasExternal(tup));
return tup;
}
- else if (HeapTupleHasExternal(tup) || tup->t_len > TOAST_TUPLE_THRESHOLD)
- return heap_toast_insert_or_update(relation, tup, NULL, options);
- else
- return tup;
+ return heap_toast_insert_or_update(relation, tup, NULL, options);
}
/*
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..abe25c05fb 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..d9d3012e65 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..b00c62d828 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/cmapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -374,6 +375,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = typeTup->typstorage;
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ to->attcompression = InvalidCompressionMethod;
ReleaseSysCache(tuple);
@@ -452,6 +454,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = typeTup->typbyval;
to->attalign = typeTup->typalign;
to->attstorage = typeTup->typstorage;
+ to->attcompression = InvalidCompressionMethod;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index eab570a867..40e5bae41c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -23,6 +24,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -41,6 +43,7 @@
#include "catalog/pg_inherits.h"
#include "catalog/pg_namespace.h"
#include "catalog/pg_opclass.h"
+#include "catalog/pg_proc.h"
#include "catalog/pg_tablespace.h"
#include "catalog/pg_trigger.h"
#include "catalog/pg_type.h"
@@ -88,6 +91,7 @@
#include "tcop/utility.h"
#include "utils/acl.h"
#include "utils/builtins.h"
+#include "utils/datum.h"
#include "utils/fmgroids.h"
#include "utils/inval.h"
#include "utils/lsyscache.h"
@@ -852,6 +856,13 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION || relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression = GetCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2391,6 +2402,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (IsValidCompression(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionName(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2425,6 +2452,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionName(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2670,6 +2698,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6211,6 +6252,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7634,6 +7684,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!IsValidCompression(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionMethod;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11711,6 +11766,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!IsValidCompression(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 9ce8f43385..a94881ec9e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..a880a95ac6 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2834,6 +2834,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b16ffb9bf7..ad65072ef2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -599,6 +599,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -634,9 +636,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3373,11 +3375,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3386,8 +3389,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3432,6 +3435,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3661,6 +3672,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15065,6 +15077,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index ec944371dd..4bca335d7e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && IsValidCompression(attribute->attcompression))
+ def->compression = GetCompressionName(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..2f3e39d072 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..95841fbd34 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -152,6 +152,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dce6af09c9..1ff10fcdbe 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -39,6 +39,7 @@
#endif
#include "access/attnum.h"
+#include "access/cmapi.h"
#include "access/sysattr.h"
#include "access/transam.h"
#include "catalog/pg_aggregate_d.h"
@@ -383,6 +384,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -888,6 +890,8 @@ main(int argc, char **argv)
* We rely on dependency information to help us determine a safe order, so
* the initial sort is mostly for cosmetic purposes: we sort by name to
* ensure that logically identical schemas will dump identically.
+ *
+ * If we do a parallel dump, we want the largest tables to go first.
*/
sortDumpableObjectsByTypeName(dobjs, numObjs);
@@ -8476,6 +8480,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8561,6 +8566,16 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "CASE WHEN a.attcompression = 'i' THEN NULL ELSE "
+ "a.attcompression END AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcompression,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8579,7 +8594,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8606,6 +8623,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8634,6 +8652,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15686,6 +15705,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15710,6 +15730,13 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ has_non_default_compression = (tbinfo->attcompression[j] &&
+ (((tbinfo->attcompression[j] != 'p') != 0)));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15745,6 +15772,28 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcompression[j] && has_non_default_compression)
+ {
+ char *cmname;
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ case 'z':
+ cmname = "zlib";
+ }
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..5ba6d99425 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..ed4744b557 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,15 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2028,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2092,6 +2104,18 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'z' ? "zlib" :
+ (compression[0] == 'i' ? "" :
+ "???"))),
+ false, false);
+ }
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9c6f5ecb6a..ad934a40c8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..9511706489
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,77 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+#include "catalog/pg_am.h"
+#include "catalog/pg_attribute.h"
+#include "nodes/pg_list.h"
+
+/* Built-in compression methods */
+#define PGLZ_COMPRESSION 'p'
+#define ZLIB_COMPRESSION 'z'
+#define InvalidCompressionMethod 'i'
+
+/*
+ * Built-in compression method-id. This is direct index into the built-in
+ * array.
+ */
+typedef enum PGCompressionID
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} PGCompressionID;
+
+#define MAX_BUILTIN_COMPRESSION_METHOD 2
+
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+char GetCompressionMethod(Form_pg_attribute att, char *compression);
+char GetCompressionMethodFromCMID(PGCompressionID cmid);
+PGCompressionID GetCompressionMethodID(char cm_method);
+char *GetCompressionName(char cm_method);
+CompressionRoutine *GetCompressionRoutine(PGCompressionID cmid);
+
+/*
+ * buit-in compression method handler function declaration
+ * XXX we can add header files for pglz and zlib and move these
+ * declarations to specific header files.
+ */
+struct varlena *pglz_cmcompress(const struct varlena *value);
+struct varlena *pglz_cmdecompress(const struct varlena *value);
+struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+struct varlena *zlib_cmcompress(const struct varlena *value);
+struct varlena *zlib_cmdecompress(const struct varlena *value);
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..45f90940e0 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..53de29da04 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and
+ 30 bits rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, char cm);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7c6268c22c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options */
+ char attcompression BKI_DEFAULT('i');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..96dbe5878a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 08f22ce211..98c4a78e0a 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index f56615393e..bceb74bb98 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e40287d25a..2d3656eec3 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1c72f23bc9..f0c0d6e247 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1021,21 +1021,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1043,11 +1043,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1076,46 +1076,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1147,11 +1147,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1160,10 +1160,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1173,10 +1173,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 7ac9df767f..326cfeaa2a 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2a18dc423e..3ee0ec8643 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3004,11 +3004,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3024,11 +3024,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: Re: [HACKERS] Custom compression methods
On Sat, Sep 19, 2020 at 1:19 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Aug 25, 2020 at 11:20 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 24, 2020 at 2:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
IIUC, the main reason for using this flag is for taking the decision
whether we need any detoasting for this tuple. For example, if we are
rewriting the table because the compression method is changed then if
HEAP_HASCUSTOMCOMPRESSED bit is not set in the tuple header and tuple
length, not tup->t_len > TOAST_TUPLE_THRESHOLD then we don't need to
call heap_toast_insert_or_update function for this tuple. Whereas if
this flag is set then we need to because we might need to uncompress
and compress back using a different compression method. The same is
the case with INSERT into SELECT * FROM.This doesn't really seem worth it to me. I don't see how we can
justify burning an on-disk bit just to save a little bit of overhead
during a rare maintenance operation. If there's a performance problem
here we need to look for another way of mitigating it. Slowing CLUSTER
and/or VACUUM FULL down by a large amount for this feature would be
unacceptable, but is that really a problem? And if so, can we solve it
without requiring this bit?Okay, if we want to avoid keeping the bit then there are multiple ways
to handle this, but the only thing is none of that will be specific to
those scenarios.
approach1. In ExecModifyTable, we can process the source tuple and see
if any of the varlena attributes is compressed and its stored
compression method is not the same as the target table attribute then
we can decompress it.
approach2. In heap_prepare_insert, always call the
heap_toast_insert_or_update, therein we can check if any of the source
tuple attributes are compressed with different compression methods
then the target table then we can decompress it.With either of the approach, we have to do this in a generic path
because the source of the tuple is not known, I mean it can be a
output from a function, or the join tuple or a subquery. So in the
attached patch, I have implemented with approach1.For testing, I have implemented using approach1 as well as using
approach2 and I have checked the performance of the pg_bench to see
whether it impacts the performance of the generic paths or not, but I
did not see any impact.I have already extracted these 2 patches from the main patch set.
But, in these patches, I am still storing the am_oid in the toast
header. I am not sure can we get rid of that at least for these 2
patches? But, then wherever we try to uncompress the tuple we need to
know the tuple descriptor to get the am_oid but I think that is not
possible in all the cases. Am I missing something here?I think we should instead use the high bits of the toast size word for
patches #1-#4, as discussed upthread.Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.Does this make sense to have Patch #3 and Patch #4, without having
Patch #5? I mean why do we need to support rewrite or preserve unless
we have the customer compression methods right? because the build-in
compression method can not be dropped so why do we need to preserve?I think that patch #3 makes sense because somebody might have a table
that is currently compressed with pglz and they want to switch to lz4,
and I think patch #4 also makes sense because they might want to start
using lz4 for future data but not force a rewrite to get rid of all
the pglz data they've already got. Those options are valuable as soon
as there is more than one possible compression algorithm, even if
they're all built in. Now, as I said upthread, it's also true that you
could do #5 before #3 and #4. I don't think that's insane. But I
prefer it in the other order, because I think having #5 without #3 and
#4 wouldn't be too much fun for users.Details of the attached patch set
0001: This provides syntax to set the compression method from the
built-in compression method (pglz or zlib). pg_attribute stores the
compression method (char) and there are conversion functions to
convert that compression method to the built-in compression array
index. As discussed up thread the first 2 bits will be storing the
compression method index using that we can directly get the handler
routing using the built-in compression method array.0002: This patch provides an option to changes the compression method
for an existing column and it will rewrite the table.Next, I will be working on providing an option to alter the
compression method without rewriting the whole table, basically, we
can provide a preserve list to preserve old compression methods.
I have rebased the patch and I have also done a couple of defect fixes
and some cleanup.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v3-0002-alter-table-set-compression.patchapplication/octet-stream; name=v3-0002-alter-table-set-compression.patchDownload
From f6eb5963a5fde099b385dafa04e884c87aed2c42 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 26 Sep 2020 16:25:19 +0530
Subject: [PATCH v3 2/2] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 ++++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
7 files changed, 113 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index f034e75ec0..8f0284689f 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8e7b608cc6..38c76dba3c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3855,6 +3857,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4370,6 +4373,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4772,6 +4776,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5396,6 +5404,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ ExecCompareCompressionMethod(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14955,6 +14966,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char cmethod;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmethod = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmethod)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmethod;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0d0082278b..0c005624e4 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2013,7 +2013,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 94ebf40e03..fb8b597b18 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 415e117407..18a672eb20 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -602,5 +602,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 96dbe5878a..1863bdcb93 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
--
2.23.0
v3-0001-Built-in-compression-method.patchapplication/octet-stream; name=v3-0001-Built-in-compression-method.patchDownload
From e42fc74c31998632854cae931c6d41522f76eb1b Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v3 1/2] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 39 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 72 +++--
src/backend/access/common/tupdesc.c | 6 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/cm_pglz.c | 114 ++++++++
src/backend/access/compression/cm_zlib.c | 129 +++++++++
src/backend/access/compression/cmapi.c | 106 +++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 4 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 50 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 28 +-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/cmapi.h | 79 ++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/pg_attribute.h | 5 +-
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
57 files changed, 1511 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cm_pglz.c
create mode 100644 src/backend/access/compression/cm_zlib.c
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 087cad184c..9a3afc3e68 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..50ff66d0fc 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,14 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +467,21 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..d9ab65114e 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionMethod);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..65070a9886 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
@@ -30,6 +31,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +59,45 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cm)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ uint32 cmid;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!IsValidCompression(cm))
+ cm = DefaultCompressionMethod;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmid = GetCompressionMethodId(cm);
+ cmroutine = GetCompressionRoutine(cmid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..ad39fd81a8 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,8 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = (att->attstorage == TYPSTORAGE_PLAIN) ?
+ InvalidCompressionMethod : DefaultCompressionMethod;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..7ea5ee2e43
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..07b7034738
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,114 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/cmapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..fd8879cc72
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,129 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "access/toast_internals.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..b0db088e4b
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,106 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "access/cmapi.h"
+
+/* Compression routines for the built-in compression methods. */
+static struct CompressionRoutine BuiltInCompressionRoutines[] =
+{
+ {"pglz", pglz_cmcompress, pglz_cmdecompress, pglz_cmdecompress_slice},
+ {"zlib", zlib_cmcompress, zlib_cmdecompress, NULL}
+};
+
+/*
+ * GetCompressionMethod - Get compression method from compression name
+ *
+ * Search the array of built-in compression methods array and return the
+ * compression method. If the compression name is not found in the buil-in
+ * compression array then return invalid compression method.
+ */
+char
+GetCompressionMethod(char *compression)
+{
+ int i;
+
+ /* Search in the built-in compression method array */
+ for (i = 0; i < MAX_BUILTIN_COMPRESSION_METHOD; i++)
+ {
+ if (strcmp(BuiltInCompressionRoutines[i].cmname, compression) == 0)
+ return GetCompressionMethodFromCompressionId(i);
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+char
+GetCompressionMethodFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodId - Translate the compression method id to compression
+ * method.
+ */
+CompressionId
+GetCompressionMethodId(char cm)
+{
+ switch (cm)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %c", cm);
+ }
+}
+
+/*
+ * GetCompressionName - Get the name compression method.
+ */
+char *
+GetCompressionName(char cm)
+{
+ if (!IsValidCompression(cm))
+ return NULL;
+
+ return BuiltInCompressionRoutines[GetCompressionMethodId(cm)].cmname;
+}
+
+/*
+ * GetCompressionRoutine - Get the compression method handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(CompressionId cmid)
+{
+ if (cmid >= MAX_BUILTIN_COMPRESSION_METHOD)
+ elog(ERROR, "Invalid compression method %d", cmid);
+
+ return &BuiltInCompressionRoutines[cmid];
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..909705d86b 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression =
+ (attrtypes[attnum]->attstorage == TYPSTORAGE_PLAIN) ?
+ InvalidCompressionMethod : DefaultCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..d9d3012e65 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2ec6d23411 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/cmapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index eab570a867..8e7b608cc6 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -558,7 +559,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static char GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -852,6 +854,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2391,6 +2402,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (IsValidCompression(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionName(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2425,6 +2452,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionName(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2670,6 +2698,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6211,6 +6252,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7634,6 +7684,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!IsValidCompression(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionMethod;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11711,6 +11766,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!IsValidCompression(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17571,3 +17638,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static char
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ char cm;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidCompressionMethod;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cm = GetCompressionMethod(compression);
+ if (!IsValidCompression(cm))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cm;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9812089161..0d0082278b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/cmapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2005,6 +2008,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionMethodFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2236,6 +2310,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ ExecCompareCompressionMethod(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 9ce8f43385..a94881ec9e 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index e2f177515d..a880a95ac6 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2834,6 +2834,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a7..94ebf40e03 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15066,6 +15078,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15584,6 +15597,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index ec944371dd..4bca335d7e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && IsValidCompression(attribute->attcompression))
+ def->compression = GetCompressionName(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..2f3e39d072 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 1017abbbe5..95841fbd34 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -152,6 +152,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f021bb72f4..ba2841f120 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -382,6 +382,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8465,6 +8466,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8550,6 +8552,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcompression,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8568,7 +8579,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8595,6 +8608,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8623,6 +8637,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15671,6 +15686,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15695,6 +15711,13 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ has_non_default_compression = (tbinfo->attcompression[j] &&
+ (((tbinfo->attcompression[j] != 'p') != 0)));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15730,6 +15753,31 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0' &&
+ has_non_default_compression)
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ case 'z':
+ cmname = "zlib";
+ }
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..484a240c50 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..4050bca831 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,15 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2028,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2092,6 +2104,18 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'z' ? "zlib" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
@@ -2744,7 +2768,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9c6f5ecb6a..ad934a40c8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..097e755eee
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,79 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. Pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define ZLIB_COMPRESSION 'z'
+#define InvalidCompressionMethod '\0'
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+#define MAX_BUILTIN_COMPRESSION_METHOD ZLIB_COMPRESSION_ID + 1
+
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern char GetCompressionMethod(char *compression);
+extern char GetCompressionMethodFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionMethodId(char cm_method);
+extern char *GetCompressionName(char cm_method);
+extern CompressionRoutine *GetCompressionRoutine(CompressionId cmid);
+
+/*
+ * buit-in compression method handler function declaration
+ * XXX we can add header files for pglz and zlib and move these
+ * declarations to specific header files.
+ */
+extern struct varlena *pglz_cmcompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+extern struct varlena *zlib_cmcompress(const struct varlena *value);
+extern struct varlena *zlib_cmdecompress(const struct varlena *value);
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..45f90940e0 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..eef44b61c8 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, char cm);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..5135992b5c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..96dbe5878a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index f56615393e..bceb74bb98 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e40287d25a..2d3656eec3 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1c72f23bc9..f0c0d6e247 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1021,21 +1021,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1043,11 +1043,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1076,46 +1076,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1147,11 +1147,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1160,10 +1160,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1173,10 +1173,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 7ac9df767f..326cfeaa2a 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 2a18dc423e..3ee0ec8643 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3004,11 +3004,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3024,11 +3024,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: Re: [HACKERS] Custom compression methods
On Mon, Sep 28, 2020 at 4:18 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Sep 19, 2020 at 1:19 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Aug 25, 2020 at 11:20 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Aug 24, 2020 at 2:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
IIUC, the main reason for using this flag is for taking the decision
whether we need any detoasting for this tuple. For example, if we are
rewriting the table because the compression method is changed then if
HEAP_HASCUSTOMCOMPRESSED bit is not set in the tuple header and tuple
length, not tup->t_len > TOAST_TUPLE_THRESHOLD then we don't need to
call heap_toast_insert_or_update function for this tuple. Whereas if
this flag is set then we need to because we might need to uncompress
and compress back using a different compression method. The same is
the case with INSERT into SELECT * FROM.This doesn't really seem worth it to me. I don't see how we can
justify burning an on-disk bit just to save a little bit of overhead
during a rare maintenance operation. If there's a performance problem
here we need to look for another way of mitigating it. Slowing CLUSTER
and/or VACUUM FULL down by a large amount for this feature would be
unacceptable, but is that really a problem? And if so, can we solve it
without requiring this bit?Okay, if we want to avoid keeping the bit then there are multiple ways
to handle this, but the only thing is none of that will be specific to
those scenarios.
approach1. In ExecModifyTable, we can process the source tuple and see
if any of the varlena attributes is compressed and its stored
compression method is not the same as the target table attribute then
we can decompress it.
approach2. In heap_prepare_insert, always call the
heap_toast_insert_or_update, therein we can check if any of the source
tuple attributes are compressed with different compression methods
then the target table then we can decompress it.With either of the approach, we have to do this in a generic path
because the source of the tuple is not known, I mean it can be a
output from a function, or the join tuple or a subquery. So in the
attached patch, I have implemented with approach1.For testing, I have implemented using approach1 as well as using
approach2 and I have checked the performance of the pg_bench to see
whether it impacts the performance of the generic paths or not, but I
did not see any impact.I have already extracted these 2 patches from the main patch set.
But, in these patches, I am still storing the am_oid in the toast
header. I am not sure can we get rid of that at least for these 2
patches? But, then wherever we try to uncompress the tuple we need to
know the tuple descriptor to get the am_oid but I think that is not
possible in all the cases. Am I missing something here?I think we should instead use the high bits of the toast size word for
patches #1-#4, as discussed upthread.Patch #3. Add support for changing the compression method associated
with a column, forcing a table rewrite.Patch #4. Add support for PRESERVE, so that you can change the
compression method associated with a column without forcing a table
rewrite, by including the old method in the PRESERVE list, or with a
rewrite, by not including it in the PRESERVE list.Does this make sense to have Patch #3 and Patch #4, without having
Patch #5? I mean why do we need to support rewrite or preserve unless
we have the customer compression methods right? because the build-in
compression method can not be dropped so why do we need to preserve?I think that patch #3 makes sense because somebody might have a table
that is currently compressed with pglz and they want to switch to lz4,
and I think patch #4 also makes sense because they might want to start
using lz4 for future data but not force a rewrite to get rid of all
the pglz data they've already got. Those options are valuable as soon
as there is more than one possible compression algorithm, even if
they're all built in. Now, as I said upthread, it's also true that you
could do #5 before #3 and #4. I don't think that's insane. But I
prefer it in the other order, because I think having #5 without #3 and
#4 wouldn't be too much fun for users.Details of the attached patch set
0001: This provides syntax to set the compression method from the
built-in compression method (pglz or zlib). pg_attribute stores the
compression method (char) and there are conversion functions to
convert that compression method to the built-in compression array
index. As discussed up thread the first 2 bits will be storing the
compression method index using that we can directly get the handler
routing using the built-in compression method array.0002: This patch provides an option to changes the compression method
for an existing column and it will rewrite the table.Next, I will be working on providing an option to alter the
compression method without rewriting the whole table, basically, we
can provide a preserve list to preserve old compression methods.I have rebased the patch and I have also done a couple of defect fixes
and some cleanup.
Here is the next patch which allows providing a PRESERVE list using
this we can avoid table rewrite while altering the compression method.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v3-0001-Built-in-compression-method.patchapplication/octet-stream; name=v3-0001-Built-in-compression-method.patchDownload
From af31c597f7a56cd615e212133037e60643f4c9c3 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v3 1/3] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 39 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 72 +++--
src/backend/access/common/tupdesc.c | 6 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/cm_pglz.c | 114 ++++++++
src/backend/access/compression/cm_zlib.c | 129 +++++++++
src/backend/access/compression/cmapi.c | 106 +++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 4 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 50 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 28 +-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/cmapi.h | 79 ++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/pg_attribute.h | 5 +-
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
57 files changed, 1511 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/cm_pglz.c
create mode 100644 src/backend/access/compression/cm_zlib.c
create mode 100644 src/backend/access/compression/cmapi.c
create mode 100644 src/include/access/cmapi.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 28f844071b..38275e9cd4 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..50ff66d0fc 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,14 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +467,21 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..d9ab65114e 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionMethod);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..65070a9886 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
@@ -30,6 +31,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +59,45 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cm)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ uint32 cmid;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!IsValidCompression(cm))
+ cm = DefaultCompressionMethod;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmid = GetCompressionMethodId(cm);
+ cmroutine = GetCompressionRoutine(cmid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..ad39fd81a8 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,8 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = (att->attstorage == TYPSTORAGE_PLAIN) ?
+ InvalidCompressionMethod : DefaultCompressionMethod;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..7ea5ee2e43
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = cm_pglz.o cm_zlib.o cmapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/cm_pglz.c b/src/backend/access/compression/cm_pglz.c
new file mode 100644
index 0000000000..07b7034738
--- /dev/null
+++ b/src/backend/access/compression/cm_pglz.c
@@ -0,0 +1,114 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/cmapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
diff --git a/src/backend/access/compression/cm_zlib.c b/src/backend/access/compression/cm_zlib.c
new file mode 100644
index 0000000000..fd8879cc72
--- /dev/null
+++ b/src/backend/access/compression/cm_zlib.c
@@ -0,0 +1,129 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cm_zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/cmapi.h"
+#include "access/toast_internals.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
diff --git a/src/backend/access/compression/cmapi.c b/src/backend/access/compression/cmapi.c
new file mode 100644
index 0000000000..b0db088e4b
--- /dev/null
+++ b/src/backend/access/compression/cmapi.c
@@ -0,0 +1,106 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/cmapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/cmapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "access/cmapi.h"
+
+/* Compression routines for the built-in compression methods. */
+static struct CompressionRoutine BuiltInCompressionRoutines[] =
+{
+ {"pglz", pglz_cmcompress, pglz_cmdecompress, pglz_cmdecompress_slice},
+ {"zlib", zlib_cmcompress, zlib_cmdecompress, NULL}
+};
+
+/*
+ * GetCompressionMethod - Get compression method from compression name
+ *
+ * Search the array of built-in compression methods array and return the
+ * compression method. If the compression name is not found in the buil-in
+ * compression array then return invalid compression method.
+ */
+char
+GetCompressionMethod(char *compression)
+{
+ int i;
+
+ /* Search in the built-in compression method array */
+ for (i = 0; i < MAX_BUILTIN_COMPRESSION_METHOD; i++)
+ {
+ if (strcmp(BuiltInCompressionRoutines[i].cmname, compression) == 0)
+ return GetCompressionMethodFromCompressionId(i);
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+char
+GetCompressionMethodFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodId - Translate the compression method id to compression
+ * method.
+ */
+CompressionId
+GetCompressionMethodId(char cm)
+{
+ switch (cm)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %c", cm);
+ }
+}
+
+/*
+ * GetCompressionName - Get the name compression method.
+ */
+char *
+GetCompressionName(char cm)
+{
+ if (!IsValidCompression(cm))
+ return NULL;
+
+ return BuiltInCompressionRoutines[GetCompressionMethodId(cm)].cmname;
+}
+
+/*
+ * GetCompressionRoutine - Get the compression method handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(CompressionId cmid)
+{
+ if (cmid >= MAX_BUILTIN_COMPRESSION_METHOD)
+ elog(ERROR, "Invalid compression method %d", cmid);
+
+ return &BuiltInCompressionRoutines[cmid];
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..909705d86b 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression =
+ (attrtypes[attnum]->attstorage == TYPSTORAGE_PLAIN) ?
+ InvalidCompressionMethod : DefaultCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..d9d3012e65 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2ec6d23411 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/cmapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e0ac4e05e5..5ef45b9b04 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/cmapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static char GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2392,6 +2403,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (IsValidCompression(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionName(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2426,6 +2453,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionName(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2671,6 +2699,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6213,6 +6254,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7636,6 +7686,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!IsValidCompression(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionMethod;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11713,6 +11768,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!IsValidCompression(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17577,3 +17644,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static char
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ char cm;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidCompressionMethod;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cm = GetCompressionMethod(compression);
+ if (!IsValidCompression(cm))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cm;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9812089161..0d0082278b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/cmapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2005,6 +2008,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionMethodFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2236,6 +2310,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ ExecCompareCompressionMethod(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0386480ab..718c2ed7fd 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,6 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a7..94ebf40e03 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15066,6 +15078,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15584,6 +15597,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..c305ad3692 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/cmapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && IsValidCompression(attribute->attcompression))
+ def->compression = GetCompressionName(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..2f3e39d072 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 76320468ba..dca6edc6df 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcompression,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8612,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8641,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15675,6 +15690,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15699,6 +15715,13 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ has_non_default_compression = (tbinfo->attcompression[j] &&
+ (((tbinfo->attcompression[j] != 'p') != 0)));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15734,6 +15757,31 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0' &&
+ has_non_default_compression)
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ case 'z':
+ cmname = "zlib";
+ }
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..484a240c50 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..4050bca831 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,15 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2028,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2092,6 +2104,18 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'z' ? "zlib" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
@@ -2744,7 +2768,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 24c7b414cf..fe2ea885a6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/cmapi.h b/src/include/access/cmapi.h
new file mode 100644
index 0000000000..097e755eee
--- /dev/null
+++ b/src/include/access/cmapi.h
@@ -0,0 +1,79 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/cmapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef CMAPI_H
+#define CMAPI_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. Pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define ZLIB_COMPRESSION 'z'
+#define InvalidCompressionMethod '\0'
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+#define MAX_BUILTIN_COMPRESSION_METHOD ZLIB_COMPRESSION_ID + 1
+
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern char GetCompressionMethod(char *compression);
+extern char GetCompressionMethodFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionMethodId(char cm_method);
+extern char *GetCompressionName(char cm_method);
+extern CompressionRoutine *GetCompressionRoutine(CompressionId cmid);
+
+/*
+ * buit-in compression method handler function declaration
+ * XXX we can add header files for pglz and zlib and move these
+ * declarations to specific header files.
+ */
+extern struct varlena *pglz_cmcompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+extern struct varlena *zlib_cmcompress(const struct varlena *value);
+extern struct varlena *zlib_cmdecompress(const struct varlena *value);
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..45f90940e0 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..eef44b61c8 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, char cm);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..5135992b5c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..96dbe5878a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression;
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e40287d25a..2d3656eec3 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 7ac9df767f..326cfeaa2a 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index af4192f9a8..64f66e2dc3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3007,11 +3007,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3027,11 +3027,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3058,11 +3058,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v3-0002-alter-table-set-compression.patchapplication/octet-stream; name=v3-0002-alter-table-set-compression.patchDownload
From 51f38edd0b8624017548450962c03046be2972d7 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 26 Sep 2020 16:25:19 +0530
Subject: [PATCH v3 2/3] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 +++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++++
src/test/regress/sql/create_cm.sql | 6 ++
9 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5ef45b9b04..6c35211945 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3856,6 +3858,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4371,6 +4374,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4774,6 +4778,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5398,6 +5406,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ ExecCompareCompressionMethod(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14957,6 +14968,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char cmethod;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmethod = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmethod)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmethod;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0d0082278b..0c005624e4 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2013,7 +2013,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 94ebf40e03..fb8b597b18 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 415e117407..18a672eb20 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -602,5 +602,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 96dbe5878a..1863bdcb93 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..0052085cb3 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -83,4 +83,19 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..b8949c8630 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -42,4 +42,10 @@ INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v3-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v3-0003-Add-support-for-PRESERVE.patchDownload
From 1b65367daa02d81926ff9c18a609265a0d66d14b Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sun, 27 Sep 2020 14:32:13 +0530
Subject: [PATCH v3 3/3] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 7 +-
src/backend/access/common/tupdesc.c | 8 +-
src/backend/access/table/toast_helper.c | 4 +-
src/backend/bootstrap/bootstrap.c | 7 +-
src/backend/catalog/heap.c | 4 +-
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 138 ++++++++++++++++++++++++
src/backend/commands/tablecmds.c | 120 +++++++++------------
src/backend/executor/nodeModifyTable.c | 11 +-
src/backend/nodes/copyfuncs.c | 16 ++-
src/backend/nodes/equalfuncs.c | 14 ++-
src/backend/nodes/outfuncs.c | 14 ++-
src/backend/parser/gram.y | 44 ++++++--
src/backend/parser/parse_utilcmd.c | 6 +-
src/include/catalog/pg_attribute.h | 10 +-
src/include/commands/defrem.h | 7 ++
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 15 ++-
src/test/regress/expected/create_cm.out | 9 ++
src/test/regress/sql/create_cm.sql | 4 +
20 files changed, 334 insertions(+), 106 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..152fb8de93 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -391,7 +391,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or partial
+ list of compression methods the table will be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ad39fd81a8..aa5b18c0c9 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -24,6 +24,7 @@
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
@@ -471,7 +472,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
- if (attr1->attcompression != attr2->attcompression)
+ if (strcmp(NameStr(attr1->attcompression), NameStr(attr2->attcompression)) != 0)
return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -667,8 +668,9 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
- att->attcompression = (att->attstorage == TYPSTORAGE_PLAIN) ?
- InvalidCompressionMethod : DefaultCompressionMethod;
+
+ /* initialize the attribute compression field based on the storage type */
+ InitAttributeCompression(att);
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..829ebb29d0 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,7 +54,9 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
- ttc->ttc_attr[i].tai_compression = att->attcompression;
+
+ /* Current compression method is stored in the first character */
+ ttc->ttc_attr[i].tai_compression = *(NameStr(att->attcompression));
if (ttc->ttc_oldvalues != NULL)
{
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 909705d86b..079921eda0 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -28,6 +28,7 @@
#include "catalog/index.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/link-canary.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
@@ -732,9 +733,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
- attrtypes[attnum]->attcompression =
- (attrtypes[attnum]->attstorage == TYPSTORAGE_PLAIN) ?
- InvalidCompressionMethod : DefaultCompressionMethod;
+
+ /* initialize the attribute compression field based on the storage type */
+ InitAttributeCompression(attrtypes[attnum]);
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index d9d3012e65..6080ba74ef 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -781,7 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
- slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = NameGetDatum(&attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1706,7 +1706,7 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
- attStruct->attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attStruct->attcompression), 0, NAMEDATALEN);
/*
* Change the column name to something that isn't likely to conflict
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..2c0f38673b
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,138 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/cmapi.h"
+#include "commands/defrem.h"
+
+
+/*
+ * CheckCompressionMethodsPreserved - Compare new compression method list with
+ * the existing list.
+ *
+ * Check whether all the previous compression methods are preserved in the new
+ * compression methods or not.
+ */
+static bool
+CheckCompressionMethodsPreserved(char *previous_cm, char *new_cm)
+{
+ int i = 0;
+
+ while (previous_cm[i] != 0)
+ {
+ if (strchr(new_cm, previous_cm[i]) == NULL)
+ return false;
+ i++;
+ }
+
+ return true;
+}
+
+/*
+ * GetAttributeCompression - Get compression for given attribute
+ */
+char *
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ ListCell *cell;
+ char *cm;
+ int preserve_idx = 0;
+
+ cm = palloc0(NAMEDATALEN);
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return NULL;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ {
+ cm[preserve_idx] = DefaultCompressionMethod;
+ return cm;
+ }
+
+ /* current compression method */
+ cm[preserve_idx] = GetCompressionMethod(compression->cmname);
+ if (!IsValidCompression(cm[preserve_idx]))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ preserve_idx++;
+
+ foreach (cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ char cm_p = GetCompressionMethod(cmname_p);
+
+ /*
+ * The compression method given in the preserve list must present in
+ * the existing compression methods for the attribute.
+ */
+ if (strchr(NameStr(att->attcompression), cm_p) == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+ cm[preserve_idx++] = cm_p;
+ }
+
+ /*
+ * If all the previous compression methods are preserved then we don't need
+ * to rewrite the table otherwise we need to.
+ */
+ if (need_rewrite)
+ {
+ if (CheckCompressionMethodsPreserved(NameStr(att->attcompression), cm))
+ *need_rewrite = false;
+ else
+ *need_rewrite = true;
+ }
+
+ return cm;
+}
+
+/*
+ * InitAttributeCompression - initialize compression method in pg_attribute
+ *
+ * If attribute storage is plain then initialize with the invalid compression
+ * otherwise initialize it with the default compression method.
+ */
+void
+InitAttributeCompression(Form_pg_attribute att)
+{
+ char *attcompression = NameStr(att->attcompression);
+
+ MemSet(NameStr(att->attcompression), 0, NAMEDATALEN);
+ if (att->attstorage != TYPSTORAGE_PLAIN)
+ attcompression[0] = DefaultCompressionMethod;
+}
+
+/*
+ * MakeColumnCompression - Construct ColumnCompression node.
+ */
+ColumnCompression *
+MakeColumnCompression(NameData *compression)
+{
+ ColumnCompression *node;
+ char cm = *(NameStr(*compression));
+
+ if (!IsValidCompression(cm))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionName(cm);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 6c35211945..1574e9fa35 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -531,7 +531,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static char GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -862,10 +862,16 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ {
+ char *cm = GetAttributeCompression(attr, colDef->compression, NULL);
+
+ if (cm == NULL)
+ MemSet(NameStr(attr->attcompression), 0, NAMEDATALEN);
+ else
+ namestrcpy(&attr->attcompression, cm);
+ }
else
- attr->attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attr->attcompression), 0, NAMEDATALEN);
}
/*
@@ -2406,19 +2412,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(attribute->attstorage))));
/* Copy/check compression parameter */
- if (IsValidCompression(attribute->attcompression))
+ if (IsValidCompression(*(NameStr(attribute->attcompression))))
{
- char *compression =
- GetCompressionName(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(&attribute->attcompression);
- if (!def->compression)
+ if (def->compression == NULL)
def->compression = compression;
- else if (strcmp(def->compression, compression))
- ereport(ERROR,
+ else if (strcmp(def->compression->cmname, compression->cmname))
+ ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
+ errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2455,7 +2461,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionName(attribute->attcompression);
+ def->compression =
+ MakeColumnCompression(&attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2706,12 +2713,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4779,7 +4786,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6269,10 +6277,17 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* create attribute compresssion record */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ {
+ char *compression =
+ GetAttributeCompression(&attribute, colDef->compression, NULL);
+
+ if (compression == NULL)
+ MemSet(NameStr(attribute.attcompression), 0, NAMEDATALEN);
+ else
+ namestrcpy(&(attribute.attcompression), compression);
+ }
else
- attribute.attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attribute.attcompression), 0, NAMEDATALEN);
/* attribute.attacl is handled by InsertPgAttributeTuples() */
@@ -7698,9 +7713,9 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
format_type_be(attrtuple->atttypid))));
/* use default compression if storage is not PLAIN */
- if (!IsValidCompression(attrtuple->attcompression) &&
+ if (!IsValidCompression(*NameStr(attrtuple->attcompression)) &&
(newstorage != TYPSTORAGE_PLAIN))
- attrtuple->attcompression = DefaultCompressionMethod;
+ InitAttributeCompression(attrtuple);
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
@@ -11783,13 +11798,10 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
/* Setup attribute compression */
- if (attTup->attstorage == TYPSTORAGE_PLAIN)
- attTup->attcompression = InvalidCompressionMethod;
- else if (!IsValidCompression(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ InitAttributeCompression(attTup);
}
else
- attTup->attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attTup->attcompression), 0, NAMEDATALEN);
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
@@ -14972,23 +14984,17 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
- char cmethod;
- Datum values[Natts_pg_attribute];
- bool nulls[Natts_pg_attribute];
- bool replace[Natts_pg_attribute];
+ char *cm;
+ bool need_rewrite;
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15007,24 +15013,20 @@ ATExecSetCompression(AlteredTableInfo *tab,
errmsg("cannot alter system column \"%s\"", column)));
/* Prevent from altering untoastable attributes with PLAIN storage */
- if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ if (atttableform->attstorage == TYPSTORAGE_PLAIN &&
+ !TypeIsToastable(atttableform->atttypid))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("column data type %s does not support compression",
format_type_be(atttableform->atttypid))));
- /* Initialize buffers for new tuple values */
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replace, false, sizeof(replace));
-
/* Get the attribute compression method. */
- cmethod = GetAttributeCompressionMethod(atttableform, compression);
-
- if (atttableform->attcompression != cmethod)
+ cm = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
- atttableform->attcompression = cmethod;
+ namestrcpy(&atttableform->attcompression, cm);
+
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -17730,27 +17732,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * GetAttributeCompressionMethod - Get compression method from compression name
- */
-static char
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- char cm;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidCompressionMethod;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionMethod;
-
- cm = GetCompressionMethod(compression);
- if (!IsValidCompression(cm))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cm;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0c005624e4..d6745959f9 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2050,11 +2051,13 @@ ExecCompareCompressionMethod(TupleTableSlot *slot, TupleDesc targetTupDesc)
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Check whether the source value's compression method is supported
+ * in the target table's attribute or not. If it is not supported
+ * then decompress the value.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionMethodFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ if (strchr(NameStr(targetTupDesc->attrs[i].attcompression),
+ GetCompressionMethodFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value))) == NULL)
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 37fe5c72e9..aac399427c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,7 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2950,6 +2950,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5631,6 +5642,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..35efa064e5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3692,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 718c2ed7fd..a1b5302d5b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,7 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2853,6 +2853,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4201,6 +4210,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index fb8b597b18..29ff3ad0f2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,35 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index c305ad3692..8faea29f17 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1065,9 +1065,9 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->storage = 0;
/* Likewise, copy compression if requested */
- if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
- && IsValidCompression(attribute->attcompression))
- def->compression = GetCompressionName(attribute->attcompression);
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION)
+ def->compression =
+ MakeColumnCompression(&attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 5135992b5c..a7ba8aac9f 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,8 +156,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
- /* attribute's compression options */
- char attcompression BKI_DEFAULT('\0');
+ /*
+ * Attribute's compression options, the first character represents the
+ * current compression method and followed by all old preserved compression
+ * methods.
+ */
+ NameData attcompression BKI_DEFAULT('\0');
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -186,7 +190,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute, attcompression) + NAMEDATALEN)
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..97e6130284 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern char *GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern void InitAttributeCompression(Form_pg_attribute att);
+extern ColumnCompression *MakeColumnCompression(NameData *compression);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..fed9cb7994 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 1863bdcb93..be99691f4a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression;
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 0052085cb3..33c7132353 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -98,4 +98,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index b8949c8630..abf34eede0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -48,4 +48,8 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: [HACKERS] Custom compression methods
Hi,
I took a look at this patch after a long time, and done a bit of a
review+testing. I haven't re-read the whole thread since 2017 so some of
the following comments might be mistaken - sorry about that :-(
1) The "cmapi.h" naming seems unnecessarily short. I'd suggest using
simply compression or something like that. I see little reason to
shorten "compression" to "cm", or to prefix files with "cm_". For
example compression/cm_zlib.c might just be compression/zlib.c.
2) I see index_form_tuple does this:
Datum cvalue = toast_compress_datum(untoasted_values[i],
DefaultCompressionMethod);
which seems wrong - why shouldn't the indexes use the same compression
method as the underlying table?
3) dumpTableSchema in pg_dump.c does this:
switch (tbinfo->attcompression[j])
{
case 'p':
cmname = "pglz";
case 'z':
cmname = "zlib";
}
which is broken as it's missing break, so 'p' will produce 'zlib'.
4) The name ExecCompareCompressionMethod is somewhat misleading, as the
functions is not merely comparing compression methods - it also
recompresses the data.
5) CheckCompressionMethodsPreserved should document what the return
value is (true when new list contains all old values, thus not requiring
a rewrite). Maybe "Compare" would be a better name?
6) The new field in ColumnDef is missing a comment.
7) It's not clear to me what "partial list" in the PRESERVE docs means.
+ which of them should be kept on the column. Without PRESERVE or partial
+ list of compression methods the table will be rewritten.8) The initial synopsis in alter_table.sgml includes the PRESERVE
syntax, but then later in the page it's omitted (yet the section talks
about the keyword).
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.
11) pg_column_compression
When specifying compression method not present in attcompression, we get
this error message and hint:
test=# alter table t alter COLUMN a set compression "pglz" preserve (zlib);
ERROR: "zlib" compression access method cannot be preserved
HINT: use "pg_column_compression" function for list of compression methods
but there is no pg_column_compression function, so the hint is wrong.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.
The reason for using the NameData is the get it in the fixed part of
the data structure.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.
Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".
I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.
Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?
Other comments look fine to me so I will work on them and post the
updated patch set.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Oct 5, 2020 at 11:17 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?Other comments look fine to me so I will work on them and post the
updated patch set.
I have fixed the other comments except this,
2) I see index_form_tuple does this:
Datum cvalue = toast_compress_datum(untoasted_values[i],
DefaultCompressionMethod);
which seems wrong - why shouldn't the indexes use the same compression
method as the underlying table?
I will fix this in the next version of the patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v4-0001-Built-in-compression-method.patchapplication/octet-stream; name=v4-0001-Built-in-compression-method.patchDownload
From 4941d3a4d6233147fd74a9d3da66c7bf96c30d36 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v4 1/3] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 39 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 72 +++--
src/backend/access/common/tupdesc.c | 6 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 106 +++++++
src/backend/access/compression/pglz.c | 114 ++++++++
src/backend/access/compression/zlib.c | 129 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 4 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 54 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 28 +-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 79 ++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/pg_attribute.h | 5 +-
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
57 files changed, 1515 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/backend/access/compression/zlib.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 28f844071b..38275e9cd4 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..340b7ef85a 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,14 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +467,21 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(TOAST_COMPRESS_METHOD(attr));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..abe99ede2c 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionMethod);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..038683a393 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
@@ -30,6 +31,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +59,45 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cm)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ uint32 cmid;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!IsValidCompression(cm))
+ cm = DefaultCompressionMethod;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmid = GetCompressionMethodId(cm);
+ cmroutine = GetCompressionRoutine(cmid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, cmid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..fc37d344fd 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,8 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = (att->attstorage == TYPSTORAGE_PLAIN) ?
+ InvalidCompressionMethod : DefaultCompressionMethod;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..4883d114cf
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o zlib.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..23996d6e7c
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,106 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+
+/* Compression routines for the built-in compression methods. */
+static struct CompressionRoutine BuiltInCompressionRoutines[] =
+{
+ {"pglz", pglz_cmcompress, pglz_cmdecompress, pglz_cmdecompress_slice},
+ {"zlib", zlib_cmcompress, zlib_cmdecompress, NULL}
+};
+
+/*
+ * GetCompressionMethod - Get compression method from compression name
+ *
+ * Search the array of built-in compression methods array and return the
+ * compression method. If the compression name is not found in the buil-in
+ * compression array then return invalid compression method.
+ */
+char
+GetCompressionMethod(char *compression)
+{
+ int i;
+
+ /* Search in the built-in compression method array */
+ for (i = 0; i < MAX_BUILTIN_COMPRESSION_METHOD; i++)
+ {
+ if (strcmp(BuiltInCompressionRoutines[i].cmname, compression) == 0)
+ return GetCompressionMethodFromCompressionId(i);
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+char
+GetCompressionMethodFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodId - Translate the compression method id to compression
+ * method.
+ */
+CompressionId
+GetCompressionMethodId(char cm)
+{
+ switch (cm)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %c", cm);
+ }
+}
+
+/*
+ * GetCompressionName - Get the name compression method.
+ */
+char *
+GetCompressionName(char cm)
+{
+ if (!IsValidCompression(cm))
+ return NULL;
+
+ return BuiltInCompressionRoutines[GetCompressionMethodId(cm)].cmname;
+}
+
+/*
+ * GetCompressionRoutine - Get the compression method handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(CompressionId cmid)
+{
+ if (cmid >= MAX_BUILTIN_COMPRESSION_METHOD)
+ elog(ERROR, "Invalid compression method %d", cmid);
+
+ return &BuiltInCompressionRoutines[cmid];
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..480844c928
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,114 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
diff --git a/src/backend/access/compression/zlib.c b/src/backend/access/compression/zlib.c
new file mode 100644
index 0000000000..38d966e389
--- /dev/null
+++ b/src/backend/access/compression/zlib.c
@@ -0,0 +1,129 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..04b1899099 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression =
+ (attrtypes[attnum]->attstorage == TYPSTORAGE_PLAIN) ?
+ InvalidCompressionMethod : DefaultCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..0c510805a2 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2228ad9dad 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e0ac4e05e5..d80aeb875d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static char GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2392,6 +2403,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (IsValidCompression(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionName(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2426,6 +2453,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionName(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2671,6 +2699,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6213,6 +6254,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7636,6 +7686,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!IsValidCompression(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionMethod;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11713,6 +11768,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!IsValidCompression(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17577,3 +17644,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static char
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ char cm;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidCompressionMethod;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cm = GetCompressionMethod(compression);
+ if (!IsValidCompression(cm))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cm;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9812089161..2559018a32 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2005,6 +2008,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionMethodFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2236,6 +2310,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CheckTargetCMAndDecompress(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0386480ab..718c2ed7fd 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,6 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 17653ef3a7..94ebf40e03 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15066,6 +15078,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15584,6 +15597,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..9b3233e052 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && IsValidCompression(attribute->attcompression))
+ def->compression = GetCompressionName(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..2f3e39d072 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 76320468ba..f5dfb4cae8 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcompression,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,9 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8612,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8641,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15675,6 +15690,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15699,6 +15715,13 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ /*
+ * Compression will require a record in
+ * pg_attr_compression
+ */
+ has_non_default_compression = (tbinfo->attcompression[j] &&
+ (((tbinfo->attcompression[j] != 'p') != 0)));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15734,6 +15757,35 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0' &&
+ has_non_default_compression)
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'z':
+ cmname = "zlib";
+ break;
+ default:
+ cmname = NULL;
+ }
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..484a240c50 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..4050bca831 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,15 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2028,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2092,6 +2104,18 @@ describeOneTableDetails(const char *schemaname,
"???")))),
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'z' ? "zlib" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
@@ -2744,7 +2768,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 24c7b414cf..fe2ea885a6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..7d35a2b222
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,79 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. Pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define ZLIB_COMPRESSION 'z'
+#define InvalidCompressionMethod '\0'
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+#define MAX_BUILTIN_COMPRESSION_METHOD ZLIB_COMPRESSION_ID + 1
+
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern char GetCompressionMethod(char *compression);
+extern char GetCompressionMethodFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionMethodId(char cm_method);
+extern char *GetCompressionName(char cm_method);
+extern CompressionRoutine *GetCompressionRoutine(CompressionId cmid);
+
+/*
+ * buit-in compression method handler function declaration
+ * XXX we can add header files for pglz and zlib and move these
+ * declarations to specific header files.
+ */
+extern struct varlena *pglz_cmcompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+extern struct varlena *zlib_cmcompress(const struct varlena *value);
+extern struct varlena *zlib_cmdecompress(const struct varlena *value);
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..45f90940e0 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..eef44b61c8 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, char cm);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..5135992b5c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression options */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index e40287d25a..2d3656eec3 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -464,10 +464,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 7ac9df767f..326cfeaa2a 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index af4192f9a8..64f66e2dc3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3007,11 +3007,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3027,11 +3027,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3058,11 +3058,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v4-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v4-0003-Add-support-for-PRESERVE.patchDownload
From d5f74659594e25a7cee79ce0725dd1b1c2491b2c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 5 Oct 2020 12:46:24 +0530
Subject: [PATCH v4 3/3] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 9 +-
src/backend/access/common/tupdesc.c | 8 +-
src/backend/access/table/toast_helper.c | 4 +-
src/backend/bootstrap/bootstrap.c | 7 +-
src/backend/catalog/heap.c | 4 +-
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 137 ++++++++++++++++++++++++
src/backend/commands/tablecmds.c | 120 +++++++++------------
src/backend/executor/nodeModifyTable.c | 11 +-
src/backend/nodes/copyfuncs.c | 16 ++-
src/backend/nodes/equalfuncs.c | 14 ++-
src/backend/nodes/outfuncs.c | 14 ++-
src/backend/parser/gram.y | 44 ++++++--
src/backend/parser/parse_utilcmd.c | 6 +-
src/include/catalog/pg_attribute.h | 10 +-
src/include/commands/defrem.h | 7 ++
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 15 ++-
src/test/regress/expected/create_cm.out | 9 ++
src/test/regress/sql/create_cm.sql | 4 +
20 files changed, 334 insertions(+), 107 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..c8e0c4985c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or partial
+ list of compression methods the table will be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index fc37d344fd..5da043067b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -24,6 +24,7 @@
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
@@ -471,7 +472,7 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
- if (attr1->attcompression != attr2->attcompression)
+ if (strcmp(NameStr(attr1->attcompression), NameStr(attr2->attcompression)) != 0)
return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -667,8 +668,9 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
- att->attcompression = (att->attstorage == TYPSTORAGE_PLAIN) ?
- InvalidCompressionMethod : DefaultCompressionMethod;
+
+ /* initialize the attribute compression field based on the storage type */
+ InitAttributeCompression(att);
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..829ebb29d0 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,7 +54,9 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
- ttc->ttc_attr[i].tai_compression = att->attcompression;
+
+ /* Current compression method is stored in the first character */
+ ttc->ttc_attr[i].tai_compression = *(NameStr(att->attcompression));
if (ttc->ttc_oldvalues != NULL)
{
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 04b1899099..c4585fa340 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -28,6 +28,7 @@
#include "catalog/index.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/link-canary.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
@@ -732,9 +733,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
- attrtypes[attnum]->attcompression =
- (attrtypes[attnum]->attstorage == TYPSTORAGE_PLAIN) ?
- InvalidCompressionMethod : DefaultCompressionMethod;
+
+ /* initialize the attribute compression field based on the storage type */
+ InitAttributeCompression(attrtypes[attnum]);
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 0c510805a2..8eb0bb0f1a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -781,7 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
- slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = NameGetDatum(&attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1706,7 +1706,7 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
- attStruct->attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attStruct->attcompression), 0, NAMEDATALEN);
/*
* Change the column name to something that isn't likely to conflict
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..dd38bce9a3
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,137 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "commands/defrem.h"
+
+
+/*
+ * CheckCompressionMethodsPreserved - Compare new compression method list with
+ * the existing list.
+ *
+ * Check whether all the previous compression methods are preserved in the new
+ * compression methods or not.
+ */
+static bool
+CheckCompressionMethodsPreserved(char *previous_cm, char *new_cm)
+{
+ int i = 0;
+
+ while (previous_cm[i] != 0)
+ {
+ if (strchr(new_cm, previous_cm[i]) == NULL)
+ return false;
+ i++;
+ }
+
+ return true;
+}
+
+/*
+ * GetAttributeCompression - Get compression for given attribute
+ */
+char *
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ ListCell *cell;
+ char *cm;
+ int preserve_idx = 0;
+
+ cm = palloc0(NAMEDATALEN);
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return NULL;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ {
+ cm[preserve_idx] = DefaultCompressionMethod;
+ return cm;
+ }
+
+ /* current compression method */
+ cm[preserve_idx] = GetCompressionMethod(compression->cmname);
+ if (!IsValidCompression(cm[preserve_idx]))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ preserve_idx++;
+
+ foreach (cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ char cm_p = GetCompressionMethod(cmname_p);
+
+ /*
+ * The compression method given in the preserve list must present in
+ * the existing compression methods for the attribute.
+ */
+ if (strchr(NameStr(att->attcompression), cm_p) == NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression method cannot be preserved", cmname_p)));
+ cm[preserve_idx++] = cm_p;
+ }
+
+ /*
+ * If all the previous compression methods are preserved then we don't need
+ * to rewrite the table otherwise we need to.
+ */
+ if (need_rewrite)
+ {
+ if (CheckCompressionMethodsPreserved(NameStr(att->attcompression), cm))
+ *need_rewrite = false;
+ else
+ *need_rewrite = true;
+ }
+
+ return cm;
+}
+
+/*
+ * InitAttributeCompression - initialize compression method in pg_attribute
+ *
+ * If attribute storage is plain then initialize with the invalid compression
+ * otherwise initialize it with the default compression method.
+ */
+void
+InitAttributeCompression(Form_pg_attribute att)
+{
+ char *attcompression = NameStr(att->attcompression);
+
+ MemSet(NameStr(att->attcompression), 0, NAMEDATALEN);
+ if (att->attstorage != TYPSTORAGE_PLAIN)
+ attcompression[0] = DefaultCompressionMethod;
+}
+
+/*
+ * MakeColumnCompression - Construct ColumnCompression node.
+ */
+ColumnCompression *
+MakeColumnCompression(NameData *compression)
+{
+ ColumnCompression *node;
+ char cm = *(NameStr(*compression));
+
+ if (!IsValidCompression(cm))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionName(cm);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 53ab899ba6..cd006199bf 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -531,7 +531,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static char GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -862,10 +862,16 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ {
+ char *cm = GetAttributeCompression(attr, colDef->compression, NULL);
+
+ if (cm == NULL)
+ MemSet(NameStr(attr->attcompression), 0, NAMEDATALEN);
+ else
+ namestrcpy(&attr->attcompression, cm);
+ }
else
- attr->attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attr->attcompression), 0, NAMEDATALEN);
}
/*
@@ -2406,19 +2412,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(attribute->attstorage))));
/* Copy/check compression parameter */
- if (IsValidCompression(attribute->attcompression))
+ if (IsValidCompression(*(NameStr(attribute->attcompression))))
{
- char *compression =
- GetCompressionName(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(&attribute->attcompression);
- if (!def->compression)
+ if (def->compression == NULL)
def->compression = compression;
- else if (strcmp(def->compression, compression))
- ereport(ERROR,
+ else if (strcmp(def->compression->cmname, compression->cmname))
+ ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
+ errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2455,7 +2461,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionName(attribute->attcompression);
+ def->compression =
+ MakeColumnCompression(&attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2706,12 +2713,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4779,7 +4786,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6269,10 +6277,17 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* create attribute compresssion record */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ {
+ char *compression =
+ GetAttributeCompression(&attribute, colDef->compression, NULL);
+
+ if (compression == NULL)
+ MemSet(NameStr(attribute.attcompression), 0, NAMEDATALEN);
+ else
+ namestrcpy(&(attribute.attcompression), compression);
+ }
else
- attribute.attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attribute.attcompression), 0, NAMEDATALEN);
/* attribute.attacl is handled by InsertPgAttributeTuples() */
@@ -7698,9 +7713,9 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
format_type_be(attrtuple->atttypid))));
/* use default compression if storage is not PLAIN */
- if (!IsValidCompression(attrtuple->attcompression) &&
+ if (!IsValidCompression(*NameStr(attrtuple->attcompression)) &&
(newstorage != TYPSTORAGE_PLAIN))
- attrtuple->attcompression = DefaultCompressionMethod;
+ InitAttributeCompression(attrtuple);
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
@@ -11783,13 +11798,10 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
{
/* Setup attribute compression */
- if (attTup->attstorage == TYPSTORAGE_PLAIN)
- attTup->attcompression = InvalidCompressionMethod;
- else if (!IsValidCompression(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ InitAttributeCompression(attTup);
}
else
- attTup->attcompression = InvalidCompressionMethod;
+ MemSet(NameStr(attTup->attcompression), 0, NAMEDATALEN);
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
@@ -14972,23 +14984,17 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
- char cmethod;
- Datum values[Natts_pg_attribute];
- bool nulls[Natts_pg_attribute];
- bool replace[Natts_pg_attribute];
+ char *cm;
+ bool need_rewrite;
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15007,24 +15013,20 @@ ATExecSetCompression(AlteredTableInfo *tab,
errmsg("cannot alter system column \"%s\"", column)));
/* Prevent from altering untoastable attributes with PLAIN storage */
- if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ if (atttableform->attstorage == TYPSTORAGE_PLAIN &&
+ !TypeIsToastable(atttableform->atttypid))
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("column data type %s does not support compression",
format_type_be(atttableform->atttypid))));
- /* Initialize buffers for new tuple values */
- memset(values, 0, sizeof(values));
- memset(nulls, false, sizeof(nulls));
- memset(replace, false, sizeof(replace));
-
/* Get the attribute compression method. */
- cmethod = GetAttributeCompressionMethod(atttableform, compression);
-
- if (atttableform->attcompression != cmethod)
+ cm = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
- atttableform->attcompression = cmethod;
+ namestrcpy(&atttableform->attcompression, cm);
+
CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -17730,27 +17732,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * GetAttributeCompressionMethod - Get compression method from compression name
- */
-static char
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- char cm;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidCompressionMethod;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionMethod;
-
- cm = GetCompressionMethod(compression);
- if (!IsValidCompression(cm))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cm;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index dd74d93384..dc82c25aee 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2050,11 +2051,13 @@ CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Check whether the source value's compression method is supported
+ * in the target table's attribute or not. If it is not supported
+ * then decompress the value.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionMethodFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ if (strchr(NameStr(targetTupDesc->attrs[i].attcompression),
+ GetCompressionMethodFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value))) == NULL)
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 37fe5c72e9..aac399427c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,7 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2950,6 +2950,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5631,6 +5642,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..35efa064e5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3692,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 718c2ed7fd..a1b5302d5b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,7 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2853,6 +2853,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4201,6 +4210,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index fb8b597b18..29ff3ad0f2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,35 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 9b3233e052..c33007e12f 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1065,9 +1065,9 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
def->storage = 0;
/* Likewise, copy compression if requested */
- if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
- && IsValidCompression(attribute->attcompression))
- def->compression = GetCompressionName(attribute->attcompression);
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION)
+ def->compression =
+ MakeColumnCompression(&attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 5135992b5c..a7ba8aac9f 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,8 +156,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
- /* attribute's compression options */
- char attcompression BKI_DEFAULT('\0');
+ /*
+ * Attribute's compression options, the first character represents the
+ * current compression method and followed by all old preserved compression
+ * methods.
+ */
+ NameData attcompression BKI_DEFAULT('\0');
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -186,7 +190,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute, attcompression) + NAMEDATALEN)
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..97e6130284 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern char *GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern void InitAttributeCompression(Form_pg_attribute att);
+extern ColumnCompression *MakeColumnCompression(NameData *compression);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..fed9cb7994 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..e413914979 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 0052085cb3..33c7132353 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -98,4 +98,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index b8949c8630..abf34eede0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -48,4 +48,8 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v4-0002-alter-table-set-compression.patchapplication/octet-stream; name=v4-0002-alter-table-set-compression.patchDownload
From 36c234e909bb93005a23caf5c299842240e1b13c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 5 Oct 2020 12:04:06 +0530
Subject: [PATCH v4 2/3] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 +++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++++
src/test/regress/sql/create_cm.sql | 6 ++
9 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d80aeb875d..53ab899ba6 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3856,6 +3858,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4371,6 +4374,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4774,6 +4778,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5398,6 +5406,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CheckTargetCMAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14957,6 +14968,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char cmethod;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmethod = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmethod)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmethod;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2559018a32..dd74d93384 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2013,7 +2013,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 94ebf40e03..fb8b597b18 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 415e117407..da20bf40fb 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -602,5 +602,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..0052085cb3 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -83,4 +83,19 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..b8949c8630 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -42,4 +42,10 @@ INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.
Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example). Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?
Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.
I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).
Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.
FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?
Not sure. My assumption was we'd end up with a new catalog, but maybe
stashing it into pg_attribute is fine. I was really thinking about two
kinds of options - compression level, and some sort of column-level
dictionary. Compression level is not necessary for decompression, but
the dictionary ID would be needed. (I think the global dictionary was
one of the use cases, aimed at JSON compression.)
But I don't think stashing it in pg_attribute means we couldn't use it
for decompression - we'd just need to keep an array of options, one for
each compression method. Keeping it in a separate new catalog might be
cleaner, and I'm not sure how large the configuration might be.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).
I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.
Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).
You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?
Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.
Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1]/messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com, it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.
[1]: /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?Not sure. My assumption was we'd end up with a new catalog, but maybe
stashing it into pg_attribute is fine. I was really thinking about two
kinds of options - compression level, and some sort of column-level
dictionary. Compression level is not necessary for decompression, but
the dictionary ID would be needed. (I think the global dictionary was
one of the use cases, aimed at JSON compression.)
Ok
But I don't think stashing it in pg_attribute means we couldn't use it
for decompression - we'd just need to keep an array of options, one for
each compression method.
Yeah, we can do that.
Keeping it in a separate new catalog might be
cleaner, and I'm not sure how large the configuration might be.
Yeah in that case it will be better to store in a separate catalog,
because sometimes if multiple attributes are using the same
compression method with the same options then we can store the same
oid in attcompression instead of duplicating the option field.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.
All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?Not sure. My assumption was we'd end up with a new catalog, but maybe
stashing it into pg_attribute is fine. I was really thinking about two
kinds of options - compression level, and some sort of column-level
dictionary. Compression level is not necessary for decompression, but
the dictionary ID would be needed. (I think the global dictionary was
one of the use cases, aimed at JSON compression.)Ok
But I don't think stashing it in pg_attribute means we couldn't use it
for decompression - we'd just need to keep an array of options, one for
each compression method.Yeah, we can do that.
Keeping it in a separate new catalog might be
cleaner, and I'm not sure how large the configuration might be.
Yeah in that case it will be better to store in a separate catalog,
because sometimes if multiple attributes are using the same
compression method with the same options then we can store the same
oid in attcompression instead of duplicating the option field.
I doubt deduplicating the options like this is (sharing options between
columns) is really worth it, as it means extra complexity e.g. during
ALTER TABLE ... SET COMPRESSION. I don't think we do that for other
catalogs, so why should we do it here?
Ultimately I think it's a question of how large we expect the options to
be, and how flexible it needs to be.
For example, what happens if the user does this:
ALTER ... SET COMPRESSION my_compression WITH (options1) PRESERVE;
ALTER ... SET COMPRESSION pglz PRESERVE;
ALTER ... SET COMPRESSION my_compression WITH (options2) PRESERVE;
I believe it's enough to keep just the last value, but maybe I'm wrong
and we need to keep the whole history?
The use case I'm thinking about is the column-level JSON compression,
where one of the options identifies the dictionary. OTOH I'm not sure
this is the right way to track this info - we need to know which options
were compressed with which options, i.e. it needs to be encoded in each
value directly. It'd also require changes to the PRESERVE handling
because it'd be necessary to identify which options to preserve ...
So maybe this is either nonsense or something we don't want to support,
and we should only allow one option for each compression method.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...
Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?Not sure. My assumption was we'd end up with a new catalog, but maybe
stashing it into pg_attribute is fine. I was really thinking about two
kinds of options - compression level, and some sort of column-level
dictionary. Compression level is not necessary for decompression, but
the dictionary ID would be needed. (I think the global dictionary was
one of the use cases, aimed at JSON compression.)Ok
But I don't think stashing it in pg_attribute means we couldn't use it
for decompression - we'd just need to keep an array of options, one for
each compression method.Yeah, we can do that.
Keeping it in a separate new catalog might be
cleaner, and I'm not sure how large the configuration might be.
Yeah in that case it will be better to store in a separate catalog,
because sometimes if multiple attributes are using the same
compression method with the same options then we can store the same
oid in attcompression instead of duplicating the option field.I doubt deduplicating the options like this is (sharing options between
columns) is really worth it, as it means extra complexity e.g. during
ALTER TABLE ... SET COMPRESSION. I don't think we do that for other
catalogs, so why should we do it here?
Yeah, valid point.
Ultimately I think it's a question of how large we expect the options to
be, and how flexible it needs to be.For example, what happens if the user does this:
ALTER ... SET COMPRESSION my_compression WITH (options1) PRESERVE;
ALTER ... SET COMPRESSION pglz PRESERVE;
ALTER ... SET COMPRESSION my_compression WITH (options2) PRESERVE;I believe it's enough to keep just the last value, but maybe I'm wrong
and we need to keep the whole history?
Currently, the syntax is like ALTER ... SET COMPRESSION my_compression
WITH (options1) PRESERVE (old_compression1, old_compression2..). But
I think if the user just gives
PRESERVE without a list then we should just preserve the latest one.
The use case I'm thinking about is the column-level JSON compression,
where one of the options identifies the dictionary. OTOH I'm not sure
this is the right way to track this info - we need to know which options
were compressed with which options, i.e. it needs to be encoded in each
value directly. It'd also require changes to the PRESERVE handling
because it'd be necessary to identify which options to preserve ...So maybe this is either nonsense or something we don't want to support,
and we should only allow one option for each compression method.
Yeah, it is a bit confusing to add the same compression method with
different compression options, then in the preserve list, we will
have to allow the option as well along with the compression method to
know which compression method with what options we want to preserve.
And also as you mentioned that in rows we need to know the option as
well. I think for solving this anyways for the custom compression
methods we will have to store the OID of the compression method in the
toast header so we can provide an intermediate catalog which will
create a new row for each combination of compression method + option
and the toast header can store the OID of that row so that we know
with which compression method + option it was compressed with.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.
OK.
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?Not sure. My assumption was we'd end up with a new catalog, but maybe
stashing it into pg_attribute is fine. I was really thinking about two
kinds of options - compression level, and some sort of column-level
dictionary. Compression level is not necessary for decompression, but
the dictionary ID would be needed. (I think the global dictionary was
one of the use cases, aimed at JSON compression.)Ok
But I don't think stashing it in pg_attribute means we couldn't use it
for decompression - we'd just need to keep an array of options, one for
each compression method.Yeah, we can do that.
Keeping it in a separate new catalog might be
cleaner, and I'm not sure how large the configuration might be.
Yeah in that case it will be better to store in a separate catalog,
because sometimes if multiple attributes are using the same
compression method with the same options then we can store the same
oid in attcompression instead of duplicating the option field.I doubt deduplicating the options like this is (sharing options between
columns) is really worth it, as it means extra complexity e.g. during
ALTER TABLE ... SET COMPRESSION. I don't think we do that for other
catalogs, so why should we do it here?Yeah, valid point.
Ultimately I think it's a question of how large we expect the options to
be, and how flexible it needs to be.For example, what happens if the user does this:
ALTER ... SET COMPRESSION my_compression WITH (options1) PRESERVE;
ALTER ... SET COMPRESSION pglz PRESERVE;
ALTER ... SET COMPRESSION my_compression WITH (options2) PRESERVE;I believe it's enough to keep just the last value, but maybe I'm wrong
and we need to keep the whole history?Currently, the syntax is like ALTER ... SET COMPRESSION my_compression
WITH (options1) PRESERVE (old_compression1, old_compression2..). But I
think if the user just gives PRESERVE without a list then we should
just preserve the latest one.
Hmmm. Not sure that's very convenient. I'd expect the most common use
case for PRESERVE being "I want to change compression for new data,
without rewrite". If PRESERVE by default preserves the latest one, that
pretty much forces users to always list all methods. I suggest
iterpreting it as "preserve everything" instead.
Another option would be to require either a list of methods, or some
keyword defining what to preserve. Like for example
... PRESERVE (m1, m2, ...)
... PRESERVE ALL
... PRESERVE LAST
Does that make sense?
The use case I'm thinking about is the column-level JSON compression,
where one of the options identifies the dictionary. OTOH I'm not sure
this is the right way to track this info - we need to know which options
were compressed with which options, i.e. it needs to be encoded in each
value directly. It'd also require changes to the PRESERVE handling
because it'd be necessary to identify which options to preserve ...So maybe this is either nonsense or something we don't want to support,
and we should only allow one option for each compression method.Yeah, it is a bit confusing to add the same compression method with
different compression options, then in the preserve list, we will
have to allow the option as well along with the compression method to
know which compression method with what options we want to preserve.And also as you mentioned that in rows we need to know the option as
well. I think for solving this anyways for the custom compression
methods we will have to store the OID of the compression method in the
toast header so we can provide an intermediate catalog which will
create a new row for each combination of compression method + option
and the toast header can store the OID of that row so that we know
with which compression method + option it was compressed with.
I agree. After thinking about this a bit more, I think we should just
keep the last options for each compression method. If we need to allow
multiple options for some future compression method, we can improve
this, but until then it'd be an over-engineering. Let's do the simplest
possible thing here.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
10) compression parameters?
I wonder if we could/should allow parameters, like compression level
(and maybe other stuff, depending on the compression method). PG13
allowed that for opclasses, so perhaps we should allow it here too.Yes, that is also in the plan. For doing this we are planning to add
an extra column in the pg_attribute which will store the compression
options for the current compression method. The original patch was
creating an extra catalog pg_column_compression, therein it maintains
the oid of the compression method as well as the compression options.
The advantage of creating an extra catalog is that we can keep the
compression options for the preserved compression methods also so that
we can support the options which can be used for decompressing the
data as well. Whereas if we want to avoid this extra catalog then we
can not use that compression option for decompressing. But most of
the options e.g. compression level are just for the compressing so it
is enough to store for the current compression method only. What's
your thoughts?Not sure. My assumption was we'd end up with a new catalog, but maybe
stashing it into pg_attribute is fine. I was really thinking about two
kinds of options - compression level, and some sort of column-level
dictionary. Compression level is not necessary for decompression, but
the dictionary ID would be needed. (I think the global dictionary was
one of the use cases, aimed at JSON compression.)Ok
But I don't think stashing it in pg_attribute means we couldn't use it
for decompression - we'd just need to keep an array of options, one for
each compression method.Yeah, we can do that.
Keeping it in a separate new catalog might be
cleaner, and I'm not sure how large the configuration might be.
Yeah in that case it will be better to store in a separate catalog,
because sometimes if multiple attributes are using the same
compression method with the same options then we can store the same
oid in attcompression instead of duplicating the option field.I doubt deduplicating the options like this is (sharing options between
columns) is really worth it, as it means extra complexity e.g. during
ALTER TABLE ... SET COMPRESSION. I don't think we do that for other
catalogs, so why should we do it here?Yeah, valid point.
Ultimately I think it's a question of how large we expect the options to
be, and how flexible it needs to be.For example, what happens if the user does this:
ALTER ... SET COMPRESSION my_compression WITH (options1) PRESERVE;
ALTER ... SET COMPRESSION pglz PRESERVE;
ALTER ... SET COMPRESSION my_compression WITH (options2) PRESERVE;I believe it's enough to keep just the last value, but maybe I'm wrong
and we need to keep the whole history?Currently, the syntax is like ALTER ... SET COMPRESSION my_compression
WITH (options1) PRESERVE (old_compression1, old_compression2..). But I
think if the user just gives PRESERVE without a list then we should
just preserve the latest one.Hmmm. Not sure that's very convenient. I'd expect the most common use
case for PRESERVE being "I want to change compression for new data,
without rewrite". If PRESERVE by default preserves the latest one, that
pretty much forces users to always list all methods. I suggest
iterpreting it as "preserve everything" instead.Another option would be to require either a list of methods, or some
keyword defining what to preserve. Like for example... PRESERVE (m1, m2, ...)
... PRESERVE ALL
... PRESERVE LASTDoes that make sense?
Yeah, this makes sense to me.
The use case I'm thinking about is the column-level JSON compression,
where one of the options identifies the dictionary. OTOH I'm not sure
this is the right way to track this info - we need to know which options
were compressed with which options, i.e. it needs to be encoded in each
value directly. It'd also require changes to the PRESERVE handling
because it'd be necessary to identify which options to preserve ...So maybe this is either nonsense or something we don't want to support,
and we should only allow one option for each compression method.Yeah, it is a bit confusing to add the same compression method with
different compression options, then in the preserve list, we will
have to allow the option as well along with the compression method to
know which compression method with what options we want to preserve.And also as you mentioned that in rows we need to know the option as
well. I think for solving this anyways for the custom compression
methods we will have to store the OID of the compression method in the
toast header so we can provide an intermediate catalog which will
create a new row for each combination of compression method + option
and the toast header can store the OID of that row so that we know
with which compression method + option it was compressed with.I agree. After thinking about this a bit more, I think we should just
keep the last options for each compression method. If we need to allow
multiple options for some future compression method, we can improve
this, but until then it'd be an over-engineering. Let's do the simplest
possible thing here.
Okay.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
I have changed the first 2 patches, basically, now we are providing a
new catalog pg_compression and the pg_attribute is storing the oid of
the compression method. The patches still need some cleanup and there
is also one open comment that for index we should use its table
compression.
I am still working on the preserve patch. For preserving the
compression method I am planning to convert the attcompression field
to the oidvector so that we can store the oid of the preserve method
also. I am not sure whether we can access this oidvector as a fixed
part of the FormData_pg_attribute or not. The reason is that for
building the tuple descriptor, we need to give the size of the fixed
part (#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))). But
if we convert this to the oidvector then we don't know the size of the
fixed part. Am I missing something?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v4-0002-alter-table-set-compression.patchapplication/octet-stream; name=v4-0002-alter-table-set-compression.patchDownload
From a722d19c3684f8d3b76cb55cd0a2a8ba6c2edf2d Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 7 Oct 2020 16:16:29 +0530
Subject: [PATCH v4 2/2] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 +++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++++
src/test/regress/sql/create_cm.sql | 6 ++
9 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dc2298d8bf..851b95b9d0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3856,6 +3858,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4371,6 +4374,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4774,6 +4778,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5398,6 +5406,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CheckTargetCMAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14957,6 +14968,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2c0a4abd5a..6249c08870 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2013,7 +2013,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9559bee017..c7b70447a9 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 415e117407..da20bf40fb 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -602,5 +602,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..0052085cb3 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -83,4 +83,19 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..b8949c8630 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -42,4 +42,10 @@ INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v4-0001-Built-in-compression-method.patchapplication/octet-stream; name=v4-0001-Built-in-compression-method.patchDownload
From 0308b372b2f0f83cd1391ce6a21c0c62fc0232f3 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v4 1/2] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 71 +++--
src/backend/access/common/tupdesc.c | 5 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 136 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/compression/zlib.c | 150 ++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 41 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 21 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
69 files changed, 1696 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/backend/access/compression/zlib.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 28f844071b..38275e9cd4 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..1460fbc431 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..0427007e2d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..771a85f5e4 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
@@ -30,6 +31,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +59,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..061a1d4649 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..4883d114cf
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o zlib.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..4c82524143
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,136 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionMethodOid - given an compression method name, look up
+ * its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionName - given an compression method oid, look up
+ * its name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION_OID:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..1779801375
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/zlib.c b/src/backend/access/compression/zlib.c
new file mode 100644
index 0000000000..9191c3b5c9
--- /dev/null
+++ b/src/backend/access/compression/zlib.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
\ No newline at end of file
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..0c510805a2 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2228ad9dad 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e0ac4e05e5..dc2298d8bf 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2392,6 +2403,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2426,6 +2453,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2671,6 +2699,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6213,6 +6254,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7636,6 +7686,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!IsValidCompression(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11713,6 +11768,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!IsValidCompression(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17577,3 +17644,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9812089161..2c0a4abd5a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2005,6 +2008,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2236,6 +2310,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CheckTargetCMAndDecompress(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0386480ab..718c2ed7fd 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,6 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0d101d8171..9559bee017 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..cd4c850ef8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..2f3e39d072 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 76320468ba..a911521e4c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8615,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8644,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15675,6 +15693,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15699,6 +15718,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15734,6 +15756,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..1443c13bbc 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
@@ -2744,7 +2781,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 24c7b414cf..fe2ea885a6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..503750228f
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression methods. Pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define ZLIB_COMPRESSION 'z'
+#define InvalidCompressionMethod '\0'
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+#define MAX_BUILTIN_COMPRESSION_METHOD ZLIB_COMPRESSION_ID + 1
+
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOid(const char *compression);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..e509f56ffd 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..1efe4d62c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ec897c2682
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,21 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'ZLIB_COMPRESSION_OID', cmname => 'zlib', cmhandler => 'zlibhandler'},
+
+]
+
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d6f3e2d286..c3db760863 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'zlib compression method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7196,6 +7205,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index af4192f9a8..64f66e2dc3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3007,11 +3007,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3027,11 +3027,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3058,11 +3058,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
I have changed the first 2 patches, basically, now we are providing a
new catalog pg_compression and the pg_attribute is storing the oid of
the compression method. The patches still need some cleanup and there
is also one open comment that for index we should use its table
compression.I am still working on the preserve patch. For preserving the
compression method I am planning to convert the attcompression field
to the oidvector so that we can store the oid of the preserve method
also. I am not sure whether we can access this oidvector as a fixed
part of the FormData_pg_attribute or not. The reason is that for
building the tuple descriptor, we need to give the size of the fixed
part (#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))). But
if we convert this to the oidvector then we don't know the size of the
fixed part. Am I missing something?
I could think of two solutions here
Sol1.
Make the first oid of the oidvector as part of the fixed size, like below
#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute, attcompression) + OidVectorSize(1))
Sol2:
Keep attcompression as oid only and for the preserve list, adds
another field in the variable part which will be of type oidvector. I
think most of the time we need to access the current compression
method and with this solution, we will be able to access that as part
of the tuple desc.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
I have changed the first 2 patches, basically, now we are providing a
new catalog pg_compression and the pg_attribute is storing the oid of
the compression method. The patches still need some cleanup and there
is also one open comment that for index we should use its table
compression.
There was some unwanted code in the previous patch so attaching the
updated patches.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v5-0002-alter-table-set-compression.patchapplication/octet-stream; name=v5-0002-alter-table-set-compression.patchDownload
From ece4bf017d4a46915a737638296da06e579cd742 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 7 Oct 2020 16:16:29 +0530
Subject: [PATCH v5 2/2] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 +++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++++
src/test/regress/sql/create_cm.sql | 6 ++
9 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 57c054cdfd..49f7dca583 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3856,6 +3858,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4371,6 +4374,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4774,6 +4778,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5398,6 +5406,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CheckTargetCMAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14957,6 +14968,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2c0a4abd5a..6249c08870 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2013,7 +2013,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9559bee017..c7b70447a9 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 415e117407..da20bf40fb 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -602,5 +602,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..0052085cb3 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -83,4 +83,19 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..b8949c8630 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -42,4 +42,10 @@ INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v5-0001-Built-in-compression-method.patchapplication/octet-stream; name=v5-0001-Built-in-compression-method.patchDownload
From 7ee9d9ae2be2826f462147e523e92d1517929b15 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v5 1/2] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 71 +++--
src/backend/access/common/tupdesc.c | 5 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 136 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/compression/zlib.c | 150 ++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 41 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 63 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 21 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
69 files changed, 1686 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/backend/access/compression/zlib.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 28f844071b..38275e9cd4 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..1460fbc431 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..0427007e2d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..771a85f5e4 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
@@ -30,6 +31,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +59,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..061a1d4649 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..4883d114cf
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o zlib.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..4c82524143
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,136 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionMethodOid - given an compression method name, look up
+ * its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionName - given an compression method oid, look up
+ * its name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION_OID:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/zlib.c b/src/backend/access/compression/zlib.c
new file mode 100644
index 0000000000..7cd106601d
--- /dev/null
+++ b/src/backend/access/compression/zlib.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
\ No newline at end of file
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2228ad9dad 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e0ac4e05e5..57c054cdfd 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2392,6 +2403,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2426,6 +2453,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2671,6 +2699,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6213,6 +6254,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7636,6 +7686,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11713,6 +11768,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17577,3 +17644,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9812089161..2c0a4abd5a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2005,6 +2008,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2236,6 +2310,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CheckTargetCMAndDecompress(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0386480ab..718c2ed7fd 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,6 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0d101d8171..9559bee017 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..cd4c850ef8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 1975d629a6..2f3e39d072 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 76320468ba..a911521e4c 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8615,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8644,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15675,6 +15693,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15699,6 +15718,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15734,6 +15756,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..1443c13bbc 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
@@ -2744,7 +2781,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 24c7b414cf..fe2ea885a6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..f905f3ecaf
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOid(const char *compression);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..e509f56ffd 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ec897c2682
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,21 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'ZLIB_COMPRESSION_OID', cmname => 'zlib', cmhandler => 'zlibhandler'},
+
+]
+
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d6f3e2d286..c3db760863 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'zlib compression method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7196,6 +7205,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index af4192f9a8..64f66e2dc3 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3007,11 +3007,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3027,11 +3027,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3058,11 +3058,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Thu, Oct 08, 2020 at 02:38:27PM +0530, Dilip Kumar wrote:
On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
I have changed the first 2 patches, basically, now we are providing a
new catalog pg_compression and the pg_attribute is storing the oid of
the compression method. The patches still need some cleanup and there
is also one open comment that for index we should use its table
compression.I am still working on the preserve patch. For preserving the
compression method I am planning to convert the attcompression field
to the oidvector so that we can store the oid of the preserve method
also. I am not sure whether we can access this oidvector as a fixed
part of the FormData_pg_attribute or not. The reason is that for
building the tuple descriptor, we need to give the size of the fixed
part (#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))). But
if we convert this to the oidvector then we don't know the size of the
fixed part. Am I missing something?I could think of two solutions here
Sol1.
Make the first oid of the oidvector as part of the fixed size, like below
#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute, attcompression) + OidVectorSize(1))Sol2:
Keep attcompression as oid only and for the preserve list, adds
another field in the variable part which will be of type oidvector. I
think most of the time we need to access the current compression
method and with this solution, we will be able to access that as part
of the tuple desc.
And is the oidvector actually needed? If we have the extra catalog,
can't we track this simply using the regular dependencies? So we'd have
the attcompression OID of the current compression method, and the
preserved values would be tracked in pg_depend.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Fri, Oct 9, 2020 at 3:24 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Thu, Oct 08, 2020 at 02:38:27PM +0530, Dilip Kumar wrote:
On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
I have changed the first 2 patches, basically, now we are providing a
new catalog pg_compression and the pg_attribute is storing the oid of
the compression method. The patches still need some cleanup and there
is also one open comment that for index we should use its table
compression.I am still working on the preserve patch. For preserving the
compression method I am planning to convert the attcompression field
to the oidvector so that we can store the oid of the preserve method
also. I am not sure whether we can access this oidvector as a fixed
part of the FormData_pg_attribute or not. The reason is that for
building the tuple descriptor, we need to give the size of the fixed
part (#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))). But
if we convert this to the oidvector then we don't know the size of the
fixed part. Am I missing something?I could think of two solutions here
Sol1.
Make the first oid of the oidvector as part of the fixed size, like below
#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute, attcompression) + OidVectorSize(1))Sol2:
Keep attcompression as oid only and for the preserve list, adds
another field in the variable part which will be of type oidvector. I
think most of the time we need to access the current compression
method and with this solution, we will be able to access that as part
of the tuple desc.And is the oidvector actually needed? If we have the extra catalog,
can't we track this simply using the regular dependencies? So we'd have
the attcompression OID of the current compression method, and the
preserved values would be tracked in pg_depend.
Right, we can do that as well. Actually, the preserved list need to
be accessed only in case of ALTER TABLE SET COMPRESSION and INSERT
INTO SELECT * FROM queries. So in such cases, I think it is okay to
get the preserved compression oids from pg_depends.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Oct 9, 2020 at 3:01 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Fri, Oct 9, 2020 at 3:24 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Thu, Oct 08, 2020 at 02:38:27PM +0530, Dilip Kumar wrote:
On Wed, Oct 7, 2020 at 5:00 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Oct 7, 2020 at 10:26 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 6, 2020 at 10:21 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Tue, Oct 06, 2020 at 11:00:55AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 9:34 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 07:57:41PM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 5:53 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 05, 2020 at 11:17:28AM +0530, Dilip Kumar wrote:
On Mon, Oct 5, 2020 at 3:37 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:Thanks, Tomas for your feedback.
9) attcompression ...
The main issue I see is what the patch does with attcompression. Instead
of just using it to store a the compression method, it's also used to
store the preserved compression methods. And using NameData to store
this seems wrong too - if we really want to store this info, the correct
way is either using text[] or inventing charvector or similar.The reason for using the NameData is the get it in the fixed part of
the data structure.Why do we need that? It's possible to have varlena fields with direct
access (see pg_index.indkey for example).I see. While making it NameData I was thinking whether we have an
option to direct access the varlena. Thanks for pointing me there. I
will change this.Adding NameData just to make
it fixed-length means we're always adding 64B even if we just need a
single byte, which means ~30% overhead for the FormData_pg_attribute.
That seems a bit unnecessary, and might be an issue with many attributes
(e.g. with many temp tables, etc.).You are right. Even I did not like to keep 64B for this, so I will change it.
But to me this seems very much like a misuse of attcompression to track
dependencies on compression methods, necessary because we don't have a
separate catalog listing compression methods. If we had that, I think we
could simply add dependencies between attributes and that catalog.Basically, up to this patch, we are having only built-in compression
methods and those can not be dropped so we don't need any dependency
at all. We just want to know what is the current compression method
and what is the preserve compression methods supported for this
attribute. Maybe we can do it better instead of using the NameData
but I don't think it makes sense to add a separate catalog?Sure, I understand what the goal was - all I'm saying is that it looks
very much like a workaround needed because we don't have the catalog.I don't quite understand how could we support custom compression methods
without listing them in some sort of catalog?Yeah for supporting custom compression we need some catalog.
Moreover, having the catalog would allow adding compression methods
(from extensions etc) instead of just having a list of hard-coded
compression methods. Which seems like a strange limitation, considering
this thread is called "custom compression methods".I think I forgot to mention while submitting the previous patch that
the next patch I am planning to submit is, Support creating the custom
compression methods wherein we can use pg_am catalog to insert the new
compression method. And for dependency handling, we can create an
attribute dependency on the pg_am row. Basically, we will create the
attribute dependency on the current compression method AM as well as
on the preserved compression methods AM. As part of this, we will
add two build-in AMs for zlib and pglz, and the attcompression field
will be converted to the oid_vector (first OID will be of the current
compression method, followed by the preserved compression method's
oids).Hmmm, ok. Not sure pg_am is the right place - compression methods don't
quite match what I though AMs are about, but maybe it's just my fault.FWIW it seems a bit strange to first do the attcompression magic and
then add the catalog later - I think we should start with the catalog
right away. The advantage is that if we end up committing only some of
the patches in this cycle, we already have all the infrastructure etc.
We can reorder that later, though.Hmm, yeah we can do this way as well that first create a new catalog
table and add entries for these two built-in methods and the
attcompression can store the oid vector. But if we only commit the
build-in compression methods part then does it make sense to create an
extra catalog or adding these build-in methods to the existing catalog
(if we plan to use pg_am). Then in attcompression instead of using
one byte for each preserve compression method, we need to use oid. So
from Robert's mail[1], it appeared to me that he wants that the
build-in compression methods part should be independently committable
and if we think from that perspective then adding a catalog doesn't
make much sense. But if we are planning to commit the custom method
also then it makes more sense to directly start with the catalog
because that way it will be easy to expand without much refactoring.[1] /messages/by-id/CA+TgmobSDVgUage9qQ5P_=F_9jaMkCgyKxUQGtFQU7oN4kX-AA@mail.gmail.com
Hmmm. Maybe I'm missing something subtle, but I think that plan can be
interpreted in various ways - it does not really say whether the initial
list of built-in methods should be in some C array, or already in a proper
catalog.All I'm saying is it seems a bit weird to first implement dependencies
based on strange (mis)use of attcompression attribute, and then replace
it with a proper catalog. My understanding is those patches are expected
to be committable one by one, but the attcompression approach seems a
bit too hacky to me - not sure I'd want to commit that ...Okay, I will change this. So I will make create a new catalog
pg_compression and add the entry for two built-in compression methods
from the very first patch.OK.
I have changed the first 2 patches, basically, now we are providing a
new catalog pg_compression and the pg_attribute is storing the oid of
the compression method. The patches still need some cleanup and there
is also one open comment that for index we should use its table
compression.I am still working on the preserve patch. For preserving the
compression method I am planning to convert the attcompression field
to the oidvector so that we can store the oid of the preserve method
also. I am not sure whether we can access this oidvector as a fixed
part of the FormData_pg_attribute or not. The reason is that for
building the tuple descriptor, we need to give the size of the fixed
part (#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))). But
if we convert this to the oidvector then we don't know the size of the
fixed part. Am I missing something?I could think of two solutions here
Sol1.
Make the first oid of the oidvector as part of the fixed size, like below
#define ATTRIBUTE_FIXED_PART_SIZE \
(offsetof(FormData_pg_attribute, attcompression) + OidVectorSize(1))Sol2:
Keep attcompression as oid only and for the preserve list, adds
another field in the variable part which will be of type oidvector. I
think most of the time we need to access the current compression
method and with this solution, we will be able to access that as part
of the tuple desc.And is the oidvector actually needed? If we have the extra catalog,
can't we track this simply using the regular dependencies? So we'd have
the attcompression OID of the current compression method, and the
preserved values would be tracked in pg_depend.Right, we can do that as well. Actually, the preserved list need to
be accessed only in case of ALTER TABLE SET COMPRESSION and INSERT
INTO SELECT * FROM queries. So in such cases, I think it is okay to
get the preserved compression oids from pg_depends.
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v6-0001-Built-in-compression-method.patchapplication/octet-stream; name=v6-0001-Built-in-compression-method.patchDownload
From e9047aeb23a9ade71d84afc82f3d3858eacdaa14 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v6 1/3] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 71 +++--
src/backend/access/common/tupdesc.c | 5 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 136 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/compression/zlib.c | 150 ++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 41 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 63 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 30 +-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 21 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
69 files changed, 1686 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/backend/access/compression/zlib.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 28f844071b..38275e9cd4 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..1460fbc431 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..0427007e2d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..771a85f5e4 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
@@ -30,6 +31,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cm_method);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +59,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..061a1d4649 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..4883d114cf
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o zlib.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..4c82524143
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,136 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionMethodOid - given an compression method name, look up
+ * its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionName - given an compression method oid, look up
+ * its name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION_OID:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/zlib.c b/src/backend/access/compression/zlib.c
new file mode 100644
index 0000000000..7cd106601d
--- /dev/null
+++ b/src/backend/access/compression/zlib.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
\ No newline at end of file
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2228ad9dad 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e0ac4e05e5..57c054cdfd 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2392,6 +2403,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2426,6 +2453,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2671,6 +2699,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6213,6 +6254,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7636,6 +7686,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11713,6 +11768,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17577,3 +17644,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9812089161..2c0a4abd5a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2005,6 +2008,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2236,6 +2310,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CheckTargetCMAndDecompress(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 0409a40b82..37fe5c72e9 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,6 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f0386480ab..718c2ed7fd 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,6 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..cd4c850ef8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 4cb27f2224..73ce6e8e45 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 88bbbd9a9e..f944e550f2 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8473,6 +8474,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8558,6 +8560,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8576,7 +8587,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8603,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8631,6 +8648,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15681,6 +15699,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15705,6 +15724,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15740,6 +15762,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 58de433fd3..1443c13bbc 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
@@ -2744,7 +2781,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 24c7b414cf..fe2ea885a6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..f905f3ecaf
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOid(const char *compression);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..e509f56ffd 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -22,22 +22,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +66,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, uint8 cm_method,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ec897c2682
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,21 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'ZLIB_COMPRESSION_OID', cmname => 'zlib', cmhandler => 'zlibhandler'},
+
+]
+
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 22340baf1c..dec136d763 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'zlib compression method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index eb9d45be5e..df899f2ed9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -855,11 +855,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -871,74 +871,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index cf2a9b4408..bf3019bb0b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 026ea880cd..68d119facd 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v6-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v6-0003-Add-support-for-PRESERVE.patchDownload
From 47f1f61c6b7f4d55372187b718e2ad8216567167 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 9 Oct 2020 14:20:13 +0530
Subject: [PATCH v6 3/3] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 213 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 115 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 16 +-
src/backend/nodes/equalfuncs.c | 14 +-
src/backend/nodes/outfuncs.c | 14 +-
src/backend/parser/gram.y | 44 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 15 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 448 insertions(+), 69 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..23dce923c0 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,16 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..15fb3bc40a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,213 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 49f7dca583..772da7522f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -863,7 +865,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -933,6 +935,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2408,17 +2424,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2455,7 +2471,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2706,12 +2722,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4779,7 +4795,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6269,8 +6286,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* create attribute compresssion record */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6445,6 +6462,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6592,6 +6610,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11577,7 +11617,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11688,7 +11732,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11799,6 +11844,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14972,23 +15022,20 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15019,9 +15066,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17731,26 +17781,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
}
}
-/*
- * GetAttributeCompressionMethod - Get compression method from compression name
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 6249c08870..4e5ac4f5f6 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2021,6 +2022,7 @@ CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -2053,8 +2055,9 @@ CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 37fe5c72e9..aac399427c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2930,7 +2930,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2950,6 +2950,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5631,6 +5642,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..35efa064e5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3692,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 718c2ed7fd..a1b5302d5b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2835,7 +2835,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2853,6 +2853,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4201,6 +4210,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..47d18a79ba 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,35 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index cd4c850ef8..c9ead775a7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1067,7 +1067,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3abc8b0972 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..e413914979 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 0052085cb3..33c7132353 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -98,4 +98,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index b8949c8630..abf34eede0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -48,4 +48,8 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v6-0002-alter-table-set-compression.patchapplication/octet-stream; name=v6-0002-alter-table-set-compression.patchDownload
From cc39c9d0963a0d549f852b34795af692a02449be Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 7 Oct 2020 16:16:29 +0530
Subject: [PATCH v6 2/3] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 +++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++++
src/test/regress/sql/create_cm.sql | 6 ++
9 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 57c054cdfd..49f7dca583 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3856,6 +3858,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4371,6 +4374,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4774,6 +4778,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5398,6 +5406,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CheckTargetCMAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14957,6 +14968,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2c0a4abd5a..6249c08870 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2013,7 +2013,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 415e117407..da20bf40fb 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -602,5 +602,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..0052085cb3 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -83,4 +83,19 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..b8949c8630 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -42,4 +42,10 @@ INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.
Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?
I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.
I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.
6. Cleanup of 0004 and 0005 as they are still WIP.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v7-0002-alter-table-set-compression.patchapplication/octet-stream; name=v7-0002-alter-table-set-compression.patchDownload
From ba0677b1d1c53e93b82d2f4a2f3949bf6fc014f4 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 7 Oct 2020 16:16:29 +0530
Subject: [PATCH v7 2/5] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 ++++
src/backend/commands/tablecmds.c | 86 +++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 +++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++++
src/test/regress/sql/create_cm.sql | 6 ++
9 files changed, 134 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1c63a8ce47..1209e7bb8f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3859,6 +3861,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4374,6 +4377,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4777,6 +4781,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5401,6 +5409,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CheckTargetCMAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14960,6 +14971,81 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 45a60ffed2..aa03921c46 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2067,7 +2067,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* tuple with target attribute and if those are different then decompress
* those attributes.
*/
-static void
+void
CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
{
int i;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index b7978cd22e..840b4d0042 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -612,5 +612,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index ee091bb01f..0052085cb3 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -83,4 +83,19 @@ SELECT length(f1) FROM zlibtest;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 56501b45b0..b8949c8630 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -42,4 +42,10 @@ INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM zlibtest;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v7-0001-Built-in-compression-method.patchapplication/octet-stream; name=v7-0001-Built-in-compression-method.patchDownload
From 45c537a05841a191d68235d30ef8815b987d54e8 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v7 1/5] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, zlib) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/Makefile | 2 +-
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 5 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 136 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/compression/zlib.c | 150 ++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/commands/tablecmds.c | 93 ++++++-
src/backend/executor/nodeModifyTable.c | 81 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 41 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 63 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 21 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/postgres.h | 10 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 86 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 45 +++
69 files changed, 1686 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/backend/access/compression/zlib.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index fd6777ae01..1cca0b8def 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>zlib</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/Makefile b/src/backend/Makefile
index 9706a95848..8ff63bc77e 100644
--- a/src/backend/Makefile
+++ b/src/backend/Makefile
@@ -48,7 +48,7 @@ OBJS = \
LIBS := $(filter-out -lpgport -lpgcommon, $(LIBS)) $(LDAP_LIBS_BE) $(ICU_LIBS)
# The backend doesn't need everything that's in LIBS, however
-LIBS := $(filter-out -lz -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
+LIBS := $(filter-out -lreadline -ledit -ltermcap -lncurses -lcurses, $(LIBS))
ifeq ($(with_systemd),yes)
LIBS += -lsystemd
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..1460fbc431 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then only decompress
+ * the slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..0427007e2d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ DefaultCompressionOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..061a1d4649 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -650,6 +653,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attisdropped = false;
att->attislocal = true;
att->attinhcount = 0;
+
/* attacl, attoptions and attfdwoptions are not present in tupledescs */
tuple = SearchSysCache1(TYPEOID, ObjectIdGetDatum(oidtypeid));
@@ -663,6 +667,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..4883d114cf
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o zlib.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..4c82524143
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,136 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionMethodOid - given an compression method name, look up
+ * its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionName - given an compression method oid, look up
+ * its name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method from id.
+ *
+ * Translate the compression id into the compression method.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case ZLIB_COMPRESSION_ID:
+ return ZLIB_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case ZLIB_COMPRESSION_OID:
+ return ZLIB_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/compression/zlib.c b/src/backend/access/compression/zlib.c
new file mode 100644
index 0000000000..7cd106601d
--- /dev/null
+++ b/src/backend/access/compression/zlib.c
@@ -0,0 +1,150 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBZ
+#include <zlib.h>
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
+ zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+#endif
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBZ
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with zlib support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+#endif
+}
\ No newline at end of file
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2228ad9dad 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 511f015a86..1c63a8ce47 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,15 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /* Store the compression method in pg_attribute. */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2406,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2456,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2702,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6216,6 +6257,15 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /* create attribute compresssion record */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7639,6 +7689,11 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /* use default compression if storage is not PLAIN */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11716,6 +11771,18 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* Setup attribute compression */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17580,3 +17647,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * GetAttributeCompressionMethod - Get compression method from compression name
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0c055ed408..45a60ffed2 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2059,6 +2062,77 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Compare the compression method of the compressed attribute in the source
+ * tuple with target attribute and if those are different then decompress
+ * those attributes.
+ */
+static void
+CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2275,6 +2349,13 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CheckTargetCMAndDecompress(slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..1791f3dc10 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,6 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e..b97a87c413 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,6 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..cd4c850ef8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7a8bf76079..33d89f6e6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ff45e3fb8c..8378bee9a5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8615,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8644,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15677,6 +15695,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15701,6 +15720,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15736,6 +15758,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6bb0316bd9..3ac1756e99 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
@@ -2744,7 +2781,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 561fe1dff9..f842e27d25 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..f905f3ecaf
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,63 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast header will store this
+ * in the first 2 bits of the length. Using this we can directly
+ * access the compression method handler routine and those will be
+ * used for the decompression.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ ZLIB_COMPRESSION_ID
+} CompressionId;
+
+
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routine.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ char cmname[64]; /* Name of the compression method */
+ cmcompress_function cmcompress;
+ cmcompress_function cmdecompress;
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOid(const char *compression);
+
+#endif /* CMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ec897c2682
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,21 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'ZLIB_COMPRESSION_OID', cmname => 'zlib', cmhandler => 'zlibhandler'},
+
+]
+
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 22340baf1c..dec136d763 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'zlib compression method handler',
+ proname => 'zlibhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'zlibhandler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..04c94f489d 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,7 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if size in va_extinfo < va_rawsize - VARHDRSZ.
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..ee091bb01f
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,86 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+ERROR: relation "zlibtest" already exists
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index cf2a9b4408..bf3019bb0b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..a9f475b069 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..56501b45b0
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,45 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION zlib);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v7-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v7-0003-Add-support-for-PRESERVE.patchDownload
From f384ccc9b193551a3cfec32b2d21be8d5c436d5b Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 9 Oct 2020 14:20:13 +0530
Subject: [PATCH v7 3/5] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 213 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 115 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 16 +-
src/backend/nodes/equalfuncs.c | 14 +-
src/backend/nodes/outfuncs.c | 14 +-
src/backend/parser/gram.y | 44 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 15 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 448 insertions(+), 69 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..23dce923c0 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,16 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..15fb3bc40a
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,213 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1209e7bb8f..c925ccc27b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -863,7 +865,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -933,6 +935,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2411,17 +2427,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2458,7 +2474,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2709,12 +2725,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4782,7 +4798,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6272,8 +6289,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
/* create attribute compresssion record */
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6448,6 +6465,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6595,6 +6613,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11580,7 +11620,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11691,7 +11735,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11802,6 +11847,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14975,23 +15025,20 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15022,9 +15069,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17734,26 +17784,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
}
}
-/*
- * GetAttributeCompressionMethod - Get compression method from compression name
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index aa03921c46..0d0d29f9ce 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2075,6 +2076,7 @@ CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -2107,8 +2109,9 @@ CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1791f3dc10..5095c0e09c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,7 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2948,6 +2948,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5629,6 +5640,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..35efa064e5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3692,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b97a87c413..4cb48c789a 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,7 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2850,6 +2850,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4198,6 +4207,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..47d18a79ba 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,35 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index cd4c850ef8..c9ead775a7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1067,7 +1067,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..3abc8b0972 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..e413914979 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 0052085cb3..33c7132353 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -98,4 +98,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index b8949c8630..abf34eede0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -48,4 +48,8 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
v7-0005-new-compression-method-extension-for-lz4_WIP.patchapplication/octet-stream; name=v7-0005-new-compression-method-extension-for-lz4_WIP.patchDownload
From 1d8b9f8e605d9efb34ccec4a4ef9f04626d1dd22 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v7 5/5] new compression method extension for lz4
---
contrib/Makefile | 1 +
contrib/cm_lz4/.gitignore | 4 +
contrib/cm_lz4/Makefile | 26 ++++++
contrib/cm_lz4/cm_lz4--1.0.sql | 13 +++
contrib/cm_lz4/cm_lz4.c | 139 +++++++++++++++++++++++++++++
contrib/cm_lz4/cm_lz4.control | 5 ++
contrib/cm_lz4/expected/cm_lz4.out | 45 ++++++++++
contrib/cm_lz4/sql/cm_lz4.sql | 21 +++++
8 files changed, 254 insertions(+)
create mode 100644 contrib/cm_lz4/.gitignore
create mode 100644 contrib/cm_lz4/Makefile
create mode 100644 contrib/cm_lz4/cm_lz4--1.0.sql
create mode 100644 contrib/cm_lz4/cm_lz4.c
create mode 100644 contrib/cm_lz4/cm_lz4.control
create mode 100644 contrib/cm_lz4/expected/cm_lz4.out
create mode 100644 contrib/cm_lz4/sql/cm_lz4.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..7d01c58678 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cm_lz4 \
citext \
cube \
dblink \
diff --git a/contrib/cm_lz4/.gitignore b/contrib/cm_lz4/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cm_lz4/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cm_lz4/Makefile b/contrib/cm_lz4/Makefile
new file mode 100644
index 0000000000..4fd281f50b
--- /dev/null
+++ b/contrib/cm_lz4/Makefile
@@ -0,0 +1,26 @@
+# contrib/cm_lz4/Makefile
+
+MODULE_big = cm_lz4
+OBJS = \
+ $(WIN32RES) \
+ cm_lz4.o
+
+EXTENSION = cm_lz4
+DATA = cm_lz4--1.0.sql
+PGFILEDESC = "lz4 compression method "
+
+SHLIB_LINK += $(filter -llz4, $(LIBS))
+
+REGRESS = cm_lz4
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cm_lz4
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cm_lz4/cm_lz4--1.0.sql b/contrib/cm_lz4/cm_lz4--1.0.sql
new file mode 100644
index 0000000000..4229c4cced
--- /dev/null
+++ b/contrib/cm_lz4/cm_lz4--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cm_lz4--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cm_lz4" to load this file. \quit
+
+CREATE FUNCTION lz4handler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD lz4 HANDLER lz4handler;
+COMMENT ON COMPRESSION METHOD lz4 IS 'lz4 compression method';
diff --git a/contrib/cm_lz4/cm_lz4.c b/contrib/cm_lz4/cm_lz4.c
new file mode 100644
index 0000000000..f2cab6421f
--- /dev/null
+++ b/contrib/cm_lz4/cm_lz4.c
@@ -0,0 +1,139 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "lz4.h"
+#include "utils/builtins.h"
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(lz4handler);
+
+void _PG_init(void);
+
+//typedef int int32;
+/*
+ * Module initialize function: initialize info about lz4
+ */
+void
+_PG_init(void)
+{
+
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + header_size);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *)value + header_size,
+ VARDATA(result),
+ VARSIZE(value) - header_size,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *)value + header_size,
+ VARDATA(result),
+ VARSIZE(value) - header_size,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/contrib/cm_lz4/cm_lz4.control b/contrib/cm_lz4/cm_lz4.control
new file mode 100644
index 0000000000..7f2b0aff00
--- /dev/null
+++ b/contrib/cm_lz4/cm_lz4.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cm_lz4 compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cm_lz4'
+relocatable = true
diff --git a/contrib/cm_lz4/expected/cm_lz4.out b/contrib/cm_lz4/expected/cm_lz4.out
new file mode 100644
index 0000000000..3d0d10476d
--- /dev/null
+++ b/contrib/cm_lz4/expected/cm_lz4.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cm_lz4;
+-- zlib compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE lz4test ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ lz4test
+ Table "public.lz4test"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+ALTER TABLE lz4test ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ lz4test
+ Table "public.lz4test"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- preserve old compression method
+ALTER TABLE lz4test ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (lz4);
+INSERT INTO lz4test VALUES (repeat('1234567890',1004));
+\d+ lz4test
+ Table "public.lz4test"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE lz4test;
diff --git a/contrib/cm_lz4/sql/cm_lz4.sql b/contrib/cm_lz4/sql/cm_lz4.sql
new file mode 100644
index 0000000000..ccde7f73ca
--- /dev/null
+++ b/contrib/cm_lz4/sql/cm_lz4.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cm_lz4;
+
+-- zlib compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+-- alter compression method with rewrite
+ALTER TABLE lz4test ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ lz4test
+ALTER TABLE lz4test ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ lz4test
+
+-- preserve old compression method
+ALTER TABLE lz4test ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (lz4);
+INSERT INTO lz4test VALUES (repeat('1234567890',1004));
+\d+ lz4test
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE lz4test;
--
2.23.0
v7-0004-Create-custom-compression-methods_WIP.patchapplication/octet-stream; name=v7-0004-Create-custom-compression-methods_WIP.patchDownload
From 1880fe2b337cfa0140136f7c5b600d435afde71c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Thu, 15 Oct 2020 14:09:13 +0530
Subject: [PATCH v7 4/5] Create custom compression methods
Provide syntax to create custom compression methods.
---
.../sgml/ref/create_compression_method.sgml | 87 +++++++++++++++
src/backend/access/common/detoast.c | 54 +++++++--
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/pglz.c | 20 ++--
src/backend/access/compression/zlib.c | 14 +--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 103 ++++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 16 ++-
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 +++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 9 ++
27 files changed, 445 insertions(+), 42 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..e9d076194b
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,87 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through this interface.
+ </para>
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the access method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression access method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ access method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+</refentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 1460fbc431..6988a496cd 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,35 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed attribute.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * Get the compression id from the toast header and check whether the
+ * value is custom compressed or not. If it is custom compressed then
+ * fetch the oid from the custome compression header. Otherwise it is
+ * compressed with the built-in compression method so get the oid of the
+ * built-in compression methods.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomeCompressionID(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -447,16 +476,20 @@ static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
CompressionRoutine *cmroutine;
- Oid cmoid;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
- /* Get compression method handler routines */
+ /* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomeCompressionID(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -471,23 +504,28 @@ static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
CompressionRoutine *cmroutine;
- Oid cmoid;
+ Oid cmoid;
+ int32 header_sz;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ cmoid = toast_get_compression_oid(attr);
/* Get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomeCompressionID(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then only decompress
* the slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..4bf383a43a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomeCompressionID(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomeCompressionID(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index 4c82524143..6e0e7d48c8 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -131,6 +131,6 @@ GetCompressionId(Oid cmoid)
case ZLIB_COMPRESSION_OID:
return ZLIB_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOME_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/zlib.c b/src/backend/access/compression/zlib.c
index 7cd106601d..eb31184aef 100644
--- a/src/backend/access/compression/zlib.c
+++ b/src/backend/access/compression/zlib.c
@@ -38,7 +38,7 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value)
+zlib_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -58,11 +58,11 @@ zlib_cmcompress(const struct varlena *value)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- tmp = (struct varlena *) palloc(valsize + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(valsize + header_size);
zp->next_in = (void *) VARDATA_ANY(value);
zp->avail_in = valsize;
zp->avail_out = valsize;
- zp->next_out = (void *) ((char *) tmp + TOAST_COMPRESS_HDRSZ);
+ zp->next_out = (void *) ((char *) tmp + header_size);
do
{
@@ -80,7 +80,7 @@ zlib_cmcompress(const struct varlena *value)
if (len > 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -94,7 +94,7 @@ zlib_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-zlib_cmdecompress(const struct varlena *value)
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
z_streamp zp;
@@ -108,8 +108,8 @@ zlib_cmdecompress(const struct varlena *value)
if (inflateInit(zp) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
- zp->next_in = (void *) ((char *) value + TOAST_COMPRESS_HDRSZ);
- zp->avail_in = VARSIZE(value) - TOAST_COMPRESS_HDRSZ;
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
zp->avail_out = VARRAWSIZE_4B_C(value);
result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6e86c66456..6db2cee3ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1487,6 +1487,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1508,7 +1509,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ed40d78d8d..1f98f28f6b 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -190,6 +190,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1010,6 +1024,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1261,6 +1276,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2272,6 +2292,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2565,6 +2586,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 15fb3bc40a..719530515b 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,114 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 26fe640e8e..57bffc4691 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0d0d29f9ce..4aad3829df 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2109,8 +2109,7 @@ CheckTargetCMAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc)
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5095c0e09c..018de03016 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4319,6 +4319,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5484,6 +5495,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 35efa064e5..5057cf7529 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2016,6 +2016,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3536,6 +3545,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 47d18a79ba..1de3e2354b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -882,6 +882,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5248,6 +5249,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6250,6 +6267,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..7ebbf51778 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2566,6 +2571,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2973,6 +2981,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3581,6 +3593,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index f905f3ecaf..dfc0c8b64f 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -26,18 +26,22 @@
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- ZLIB_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ ZLIB_COMPRESSION_ID = 1,
+ CUSTOME_COMPRESSION_ID = 3
} CompressionId;
-
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomeCompressionID(cmid) ((cmid) == CUSTOME_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression routine.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..3c280ce953 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the stored compression method
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e413914979..0f4aee5a6d 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1716,6 +1716,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2438,6 +2439,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 04c94f489d..bb4ca6a383 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 33c7132353..bf89651b4f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -107,4 +107,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | zlib | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index abf34eede0..273dcb83b9 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -52,4 +52,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION zlib PRESERVE (pglz);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, zlibtest;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.
In the updated version I have worked on some of the listed items
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?
Done
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.
Done
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.
1,3,5 points are still pending.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v8-0002-alter-table-set-compression.patchapplication/octet-stream; name=v8-0002-alter-table-set-compression.patchDownload
From 0268c0eb0564537adbee821b4c3f6917d0c2c835 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v8 2/5] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 69799f12a1..34018bc300 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3862,6 +3864,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4377,6 +4380,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4780,6 +4784,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5404,6 +5412,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14970,6 +14981,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index c377547551..9ef060f7eb 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2068,7 +2068,7 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index b7978cd22e..717e1b1593 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -612,5 +612,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v8-0001-Built-in-compression-method.patchapplication/octet-stream; name=v8-0001-Built-in-compression-method.patchDownload
From 449c5e742cfc02b99ec9c83c0fab15b92729d5ad Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v8 1/5] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 129 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 41 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1763 insertions(+), 584 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index 071d050ef0..41fa58944d 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index fd6777ae01..f083602f7f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..cbb30adafd
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,129 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2dcad80e69 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -345,6 +346,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 511f015a86..69799f12a1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6216,6 +6260,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7639,6 +7695,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11716,6 +11780,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17580,3 +17657,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0c055ed408..c377547551 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2059,6 +2062,79 @@ tupconv_map_for_subplan(ModifyTableState *mtstate, int whichplan)
return mtstate->mt_per_subplan_tupconv_maps[whichplan];
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2275,6 +2351,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..1791f3dc10 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,6 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e..b97a87c413 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,6 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 0dc03dd984..cd4c850ef8 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1063,6 +1064,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7a8bf76079..33d89f6e6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ff45e3fb8c..8378bee9a5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8615,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8644,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15677,6 +15695,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15701,6 +15720,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15736,6 +15758,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6bb0316bd9..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
@@ -2744,7 +2781,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 561fe1dff9..f842e27d25 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 22340baf1c..1b0e1ce866 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index cf2a9b4408..bf3019bb0b 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..a9f475b069 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v8-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v8-0004-Create-custom-compression-methods.patchDownload
From 30c5574083c1da0614134312e1ea6b57be956d48 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v8 4/5] Create custom compression methods
Provide syntax to create custom compression methods.
---
.../sgml/ref/create_compression_method.sgml | 87 +++++++++++++++
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 103 ++++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 +++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
27 files changed, 455 insertions(+), 46 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..e9d076194b
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,87 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through this interface.
+ </para>
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the access method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression access method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ access method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+</refentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index cbb30adafd..51bb2cdb28 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,15 +39,15 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -62,7 +62,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -70,9 +70,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -88,17 +88,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6e86c66456..6db2cee3ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1487,6 +1487,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1508,7 +1509,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ed40d78d8d..6014ca3d58 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -190,6 +190,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1010,6 +1024,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1261,6 +1276,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2272,6 +2292,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2565,6 +2586,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 8fe0a8bbe0..23fc793a9f 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,114 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 26fe640e8e..57bffc4691 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index af6d363157..fc44fbbaa1 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2111,8 +2111,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5095c0e09c..018de03016 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4319,6 +4319,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5484,6 +5495,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 35efa064e5..5057cf7529 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2016,6 +2016,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3536,6 +3545,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 47d18a79ba..1de3e2354b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -882,6 +882,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5248,6 +5249,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6250,6 +6267,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..7ebbf51778 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2566,6 +2571,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2973,6 +2981,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3581,6 +3593,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e413914979..25dc49dc0e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1716,6 +1716,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2438,6 +2439,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v8-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v8-0003-Add-support-for-PRESERVE.patchDownload
From 3aa4e4c458a62a05074a28c373f840daf8632ae1 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v8 3/5] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 216 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 16 +-
src/backend/nodes/equalfuncs.c | 14 +-
src/backend/nodes/outfuncs.c | 14 +-
src/backend/parser/gram.y | 44 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 15 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 451 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..23dce923c0 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,16 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..8fe0a8bbe0
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,216 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 34018bc300..7dfc4c6920 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -866,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2414,17 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2477,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2728,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4785,7 +4801,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6278,8 +6295,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6454,6 +6471,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6601,6 +6619,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11589,7 +11629,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11700,7 +11744,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11812,6 +11857,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14985,24 +15035,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15033,9 +15080,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17788,27 +17838,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 9ef060f7eb..af6d363157 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2077,6 +2078,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -2109,8 +2111,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1791f3dc10..5095c0e09c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,7 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2948,6 +2948,17 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5629,6 +5640,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..35efa064e5 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,15 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3692,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b97a87c413..4cb48c789a 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,7 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2850,6 +2850,15 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4198,6 +4207,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..47d18a79ba 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,35 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index cd4c850ef8..c9ead775a7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1067,7 +1067,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..bd5a2e3834 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..e413914979 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,19 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v8-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v8-0005-new-compression-method-extension-for-zlib.patchDownload
From f737d35a38690e841d44bfca16cabf11ca1542f4 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v8 5/5] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
Re: [HACKERS] Custom compression methods
On Thu, Oct 8, 2020 at 5:54 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
And is the oidvector actually needed? If we have the extra catalog,
can't we track this simply using the regular dependencies? So we'd have
the attcompression OID of the current compression method, and the
preserved values would be tracked in pg_depend.
If we go that route, we have to be sure that no such dependencies can
exist for any other reason. Otherwise, there would be confusion about
whether the dependency was there because values of that type were
being preserved in the table, or whether it was for the hypothetical
other reason. Now, admittedly, I can't quite think how that would
happen. For example, if the attribute default expression somehow
embedded a reference to a compression AM, that wouldn't cause this
problem, because the dependency would be on the attribute default
rather than the attribute itself. So maybe it's fine.
--
Robert Haas
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote:
On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.In the updated version I have worked on some of the listed items
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?Done
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.Done
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.1,3,5 points are still pending.
Thanks. I took a quick look at the patches and I think it seems fine. I
have one question, though - toast_compress_datum contains this code:
/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value);
if (!tmp)
return PointerGetDatum(NULL);
Shouldn't this really throw an error instead? I mean, if the compression
library returns NULL, isn't that an error?
regards
Show quoted text
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Oct 22, 2020 at 2:11 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote:
On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.In the updated version I have worked on some of the listed items
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?Done
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.Done
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.1,3,5 points are still pending.
Thanks. I took a quick look at the patches and I think it seems fine. I
have one question, though - toast_compress_datum contains this code:/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value);
if (!tmp)
return PointerGetDatum(NULL);Shouldn't this really throw an error instead? I mean, if the compression
library returns NULL, isn't that an error?
I don't think that we can throw an error here because pglz_compress
might return -1 if it finds that it can not reduce the size of the
data and we consider such data as "incompressible data" and return
NULL. In such a case the caller will try to compress another
attribute of the tuple. I think we can handle such cases in the
specific handler functions.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Oct 21, 2020 at 8:51 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Oct 8, 2020 at 5:54 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:And is the oidvector actually needed? If we have the extra catalog,
can't we track this simply using the regular dependencies? So we'd have
the attcompression OID of the current compression method, and the
preserved values would be tracked in pg_depend.If we go that route, we have to be sure that no such dependencies can
exist for any other reason. Otherwise, there would be confusion about
whether the dependency was there because values of that type were
being preserved in the table, or whether it was for the hypothetical
other reason. Now, admittedly, I can't quite think how that would
happen. For example, if the attribute default expression somehow
embedded a reference to a compression AM, that wouldn't cause this
problem, because the dependency would be on the attribute default
rather than the attribute itself. So maybe it's fine.
Yeah, and moreover in the new patchset, we are storing the compression
methods in the new catalog 'pg_compression' instead of merging with
the pg_am. So I think only for the preserve purpose we will maintain
the attribute -> pg_compression dependency so it should be fine.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Oct 22, 2020 at 10:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 22, 2020 at 2:11 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote:
On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.In the updated version I have worked on some of the listed items
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?Done
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.Done
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.1,3,5 points are still pending.
Thanks. I took a quick look at the patches and I think it seems fine. I
have one question, though - toast_compress_datum contains this code:/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value);
if (!tmp)
return PointerGetDatum(NULL);Shouldn't this really throw an error instead? I mean, if the compression
library returns NULL, isn't that an error?I don't think that we can throw an error here because pglz_compress
might return -1 if it finds that it can not reduce the size of the
data and we consider such data as "incompressible data" and return
NULL. In such a case the caller will try to compress another
attribute of the tuple. I think we can handle such cases in the
specific handler functions.
I have added the compression failure error in lz4.c, please refer
lz4_cmcompress in v9-0001 patch. Apart from that, I have also
supported the PRESERVE ALL syntax to preserve all the existing
compression methods. I have also rebased the patch on the current
head.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v9-0002-alter-table-set-compression.patchapplication/octet-stream; name=v9-0002-alter-table-set-compression.patchDownload
From 5ef9a179d8973d0fdb182d78e3793d5b3ad8915c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v9 2/5] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dc9c6cb1ab..43f79242e8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3862,6 +3864,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4379,6 +4382,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4782,6 +4786,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5406,6 +5414,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14972,6 +14983,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 42f1a1ae9d..643ba631ff 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1927,7 +1927,7 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index b7978cd22e..717e1b1593 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -612,5 +612,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v9-0001-Built-in-compression-method.patchapplication/octet-stream; name=v9-0001-Built-in-compression-method.patchDownload
From de582d7e3042db3f0d52dfe9bd9b4aab00afaabc Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v9 1/5] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 127 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 41 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1761 insertions(+), 584 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index ace4ed5dec..d6a8b35632 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index fd6777ae01..f083602f7f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..edc8f5ed22
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,127 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2dcad80e69 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -345,6 +346,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..dc9c6cb1ab 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6218,6 +6262,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7641,6 +7697,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11718,6 +11782,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17582,3 +17659,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index a33423c896..42f1a1ae9d 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1918,6 +1921,79 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2123,6 +2199,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..1791f3dc10 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,6 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e..b97a87c413 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,6 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 015b0538e3..4f04c128df 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1065,6 +1066,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7a8bf76079..33d89f6e6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ff45e3fb8c..8378bee9a5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8615,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8644,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15677,6 +15695,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15701,6 +15720,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15736,6 +15758,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 6bb0316bd9..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
@@ -2744,7 +2781,7 @@ describeOneTableDetails(const char *schemaname,
/* Show the stats target if it's not default */
if (strcmp(PQgetvalue(result, i, 8), "-1") != 0)
appendPQExpBuffer(&buf, "; STATISTICS %s",
- PQgetvalue(result, i, 8));
+ PQgetvalue(result, i, 8));
printTableAddFooter(&cont, buf.data);
}
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 561fe1dff9..f842e27d25 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2065,11 +2065,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bbcac69d48..85b2323121 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 492cdcf74c..acf4039222 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..a9f475b069 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v9-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v9-0004-Create-custom-compression-methods.patchDownload
From f328e79faefb9600ee2e57a40c819f32432b4364 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v9 4/5] Create custom compression methods
Provide syntax to create custom compression methods.
---
.../sgml/ref/create_compression_method.sgml | 87 +++++++++++++++
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 103 ++++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 +++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
27 files changed, 455 insertions(+), 46 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..e9d076194b
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,87 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through this interface.
+ </para>
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the access method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression access method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ access method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+</refentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index edc8f5ed22..52c167a35d 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,10 +39,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -50,7 +50,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -60,7 +60,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -68,9 +68,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -86,17 +86,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6e86c66456..6db2cee3ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1487,6 +1487,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1508,7 +1509,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ed40d78d8d..6014ca3d58 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -190,6 +190,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1010,6 +1024,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1261,6 +1276,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2272,6 +2292,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2565,6 +2586,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a1be6ef7be..d47d55b119 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,114 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 26fe640e8e..57bffc4691 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index efcbb663af..64fc13d4d0 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1970,8 +1970,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2d1f830d9d..53c8ad462e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4320,6 +4320,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5485,6 +5496,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 365957cfe7..f3023ae03f 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2016,6 +2016,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3537,6 +3546,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a1755c6fc7..35c69744e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -882,6 +882,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5256,6 +5257,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6258,6 +6275,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..7ebbf51778 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2566,6 +2571,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2973,6 +2981,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3581,6 +3593,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3c1ee794b5..f7870c469b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1717,6 +1717,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2439,6 +2440,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v9-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v9-0003-Add-support-for-PRESERVE.patchDownload
From b8cfafee6e141c9a8ff447154f0d47e8ae87f398 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v9 3/5] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 222 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 469 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..23dce923c0 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,16 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a1be6ef7be
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,222 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 43f79242e8..80bdb79530 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -866,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2414,17 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2477,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2728,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4787,7 +4803,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6280,8 +6297,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6456,6 +6473,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6603,6 +6621,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11591,7 +11631,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11702,7 +11746,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11814,6 +11859,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14987,24 +15037,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15035,9 +15082,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17790,27 +17840,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 643ba631ff..efcbb663af 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1936,6 +1937,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1968,8 +1970,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1791f3dc10..2d1f830d9d 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,7 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2948,6 +2948,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5629,6 +5641,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..365957cfe7 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3693,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b97a87c413..2ef640b3b7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,7 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2850,6 +2850,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4198,6 +4208,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..a1755c6fc7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4f04c128df..db7d381b53 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1069,7 +1069,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..bd5a2e3834 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..3c1ee794b5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +661,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v9-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v9-0005-new-compression-method-extension-for-zlib.patchDownload
From 828229b5f6382326f4ac62ebb6ddcd2e6b4e2bc0 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v9 5/5] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
Re: [HACKERS] Custom compression methods
On Thu, Oct 22, 2020 at 5:56 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 22, 2020 at 10:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 22, 2020 at 2:11 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote:
On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.In the updated version I have worked on some of the listed items
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?Done
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.Done
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.1,3,5 points are still pending.
Thanks. I took a quick look at the patches and I think it seems fine. I
have one question, though - toast_compress_datum contains this code:/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value);
if (!tmp)
return PointerGetDatum(NULL);Shouldn't this really throw an error instead? I mean, if the compression
library returns NULL, isn't that an error?I don't think that we can throw an error here because pglz_compress
might return -1 if it finds that it can not reduce the size of the
data and we consider such data as "incompressible data" and return
NULL. In such a case the caller will try to compress another
attribute of the tuple. I think we can handle such cases in the
specific handler functions.I have added the compression failure error in lz4.c, please refer
lz4_cmcompress in v9-0001 patch. Apart from that, I have also
supported the PRESERVE ALL syntax to preserve all the existing
compression methods. I have also rebased the patch on the current
head.
I have added the next patch to support the compression options. I am
storing the compression options only for the latest compression
method. Basically, based on this design we would be able to support
the options which are used only for compressions. As of now, the
compression option infrastructure is added and the compression options
for inbuilt method pglz and the external method zlib are added. Next,
I will work on adding the options for the lz4 method.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v10-0002-alter-table-set-compression.patchapplication/octet-stream; name=v10-0002-alter-table-set-compression.patchDownload
From 2356b89321a57a8a8268cd025456fe80518e4688 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v10 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dc9c6cb1ab..43f79242e8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3862,6 +3864,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4379,6 +4382,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4782,6 +4786,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5406,6 +5414,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14972,6 +14983,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 42f1a1ae9d..643ba631ff 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1927,7 +1927,7 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index b7978cd22e..717e1b1593 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -612,5 +612,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v10-0001-Built-in-compression-method.patchapplication/octet-stream; name=v10-0001-Built-in-compression-method.patchDownload
From b06eec9357d2c1ccc3df3448de3b73f2d67e57c6 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v10 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 127 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 39 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1760 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index ace4ed5dec..d6a8b35632 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index fd6777ae01..f083602f7f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..edc8f5ed22
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,127 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2dcad80e69 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -345,6 +346,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..dc9c6cb1ab 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6218,6 +6262,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7641,6 +7697,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11718,6 +11782,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17582,3 +17659,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index a33423c896..42f1a1ae9d 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1918,6 +1921,79 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2123,6 +2199,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..1791f3dc10 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,6 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e..b97a87c413 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,6 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 015b0538e3..4f04c128df 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1065,6 +1066,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7a8bf76079..33d89f6e6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index a6a8e6f2fd..c41466e41b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ff45e3fb8c..8378bee9a5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8469,6 +8470,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8554,6 +8556,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8572,7 +8583,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8599,6 +8615,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8627,6 +8644,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15677,6 +15695,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15701,6 +15720,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15736,6 +15758,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 07d640021c..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..8d1231a9cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2068,11 +2068,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index bbcac69d48..85b2323121 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 492cdcf74c..acf4039222 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..a9f475b069 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v10-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v10-0005-new-compression-method-extension-for-zlib.patchDownload
From b2030c766fc8700184011dd5ee7c64ae4d8b7654 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v10 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
v10-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v10-0003-Add-support-for-PRESERVE.patchDownload
From e123ac39c7b8ae9c35a31205bb6ae7724c75d273 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v10 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 222 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 470 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..733ee0cf20 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten. If PRESERVE ALL is specified then all the previous methods
+ will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a1be6ef7be
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,222 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 43f79242e8..80bdb79530 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -866,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2414,17 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2477,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2728,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4787,7 +4803,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6280,8 +6297,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6456,6 +6473,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6603,6 +6621,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11591,7 +11631,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11702,7 +11746,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11814,6 +11859,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14987,24 +15037,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15035,9 +15082,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17790,27 +17840,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 643ba631ff..efcbb663af 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1936,6 +1937,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1968,8 +1970,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1791f3dc10..2d1f830d9d 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,7 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2948,6 +2948,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5629,6 +5641,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..365957cfe7 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3693,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b97a87c413..2ef640b3b7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,7 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2850,6 +2850,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4198,6 +4208,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..a1755c6fc7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4f04c128df..db7d381b53 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1069,7 +1069,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..bd5a2e3834 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..3c1ee794b5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +661,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v10-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v10-0004-Create-custom-compression-methods.patchDownload
From d27d431c468134a4796f02b6e0f3782fc0e22b83 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v10 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
.../sgml/ref/create_compression_method.sgml | 87 +++++++++++++++
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 103 ++++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 +++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
27 files changed, 455 insertions(+), 46 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..e9d076194b
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,87 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through this interface.
+ </para>
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the access method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression access method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ access method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+</refentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index edc8f5ed22..52c167a35d 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,10 +39,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -50,7 +50,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -60,7 +60,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -68,9 +68,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -86,17 +86,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6e86c66456..6db2cee3ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1487,6 +1487,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1508,7 +1509,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ed40d78d8d..6014ca3d58 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -190,6 +190,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1010,6 +1024,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1261,6 +1276,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2272,6 +2292,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2565,6 +2586,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a1be6ef7be..d47d55b119 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,114 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 26fe640e8e..57bffc4691 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index efcbb663af..64fc13d4d0 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1970,8 +1970,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2d1f830d9d..53c8ad462e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4320,6 +4320,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5485,6 +5496,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 365957cfe7..f3023ae03f 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2016,6 +2016,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3537,6 +3546,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a1755c6fc7..35c69744e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -882,6 +882,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5256,6 +5257,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6258,6 +6275,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..7ebbf51778 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2566,6 +2571,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2973,6 +2981,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3581,6 +3593,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3c1ee794b5..f7870c469b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1717,6 +1717,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2439,6 +2440,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v10-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v10-0006-Support-compression-methods-options.patchDownload
From 1ee11643fa63c59b032aca475b1144c8f1f6063b Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 24 Oct 2020 20:16:48 +0530
Subject: [PATCH v10 6/6] Support compression methods options
---
contrib/cmzlib/zlib.c | 74 ++++++++++++++--
doc/src/sgml/ref/alter_table.sgml | 18 ++--
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/common/indextuple.c | 2 +-
src/backend/access/common/reloptions.c | 42 ++++++++++
src/backend/access/common/toast_internals.c | 9 +-
src/backend/access/compression/lz4.c | 4 +-
src/backend/access/compression/pglz.c | 93 ++++++++++++++++++++-
src/backend/access/table/toast_helper.c | 25 +++++-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 3 +-
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 18 +++-
src/backend/commands/foreigncmds.c | 44 ----------
src/backend/commands/tablecmds.c | 34 ++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/include/access/compressionapi.h | 14 +++-
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 3 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/create_cm.out | 14 ++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/create_cm.sql | 8 ++
31 files changed, 369 insertions(+), 93 deletions(-)
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
index f24fc1c936..7d0a256f6f 100644
--- a/contrib/cmzlib/zlib.c
+++ b/contrib/cmzlib/zlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,61 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +108,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -150,6 +204,8 @@ zlibhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
routine->cmcompress = zlib_cmcompress;
routine->cmdecompress = zlib_cmdecompress;
routine->cmdecompress_slice = NULL;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 733ee0cf20..58782153a6 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,17 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods. The PRESERVE list
- contains list of compression methods used on the column and determines
- which of them should be kept on the column. Without PRESERVE or if all
- the previous compression methods are not preserved then the table will
- be rewritten. If PRESERVE ALL is specified then all the previous methods
- will be preserved and the table will not be rewritten.
+ set from available built-in compression methods. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression methods used
+ on the column and determines which of them should be kept on the column.
+ Without PRESERVE or if all the previous compression methods are not
+ preserved then the table will be rewritten. If PRESERVE ALL is specified
+ then all the previous methods will be preserved and the table will not be
+ rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index f083602f7f..c32a7ae835 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -968,13 +968,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
can be set from the available built-in compression methods. The available
options are <literal>pglz</literal> and <literal>lz4</literal>. The
- default compression method is <literal>pglz</literal>.
+ default compression method is <literal>pglz</literal>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f81fc8ea66..616f31ba47 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -105,7 +105,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
att->attstorage == TYPSTORAGE_MAIN))
{
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, NULL);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..3185082034 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,45 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d69dd907c9..34343695ea 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -62,10 +62,11 @@ toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
CompressionId cmid;
CompressionRoutine *cmroutine;
@@ -80,10 +81,14 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = GetCompressionRoutine(cmoid);
cmid = GetCompressionId(cmoid);
+ if (cmroutine->cminitstate)
+ options = cmroutine->cminitstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value,
IsCustomCompression(cmid) ?
- TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ,
+ options);
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index 52c167a35d..fc5bcaa351 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize;
int32 len;
@@ -120,6 +120,8 @@ lz4handler(PG_FUNCTION_ARGS)
#else
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = NULL;
+ routine->cminitstate = NULL;
routine->cmcompress = lz4_cmcompress;
routine->cmdecompress = lz4_cmdecompress;
routine->cmdecompress_slice = lz4_cmdecompress_slice;
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 931394f779..607849a73d 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -15,11 +15,92 @@
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -122,6 +205,8 @@ pglzhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
routine->cmcompress = pglz_cmcompress;
routine->cmdecompress = pglz_cmdecompress;
routine->cmdecompress_slice = pglz_cmdecompress_slice;
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..1f842abfaf 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,12 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -43,6 +44,7 @@ toast_tuple_init(ToastTupleContext *ttc)
TupleDesc tupleDesc = ttc->ttc_rel->rd_att;
int numAttrs = tupleDesc->natts;
int i;
+ Datum attcmoptions;
ttc->ttc_flags = 0;
@@ -51,10 +53,28 @@ toast_tuple_init(ToastTupleContext *ttc)
Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
struct varlena *old_value;
struct varlena *new_value;
+ HeapTuple attr_tuple;
+ bool isNull;
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ ttc->ttc_attr[i].tai_cmoptions = NULL;
+ else
+ ttc->ttc_attr[i].tai_cmoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +250,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5f77b61a9b..d7b201fe5a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1150,6 +1158,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1407,7 +1416,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 2dcad80e69..4f898bb65e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -505,7 +505,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, NULL, indstate);
CatalogCloseIndexes(indstate);
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c40d25b301..6f8dd45d23 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -255,6 +255,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 0d647e912c..5c77a36699 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -680,6 +680,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index d47d55b119..21afea729a 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -228,7 +228,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
ListCell *cell;
@@ -247,6 +247,22 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionRoutine *routine = GetCompressionRoutine(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(compression->options);
+
+ pfree(routine);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 80bdb79530..358c56ec03 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -610,6 +610,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -814,6 +815,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -867,8 +870,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -922,8 +926,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -6136,6 +6143,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6298,7 +6306,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression, NULL);
+ colDef->compression, &acoptions, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6308,7 +6316,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -15046,9 +15054,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
ListCell *lc;
@@ -15082,7 +15092,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression,
+ &acoptions, &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15091,7 +15102,18 @@ ATExecSetCompression(AlteredTableInfo *tab,
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ /* update existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 53c8ad462e..30833e32a4 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2955,6 +2955,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f3023ae03f..919b0745f8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2622,6 +2622,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 2ef640b3b7..76473a9749 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2857,6 +2857,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 35c69744e2..9bac7f1932 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3454,11 +3455,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3466,14 +3473,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 4d4326ff57..fb8e44754e 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_compression_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -42,7 +43,12 @@ typedef enum CompressionId
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size,
+ void *options);
+typedef struct varlena *(*cmdecompress_function)(const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)(
const struct varlena *value,
@@ -61,11 +67,17 @@ struct CompressionRoutine
/* name of the compression method */
char cmname[64];
+ /* compression option check, can be NULL */
+ cmcheck_function cmcheck;
+
+ /* compression option intialize, can be NULL */
+ cminitstate_function cminitstate;
+
/* compression routine for the compression method */
cmcompress_function cmcompress;
/* decompression routine for the compression method */
- cmcompress_function cmdecompress;
+ cmdecompress_function cmdecompress;
/* slice decompression routine for the compression method */
cmdecompress_slice_function cmdecompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a88e3daa6a..9071e80714 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -33,6 +33,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 48ca172eae..53becd2057 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -67,7 +67,7 @@ do { \
#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
(((toast_compress_header_custom *) (ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index d31141c1a2..fd25321f1a 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 7b27df7464..69132d4b1d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -171,6 +171,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8952b2b70d..fcbdbfa401 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -137,6 +137,7 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,7 +151,7 @@ extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
+ Datum *acoptions, bool *need_rewrite);
extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f7870c469b..3bd758b7d8 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -635,6 +635,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 897a61689f..8a989e0ddd 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -121,4 +121,18 @@ SELECT length(f1) FROM cmmove2;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz2 | |
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (level '9');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8d4f9e5ea0..7935170622 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -98,6 +98,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -108,5 +109,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index a0a0aa13f0..0df4d3b60c 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -58,4 +58,12 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
SELECT length(f1) FROM cmmove2;
\d+ cmmove2
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (level '9');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Tue, Oct 27, 2020 at 10:54 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 22, 2020 at 5:56 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 22, 2020 at 10:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 22, 2020 at 2:11 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Wed, Oct 21, 2020 at 01:59:50PM +0530, Dilip Kumar wrote:
On Sat, Oct 17, 2020 at 11:34 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Oct 13, 2020 at 10:30 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Oct 12, 2020 at 7:32 PM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Mon, Oct 12, 2020 at 02:28:43PM +0530, Dilip Kumar wrote:
...
I have worked on this patch, so as discussed now I am maintaining the
preserved compression methods using dependency. Still PRESERVE ALL
syntax is not supported, I will work on that part.Cool, I'll take a look. What's your opinion on doing it this way? Do you
think it's cleaner / more elegant, or is it something contrary to what
the dependencies are meant to do?I think this looks much cleaner. Moreover, I feel that once we start
supporting the custom compression methods then we anyway have to
maintain the dependency so using that for finding the preserved
compression method is good option.I have also implemented the next set of patches.
0004 -> Provide a way to create custom compression methods
0005 -> Extention to implement lz4 as a custom compression method.In the updated version I have worked on some of the listed items
A pending list of items:
1. Provide support for handling the compression option
- As discussed up thread I will store the compression option of the
latest compression method in a new field in pg_atrribute table
2. As of now I have kept zlib as the second built-in option and lz4 as
a custom compression extension. In Offlist discussion with Robert, he
suggested that we should keep lz4 as the built-in method and we can
move zlib as an extension because lz4 is faster than zlib so better to
keep that as the built-in method. So in the next version, I will
change that. Any different opinion on this?Done
3. Improve the documentation, especially for create_compression_method.
4. By default support table compression method for the index.Done
5. Support the PRESERVE ALL option so that we can preserve all
existing lists of compression methods without providing the whole
list.1,3,5 points are still pending.
Thanks. I took a quick look at the patches and I think it seems fine. I
have one question, though - toast_compress_datum contains this code:/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value);
if (!tmp)
return PointerGetDatum(NULL);Shouldn't this really throw an error instead? I mean, if the compression
library returns NULL, isn't that an error?I don't think that we can throw an error here because pglz_compress
might return -1 if it finds that it can not reduce the size of the
data and we consider such data as "incompressible data" and return
NULL. In such a case the caller will try to compress another
attribute of the tuple. I think we can handle such cases in the
specific handler functions.I have added the compression failure error in lz4.c, please refer
lz4_cmcompress in v9-0001 patch. Apart from that, I have also
supported the PRESERVE ALL syntax to preserve all the existing
compression methods. I have also rebased the patch on the current
head.I have added the next patch to support the compression options. I am
storing the compression options only for the latest compression
method. Basically, based on this design we would be able to support
the options which are used only for compressions. As of now, the
compression option infrastructure is added and the compression options
for inbuilt method pglz and the external method zlib are added. Next,
I will work on adding the options for the lz4 method.
In the attached patch set I have also included the compression option
support for lz4. As of now, I have only supported the acceleration
for LZ4_compress_fast. There is also support for the dictionary-based
compression but if we try to support that then we will need the
dictionary for decompression also. Since we are only keeping the
options for the current compression methods, we can not support
dictionary-based options as of now.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v11-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v11-0004-Create-custom-compression-methods.patchDownload
From 8e64479e86091e6eca854c0753ae8370b1de996b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v11 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
.../sgml/ref/create_compression_method.sgml | 87 +++++++++++++++
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 103 ++++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 +++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
27 files changed, 455 insertions(+), 46 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..e9d076194b
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,87 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ <productname>PostgreSQL</productname> supports two internal
+ built-in compression methods (<literal>pglz</literal>
+ and <literal>zlib</literal>), and also allows to add more custom compression
+ methods through this interface.
+ </para>
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the access method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression access method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ access method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+</refentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index edc8f5ed22..52c167a35d 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,10 +39,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -50,7 +50,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -60,7 +60,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -68,9 +68,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -86,17 +86,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6e86c66456..6db2cee3ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1487,6 +1487,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1508,7 +1509,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ed40d78d8d..6014ca3d58 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -190,6 +190,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1010,6 +1024,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1261,6 +1276,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2272,6 +2292,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2565,6 +2586,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a1be6ef7be..d47d55b119 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,114 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 26fe640e8e..57bffc4691 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5257aeb66b..6653bd4d9a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1970,8 +1970,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2d1f830d9d..53c8ad462e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4320,6 +4320,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5485,6 +5496,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 365957cfe7..f3023ae03f 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2016,6 +2016,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3537,6 +3546,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a1755c6fc7..35c69744e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -882,6 +882,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5256,6 +5257,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6258,6 +6275,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..7ebbf51778 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2566,6 +2571,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2973,6 +2981,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3581,6 +3593,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3c1ee794b5..f7870c469b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1717,6 +1717,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2439,6 +2440,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v11-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v11-0005-new-compression-method-extension-for-zlib.patchDownload
From b2f76ffebb3d195095403e9eb33817b2690f958d Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v11 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
v11-0001-Built-in-compression-method.patchapplication/octet-stream; name=v11-0001-Built-in-compression-method.patchDownload
From 06281c69cd07e5a07d896a0613f26758b4cbb8d1 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v11 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 127 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 39 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1760 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index ace4ed5dec..d6a8b35632 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index fd6777ae01..f083602f7f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..edc8f5ed22
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,127 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2dcad80e69 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -345,6 +346,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..dc9c6cb1ab 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6218,6 +6262,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7641,6 +7697,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11718,6 +11782,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17582,3 +17659,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 29e07b7228..664e6d983c 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1918,6 +1921,79 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2123,6 +2199,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..1791f3dc10 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,6 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e..b97a87c413 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,6 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 015b0538e3..4f04c128df 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1065,6 +1066,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7a8bf76079..33d89f6e6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 6c79319164..7585101489 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 03f9d4d9e8..e90b230f76 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8472,6 +8473,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8557,6 +8559,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8575,7 +8586,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8602,6 +8618,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8630,6 +8647,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15680,6 +15698,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15704,6 +15723,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15739,6 +15761,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 07d640021c..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..8d1231a9cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2068,11 +2068,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a66870bcc0..8e8d7ac522 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 492cdcf74c..acf4039222 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 7a46a13252..eb3baf0231 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v11-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v11-0003-Add-support-for-PRESERVE.patchDownload
From f593daa79daa902c9a805c5aa985923288704a5a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v11 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 222 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 470 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..733ee0cf20 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten. If PRESERVE ALL is specified then all the previous methods
+ will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a1be6ef7be
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,222 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 43f79242e8..80bdb79530 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -866,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2414,17 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2477,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2728,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4787,7 +4803,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6280,8 +6297,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6456,6 +6473,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6603,6 +6621,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11591,7 +11631,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11702,7 +11746,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11814,6 +11859,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14987,24 +15037,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15035,9 +15082,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17790,27 +17840,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0fb5a1bbe5..5257aeb66b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1936,6 +1937,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1968,8 +1970,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1791f3dc10..2d1f830d9d 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,7 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2948,6 +2948,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5629,6 +5641,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..365957cfe7 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3693,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b97a87c413..2ef640b3b7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,7 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2850,6 +2850,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4198,6 +4208,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..a1755c6fc7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4f04c128df..db7d381b53 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1069,7 +1069,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..bd5a2e3834 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..3c1ee794b5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +661,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v11-0002-alter-table-set-compression.patchapplication/octet-stream; name=v11-0002-alter-table-set-compression.patchDownload
From 162c460dd41e965595ae9a9f3eeb2bbd28dab382 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v11 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dc9c6cb1ab..43f79242e8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3862,6 +3864,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4379,6 +4382,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4782,6 +4786,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5406,6 +5414,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14972,6 +14983,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 664e6d983c..0fb5a1bbe5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1927,7 +1927,7 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..a70e68f155 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v11-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v11-0006-Support-compression-methods-options.patchDownload
From eb55f2ba0cf4d2a7479f24d983c8f020b608e51d Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 24 Oct 2020 20:16:48 +0530
Subject: [PATCH v11 6/6] Support compression methods options
---
contrib/cmzlib/zlib.c | 75 +++++++++++++++--
doc/src/sgml/ref/alter_table.sgml | 18 ++--
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/common/indextuple.c | 2 +-
src/backend/access/common/reloptions.c | 42 ++++++++++
src/backend/access/common/toast_internals.c | 13 ++-
src/backend/access/compression/lz4.c | 69 ++++++++++++++-
src/backend/access/compression/pglz.c | 93 ++++++++++++++++++++-
src/backend/access/table/toast_helper.c | 25 +++++-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 3 +-
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 18 +++-
src/backend/commands/foreigncmds.c | 44 ----------
src/backend/commands/tablecmds.c | 34 ++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/include/access/compressionapi.h | 14 +++-
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 3 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/create_cm.out | 14 ++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/create_cm.sql | 8 ++
31 files changed, 436 insertions(+), 96 deletions(-)
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
index f24fc1c936..fa7c657bfa 100644
--- a/contrib/cmzlib/zlib.c
+++ b/contrib/cmzlib/zlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -150,6 +205,8 @@ zlibhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
routine->cmcompress = zlib_cmcompress;
routine->cmdecompress = zlib_cmdecompress;
routine->cmdecompress_slice = NULL;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 733ee0cf20..58782153a6 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,17 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods. The PRESERVE list
- contains list of compression methods used on the column and determines
- which of them should be kept on the column. Without PRESERVE or if all
- the previous compression methods are not preserved then the table will
- be rewritten. If PRESERVE ALL is specified then all the previous methods
- will be preserved and the table will not be rewritten.
+ set from available built-in compression methods. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression methods used
+ on the column and determines which of them should be kept on the column.
+ Without PRESERVE or if all the previous compression methods are not
+ preserved then the table will be rewritten. If PRESERVE ALL is specified
+ then all the previous methods will be preserved and the table will not be
+ rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index f083602f7f..c32a7ae835 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -968,13 +968,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
can be set from the available built-in compression methods. The available
options are <literal>pglz</literal> and <literal>lz4</literal>. The
- default compression method is <literal>pglz</literal>.
+ default compression method is <literal>pglz</literal>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f81fc8ea66..616f31ba47 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -105,7 +105,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
att->attstorage == TYPSTORAGE_MAIN))
{
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, NULL);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..3185082034 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,45 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d69dd907c9..de5d0f674e 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -62,10 +62,11 @@ toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
CompressionId cmid;
CompressionRoutine *cmroutine;
@@ -80,10 +81,18 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = GetCompressionRoutine(cmoid);
cmid = GetCompressionId(cmoid);
+ if (cmroutine->cminitstate)
+ options = cmroutine->cminitstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value,
IsCustomCompression(cmid) ?
- TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ,
+ options);
+
+ if (options != NULL)
+ pfree(options);
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index 52c167a35d..533bda5239 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -16,12 +16,70 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
#ifdef HAVE_LIBLZ4
#include "lz4.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -29,11 +87,12 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -41,9 +100,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -120,6 +179,8 @@ lz4handler(PG_FUNCTION_ARGS)
#else
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = lz4_cmcheck;
+ routine->cminitstate = lz4_cminitstate;
routine->cmcompress = lz4_cmcompress;
routine->cmdecompress = lz4_cmdecompress;
routine->cmdecompress_slice = lz4_cmdecompress_slice;
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 931394f779..607849a73d 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -15,11 +15,92 @@
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -122,6 +205,8 @@ pglzhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
routine->cmcompress = pglz_cmcompress;
routine->cmdecompress = pglz_cmdecompress;
routine->cmdecompress_slice = pglz_cmdecompress_slice;
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..1f842abfaf 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,12 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -43,6 +44,7 @@ toast_tuple_init(ToastTupleContext *ttc)
TupleDesc tupleDesc = ttc->ttc_rel->rd_att;
int numAttrs = tupleDesc->natts;
int i;
+ Datum attcmoptions;
ttc->ttc_flags = 0;
@@ -51,10 +53,28 @@ toast_tuple_init(ToastTupleContext *ttc)
Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
struct varlena *old_value;
struct varlena *new_value;
+ HeapTuple attr_tuple;
+ bool isNull;
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ ttc->ttc_attr[i].tai_cmoptions = NULL;
+ else
+ ttc->ttc_attr[i].tai_cmoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +250,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5f77b61a9b..d7b201fe5a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1150,6 +1158,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1407,7 +1416,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 2dcad80e69..4f898bb65e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -505,7 +505,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, NULL, indstate);
CatalogCloseIndexes(indstate);
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c40d25b301..6f8dd45d23 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -255,6 +255,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 0d647e912c..5c77a36699 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -680,6 +680,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index d47d55b119..21afea729a 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -228,7 +228,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
ListCell *cell;
@@ -247,6 +247,22 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionRoutine *routine = GetCompressionRoutine(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(compression->options);
+
+ pfree(routine);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 80bdb79530..358c56ec03 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -610,6 +610,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -814,6 +815,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -867,8 +870,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -922,8 +926,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -6136,6 +6143,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6298,7 +6306,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression, NULL);
+ colDef->compression, &acoptions, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6308,7 +6316,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -15046,9 +15054,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
ListCell *lc;
@@ -15082,7 +15092,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression,
+ &acoptions, &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15091,7 +15102,18 @@ ATExecSetCompression(AlteredTableInfo *tab,
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ /* update existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 53c8ad462e..30833e32a4 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2955,6 +2955,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f3023ae03f..919b0745f8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2622,6 +2622,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 2ef640b3b7..76473a9749 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2857,6 +2857,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 35c69744e2..9bac7f1932 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3454,11 +3455,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3466,14 +3473,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 4d4326ff57..fb8e44754e 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_compression_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -42,7 +43,12 @@ typedef enum CompressionId
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size,
+ void *options);
+typedef struct varlena *(*cmdecompress_function)(const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)(
const struct varlena *value,
@@ -61,11 +67,17 @@ struct CompressionRoutine
/* name of the compression method */
char cmname[64];
+ /* compression option check, can be NULL */
+ cmcheck_function cmcheck;
+
+ /* compression option intialize, can be NULL */
+ cminitstate_function cminitstate;
+
/* compression routine for the compression method */
cmcompress_function cmcompress;
/* decompression routine for the compression method */
- cmcompress_function cmdecompress;
+ cmdecompress_function cmdecompress;
/* slice decompression routine for the compression method */
cmdecompress_slice_function cmdecompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a88e3daa6a..9071e80714 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -33,6 +33,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 48ca172eae..53becd2057 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -67,7 +67,7 @@ do { \
#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
(((toast_compress_header_custom *) (ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index d31141c1a2..fd25321f1a 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 7b27df7464..69132d4b1d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -171,6 +171,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8952b2b70d..fcbdbfa401 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -137,6 +137,7 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,7 +151,7 @@ extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
+ Datum *acoptions, bool *need_rewrite);
extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f7870c469b..3bd758b7d8 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -635,6 +635,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 897a61689f..8a989e0ddd 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -121,4 +121,18 @@ SELECT length(f1) FROM cmmove2;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz2 | |
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (level '9');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8d4f9e5ea0..7935170622 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -98,6 +98,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -108,5 +109,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index a0a0aa13f0..0df4d3b60c 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -58,4 +58,12 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
SELECT length(f1) FROM cmmove2;
\d+ cmmove2
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (level '9');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote:
...
I have added the next patch to support the compression options. I am
storing the compression options only for the latest compression
method. Basically, based on this design we would be able to support
the options which are used only for compressions. As of now, the
compression option infrastructure is added and the compression options
for inbuilt method pglz and the external method zlib are added. Next,
I will work on adding the options for the lz4 method.In the attached patch set I have also included the compression option
support for lz4. As of now, I have only supported the acceleration
for LZ4_compress_fast. There is also support for the dictionary-based
compression but if we try to support that then we will need the
dictionary for decompression also. Since we are only keeping the
options for the current compression methods, we can not support
dictionary-based options as of now.
OK, thanks. Do you have any other plans to improve this patch series? I
plan to do some testing and review, but if you're likely to post another
version soon then I'd wait a bit.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
Re: [HACKERS] Custom compression methods
On Thu, Oct 29, 2020 at 12:31 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:
On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote:
...
I have added the next patch to support the compression options. I am
storing the compression options only for the latest compression
method. Basically, based on this design we would be able to support
the options which are used only for compressions. As of now, the
compression option infrastructure is added and the compression options
for inbuilt method pglz and the external method zlib are added. Next,
I will work on adding the options for the lz4 method.In the attached patch set I have also included the compression option
support for lz4. As of now, I have only supported the acceleration
for LZ4_compress_fast. There is also support for the dictionary-based
compression but if we try to support that then we will need the
dictionary for decompression also. Since we are only keeping the
options for the current compression methods, we can not support
dictionary-based options as of now.OK, thanks. Do you have any other plans to improve this patch series? I
plan to do some testing and review, but if you're likely to post another
version soon then I'd wait a bit.
There was some issue in create_compression_method.sgml and the
drop_compression_method.sgml was missing. I have fixed that in the
attached patch. Now I am not planning to change anything soon so you
can review. Thanks.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v12-0002-alter-table-set-compression.patchapplication/octet-stream; name=v12-0002-alter-table-set-compression.patchDownload
From 162c460dd41e965595ae9a9f3eeb2bbd28dab382 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v12 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dc9c6cb1ab..43f79242e8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3862,6 +3864,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4379,6 +4382,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4782,6 +4786,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE);
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5406,6 +5414,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -14972,6 +14983,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 664e6d983c..0fb5a1bbe5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1927,7 +1927,7 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 712c564858..289c8c97ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2266,6 +2266,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..a70e68f155 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 051fef925f..4656bb7e58 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1851,7 +1851,8 @@ typedef enum AlterTableType
AT_DetachPartition, /* DETACH PARTITION */
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
- AT_DropIdentity /* DROP IDENTITY */
+ AT_DropIdentity, /* DROP IDENTITY */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v12-0001-Built-in-compression-method.patchapplication/octet-stream; name=v12-0001-Built-in-compression-method.patchDownload
From 06281c69cd07e5a07d896a0613f26758b4cbb8d1 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v12 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 127 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 39 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/indexing.h | 5 +
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 43 +++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1760 insertions(+), 583 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index ace4ed5dec..d6a8b35632 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index fd6777ae01..f083602f7f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -966,6 +967,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..edc8f5ed22
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,127 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 76b2f5066f..ca184909a6 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 93cf6d4368..b4c6a33c8d 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -81,7 +81,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/,\
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 67144aa3c9..5f77b61a9b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0974f3e23a..2dcad80e69 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -345,6 +346,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a29c14bf1c..dc9c6cb1ab 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +855,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6218,6 +6262,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7641,6 +7697,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11718,6 +11782,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17582,3 +17659,27 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 29e07b7228..664e6d983c 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1918,6 +1921,79 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2123,6 +2199,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2b4d7654cc..1791f3dc10 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,6 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index e2d1b987bf..ab893ec26a 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,6 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 08a049232e..b97a87c413 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,6 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 480d168346..712c564858 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3371,11 +3373,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3384,8 +3387,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3430,6 +3433,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15097,6 +15109,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15615,6 +15628,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 015b0538e3..4f04c128df 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1065,6 +1066,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 7a8bf76079..33d89f6e6d 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 53d9ddf159..486ac98e5b 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -31,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -288,6 +289,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 6c79319164..7585101489 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 03f9d4d9e8..e90b230f76 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -381,6 +381,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8472,6 +8473,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8557,6 +8559,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8575,7 +8586,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8602,6 +8618,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8630,6 +8647,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15680,6 +15698,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15704,6 +15723,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15739,6 +15761,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index e0b42e8391..33f87a1708 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 07d640021c..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b2b4f1fd4d..8d1231a9cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2068,11 +2068,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/indexing.h b/src/include/catalog/indexing.h
index d86729dc6c..3e13f03cae 100644
--- a/src/include/catalog/indexing.h
+++ b/src/include/catalog/indexing.h
@@ -101,6 +101,11 @@ DECLARE_UNIQUE_INDEX(pg_attrdef_adrelid_adnum_index, 2656, on pg_attrdef using b
DECLARE_UNIQUE_INDEX(pg_attrdef_oid_index, 2657, on pg_attrdef using btree(oid oid_ops));
#define AttrDefaultOidIndexId 2657
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnam_index, 2658, on pg_attribute using btree(attrelid oid_ops, attname name_ops));
#define AttributeRelidNameIndexId 2658
DECLARE_UNIQUE_INDEX(pg_attribute_relid_attnum_index, 2659, on pg_attribute using btree(attrelid oid_ops, attnum int2_ops));
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a4cc80adad..7b27df7464 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..350557cec5
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,43 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index a66870bcc0..8e8d7ac522 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -942,6 +942,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7208,6 +7217,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index b2cec07416..1db95ec5d2 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -588,6 +588,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 60c2f45466..051fef925f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -647,6 +647,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index 1fc266dd65..9a23e0650a 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1045,21 +1045,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1067,11 +1067,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1100,46 +1100,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1171,11 +1171,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1184,10 +1184,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1197,10 +1197,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8538173ff8..8d4f9e5ea0 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 492cdcf74c..acf4039222 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3013,11 +3013,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3033,11 +3033,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3064,11 +3064,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 7a46a13252..eb3baf0231 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v12-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v12-0005-new-compression-method-extension-for-zlib.patchDownload
From bb15ba18d007ea5fbfd7d71c146eb8ba0dc6d625 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v12 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
v12-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v12-0004-Create-custom-compression-methods.patchDownload
From 870f0eb04a09495248597a4717ead34525a46c0c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v12 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
doc/src/sgml/ref/allfiles.sgml | 4 +-
.../sgml/ref/create_compression_method.sgml | 110 ++++++++++++++++++
doc/src/sgml/ref/drop_compression_method.sgml | 110 ++++++++++++++++++
doc/src/sgml/reference.sgml | 2 +
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 103 ++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 ++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
30 files changed, 593 insertions(+), 47 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
create mode 100644 doc/src/sgml/ref/drop_compression_method.sgml
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 0f0064150c..354ebb4b75 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -62,6 +62,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
@@ -105,10 +106,11 @@ Complete list of usable sgml source files in this directory.
<!ENTITY delete SYSTEM "delete.sgml">
<!ENTITY discard SYSTEM "discard.sgml">
<!ENTITY do SYSTEM "do.sgml">
-<!ENTITY dropAccessMethod SYSTEM "drop_access_method.sgml">
+<!ENTITY dropCompressionMethod SYSTEM "drop_access_method.sgml">
<!ENTITY dropAggregate SYSTEM "drop_aggregate.sgml">
<!ENTITY dropCast SYSTEM "drop_cast.sgml">
<!ENTITY dropCollation SYSTEM "drop_collation.sgml">
+<!ENTITY dropAccessMethod SYSTEM "drop_compression_method.sgml">
<!ENTITY dropConversion SYSTEM "drop_conversion.sgml">
<!ENTITY dropDatabase SYSTEM "drop_database.sgml">
<!ENTITY dropDomain SYSTEM "drop_domain.sgml">
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..560a7f2942
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,110 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the compression method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ compression method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Create an compression method <literal>pglz2</literal> with
+ handler function <literal>pglzhandler</literal>:
+<programlisting>
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> is a
+ <productname>PostgreSQL</productname> extension.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-drop-compression-method"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/drop_compression_method.sgml b/doc/src/sgml/ref/drop_compression_method.sgml
new file mode 100644
index 0000000000..2a7399f6c2
--- /dev/null
+++ b/doc/src/sgml/ref/drop_compression_method.sgml
@@ -0,0 +1,110 @@
+<!--
+doc/src/sgml/ref/drop_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-drop-compression-method">
+ <indexterm zone="sql-drop-compression-method">
+ <primary>DROP COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>DROP COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>DROP COMPRESSION METHOD</refname>
+ <refpurpose>remove a compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+DROP COMPRESSION METHOD [ IF EXISTS ] <replaceable class="parameter">name</replaceable> [ CASCADE | RESTRICT ]
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>DROP COMPRESSION METHOD</command> removes an existing compression method.
+ Only superusers can drop compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>IF EXISTS</literal></term>
+ <listitem>
+ <para>
+ Do not throw an error if the compression method does not exist.
+ A notice is issued in this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of an existing compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>CASCADE</literal></term>
+ <listitem>
+ <para>
+ Automatically drop objects that depend on the compression method
+ and in turn all objects that depend on those objects
+ (see <xref linkend="ddl-depend"/>).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>RESTRICT</literal></term>
+ <listitem>
+ <para>
+ Refuse to drop the compression method if any objects depend on it.
+ This is the default.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Drop the compression method <literal>pglz2</literal>:
+<programlisting>
+DROP COMPRESSION METHOD pglz2;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ <command>DROP COMPRESSION METHOD</command> is a
+ <productname>PostgreSQL</productname> extension.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-create-compression-method"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index dd2bddab8c..381f52a359 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -90,6 +90,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
@@ -137,6 +138,7 @@
&dropAggregate;
&dropCast;
&dropCollation;
+ &dropCompressionMethod;
&dropConversion;
&dropDatabase;
&dropDomain;
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index edc8f5ed22..52c167a35d 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,10 +39,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -50,7 +50,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -60,7 +60,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -68,9 +68,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -86,17 +86,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 6e86c66456..6db2cee3ca 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1487,6 +1487,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1508,7 +1509,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index ed40d78d8d..6014ca3d58 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -190,6 +190,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1010,6 +1024,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1261,6 +1276,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2272,6 +2292,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2565,6 +2586,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a1be6ef7be..d47d55b119 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,114 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 26fe640e8e..57bffc4691 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5257aeb66b..6653bd4d9a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1970,8 +1970,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2d1f830d9d..53c8ad462e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4320,6 +4320,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5485,6 +5496,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 365957cfe7..f3023ae03f 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2016,6 +2016,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3537,6 +3546,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index a1755c6fc7..35c69744e2 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -882,6 +882,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5256,6 +5257,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6258,6 +6275,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index 9a35147b26..7ebbf51778 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2566,6 +2571,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2973,6 +2981,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3581,6 +3593,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 3c1ee794b5..f7870c469b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1717,6 +1717,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2439,6 +2440,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v12-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v12-0003-Add-support-for-PRESERVE.patchDownload
From f593daa79daa902c9a805c5aa985923288704a5a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v12 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 222 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 470 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..733ee0cf20 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten. If PRESERVE ALL is specified then all the previous methods
+ will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index f515e2c308..6e86c66456 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -180,7 +181,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1506,6 +1508,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2843,6 +2846,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 4815f6ca7e..ed40d78d8d 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -30,6 +31,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3907,6 +3909,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4479,6 +4490,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5763,6 +5778,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a1be6ef7be
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,222 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 8bb17c34f5..26fe640e8e 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 43f79242e8..80bdb79530 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,8 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
Relation partitionTbl);
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -866,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2414,17 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2477,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2728,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4787,7 +4803,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecDetachPartition(rel, ((PartitionCmd *) cmd->def)->name);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6280,8 +6297,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6456,6 +6473,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6603,6 +6621,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11591,7 +11631,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11702,7 +11746,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11814,6 +11859,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -14987,24 +15037,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15035,9 +15082,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17790,27 +17840,3 @@ ATDetachCheckNoForeignKeyRefs(Relation partition)
table_close(rel, NoLock);
}
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0fb5a1bbe5..5257aeb66b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1936,6 +1937,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1968,8 +1970,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1791f3dc10..2d1f830d9d 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2928,7 +2928,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2948,6 +2948,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5629,6 +5641,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ab893ec26a..365957cfe7 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2588,7 +2588,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2608,6 +2608,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3683,6 +3693,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b97a87c413..2ef640b3b7 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2832,7 +2832,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2850,6 +2850,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4198,6 +4208,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 289c8c97ee..a1755c6fc7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2267,12 +2269,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3387,7 +3389,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,13 +3444,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4f04c128df..db7d381b53 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1069,7 +1069,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index a8f7e9965b..bd5a2e3834 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 4656bb7e58..3c1ee794b5 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -647,7 +661,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 6ace7662ee..a5abdcaf85 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2072,7 +2073,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2091,6 +2092,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
@@ -2099,7 +2101,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(8 rows)
+(9 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v12-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v12-0006-Support-compression-methods-options.patchDownload
From 68ea22a213e7816b6f255d01b4e869745cdb7527 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 24 Oct 2020 20:16:48 +0530
Subject: [PATCH v12 6/6] Support compression methods options
---
contrib/cmzlib/zlib.c | 75 +++++++++++++++--
doc/src/sgml/ref/alter_table.sgml | 18 ++--
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/common/indextuple.c | 2 +-
src/backend/access/common/reloptions.c | 42 ++++++++++
src/backend/access/common/toast_internals.c | 13 ++-
src/backend/access/compression/lz4.c | 69 ++++++++++++++-
src/backend/access/compression/pglz.c | 93 ++++++++++++++++++++-
src/backend/access/table/toast_helper.c | 25 +++++-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 3 +-
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 18 +++-
src/backend/commands/foreigncmds.c | 44 ----------
src/backend/commands/tablecmds.c | 34 ++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/include/access/compressionapi.h | 14 +++-
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 3 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/create_cm.out | 14 ++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/create_cm.sql | 8 ++
31 files changed, 436 insertions(+), 96 deletions(-)
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
index f24fc1c936..fa7c657bfa 100644
--- a/contrib/cmzlib/zlib.c
+++ b/contrib/cmzlib/zlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -150,6 +205,8 @@ zlibhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
routine->cmcompress = zlib_cmcompress;
routine->cmdecompress = zlib_cmdecompress;
routine->cmdecompress_slice = NULL;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 733ee0cf20..58782153a6 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,17 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods. The PRESERVE list
- contains list of compression methods used on the column and determines
- which of them should be kept on the column. Without PRESERVE or if all
- the previous compression methods are not preserved then the table will
- be rewritten. If PRESERVE ALL is specified then all the previous methods
- will be preserved and the table will not be rewritten.
+ set from available built-in compression methods. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression methods used
+ on the column and determines which of them should be kept on the column.
+ Without PRESERVE or if all the previous compression methods are not
+ preserved then the table will be rewritten. If PRESERVE ALL is specified
+ then all the previous methods will be preserved and the table will not be
+ rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index f083602f7f..c32a7ae835 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -968,13 +968,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
can be set from the available built-in compression methods. The available
options are <literal>pglz</literal> and <literal>lz4</literal>. The
- default compression method is <literal>pglz</literal>.
+ default compression method is <literal>pglz</literal>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f81fc8ea66..616f31ba47 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -105,7 +105,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
att->attstorage == TYPSTORAGE_MAIN))
{
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, NULL);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..3185082034 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,45 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d69dd907c9..de5d0f674e 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -62,10 +62,11 @@ toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
CompressionId cmid;
CompressionRoutine *cmroutine;
@@ -80,10 +81,18 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = GetCompressionRoutine(cmoid);
cmid = GetCompressionId(cmoid);
+ if (cmroutine->cminitstate)
+ options = cmroutine->cminitstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value,
IsCustomCompression(cmid) ?
- TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ,
+ options);
+
+ if (options != NULL)
+ pfree(options);
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index 52c167a35d..533bda5239 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -16,12 +16,70 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
#ifdef HAVE_LIBLZ4
#include "lz4.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -29,11 +87,12 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -41,9 +100,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -120,6 +179,8 @@ lz4handler(PG_FUNCTION_ARGS)
#else
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = lz4_cmcheck;
+ routine->cminitstate = lz4_cminitstate;
routine->cmcompress = lz4_cmcompress;
routine->cmdecompress = lz4_cmdecompress;
routine->cmdecompress_slice = lz4_cmdecompress_slice;
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 931394f779..607849a73d 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -15,11 +15,92 @@
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -122,6 +205,8 @@ pglzhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
routine->cmcompress = pglz_cmcompress;
routine->cmdecompress = pglz_cmdecompress;
routine->cmdecompress_slice = pglz_cmdecompress_slice;
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..1f842abfaf 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,12 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -43,6 +44,7 @@ toast_tuple_init(ToastTupleContext *ttc)
TupleDesc tupleDesc = ttc->ttc_rel->rd_att;
int numAttrs = tupleDesc->natts;
int i;
+ Datum attcmoptions;
ttc->ttc_flags = 0;
@@ -51,10 +53,28 @@ toast_tuple_init(ToastTupleContext *ttc)
Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
struct varlena *old_value;
struct varlena *new_value;
+ HeapTuple attr_tuple;
+ bool isNull;
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ ttc->ttc_attr[i].tai_cmoptions = NULL;
+ else
+ ttc->ttc_attr[i].tai_cmoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +250,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 5f77b61a9b..d7b201fe5a 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1150,6 +1158,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1407,7 +1416,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 2dcad80e69..4f898bb65e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -505,7 +505,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, NULL, indstate);
CatalogCloseIndexes(indstate);
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index c40d25b301..6f8dd45d23 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -255,6 +255,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 0d647e912c..5c77a36699 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -680,6 +680,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index d47d55b119..21afea729a 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -228,7 +228,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
ListCell *cell;
@@ -247,6 +247,22 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionRoutine *routine = GetCompressionRoutine(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(compression->options);
+
+ pfree(routine);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 80bdb79530..358c56ec03 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -610,6 +610,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -814,6 +815,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -867,8 +870,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -922,8 +926,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -6136,6 +6143,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6298,7 +6306,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression, NULL);
+ colDef->compression, &acoptions, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6308,7 +6316,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -15046,9 +15054,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
ListCell *lc;
@@ -15082,7 +15092,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression,
+ &acoptions, &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15091,7 +15102,18 @@ ATExecSetCompression(AlteredTableInfo *tab,
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ /* update existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 53c8ad462e..30833e32a4 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2955,6 +2955,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f3023ae03f..919b0745f8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2622,6 +2622,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 2ef640b3b7..76473a9749 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2857,6 +2857,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 35c69744e2..9bac7f1932 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3454,11 +3455,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3466,14 +3473,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 4d4326ff57..fb8e44754e 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_compression_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -42,7 +43,12 @@ typedef enum CompressionId
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size,
+ void *options);
+typedef struct varlena *(*cmdecompress_function)(const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)(
const struct varlena *value,
@@ -61,11 +67,17 @@ struct CompressionRoutine
/* name of the compression method */
char cmname[64];
+ /* compression option check, can be NULL */
+ cmcheck_function cmcheck;
+
+ /* compression option intialize, can be NULL */
+ cminitstate_function cminitstate;
+
/* compression routine for the compression method */
cmcompress_function cmcompress;
/* decompression routine for the compression method */
- cmcompress_function cmdecompress;
+ cmdecompress_function cmdecompress;
/* slice decompression routine for the compression method */
cmdecompress_slice_function cmdecompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a88e3daa6a..9071e80714 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -33,6 +33,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 48ca172eae..53becd2057 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -67,7 +67,7 @@ do { \
#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
(((toast_compress_header_custom *) (ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index d31141c1a2..fd25321f1a 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 7b27df7464..69132d4b1d 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -171,6 +171,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8952b2b70d..fcbdbfa401 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -137,6 +137,7 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,7 +151,7 @@ extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
+ Datum *acoptions, bool *need_rewrite);
extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f7870c469b..3bd758b7d8 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -635,6 +635,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 897a61689f..c9f324ed4c 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -121,4 +121,18 @@ SELECT length(f1) FROM cmmove2;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz2 | |
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 8d4f9e5ea0..7935170622 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -98,6 +98,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -108,5 +109,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index a0a0aa13f0..9135ce5e5d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -58,4 +58,12 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
SELECT length(f1) FROM cmmove2;
\d+ cmmove2
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Thu, Oct 29, 2020 at 12:07 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 29, 2020 at 12:31 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote:
...
I have added the next patch to support the compression options. I am
storing the compression options only for the latest compression
method. Basically, based on this design we would be able to support
the options which are used only for compressions. As of now, the
compression option infrastructure is added and the compression options
for inbuilt method pglz and the external method zlib are added. Next,
I will work on adding the options for the lz4 method.In the attached patch set I have also included the compression option
support for lz4. As of now, I have only supported the acceleration
for LZ4_compress_fast. There is also support for the dictionary-based
compression but if we try to support that then we will need the
dictionary for decompression also. Since we are only keeping the
options for the current compression methods, we can not support
dictionary-based options as of now.OK, thanks. Do you have any other plans to improve this patch series? I
plan to do some testing and review, but if you're likely to post another
version soon then I'd wait a bit.There was some issue in create_compression_method.sgml and the
drop_compression_method.sgml was missing. I have fixed that in the
attached patch. Now I am not planning to change anything soon so you
can review. Thanks.
The patches were not applying on the current head so I have re-based them.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v13-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v13-0003-Add-support-for-PRESERVE.patchDownload
From bcd066e801f821b30d6ba5adcaa7611060edd9fe Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v13 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 222 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 470 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..733ee0cf20 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten. If PRESERVE ALL is specified then all the previous methods
+ will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b0d037600e..076ef56ba5 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -181,7 +182,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1585,6 +1587,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2977,6 +2980,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 3662a8ebb6..8f95da758f 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -29,6 +30,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3906,6 +3908,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4478,6 +4489,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5762,6 +5777,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a1be6ef7be
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,222 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 3ffba4e63e..172242ae37 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 366ffb792e..e82b362bad 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -563,8 +566,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -589,6 +590,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -867,7 +869,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -937,6 +939,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2415,17 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2462,7 +2478,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2713,12 +2729,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4803,7 +4819,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6296,8 +6313,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6472,6 +6489,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6619,6 +6637,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11623,7 +11663,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11734,7 +11778,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11846,6 +11891,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15019,24 +15069,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15067,9 +15114,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17836,27 +17886,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0fb5a1bbe5..5257aeb66b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1936,6 +1937,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1968,8 +1970,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 279757e63e..5413bd92a4 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2929,7 +2929,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2949,6 +2949,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5618,6 +5630,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 635b0ea10c..09882c9375 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2581,7 +2581,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2601,6 +2601,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3673,6 +3683,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 1040311bf3..22757dfea9 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2833,7 +2833,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2851,6 +2851,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4199,6 +4209,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8acf53d2ed..deb82ea660 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2266,12 +2268,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3394,7 +3396,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3449,13 +3451,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 7482461dbe..5ff4b31703 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1070,7 +1070,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 901d5019cd..fcc834e371 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f07825edf1..e0a51f0145 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -625,6 +625,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -648,7 +662,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 93a8736a3f..e006c04c7c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
@@ -2074,7 +2075,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(10 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,6 +2094,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
@@ -2103,7 +2105,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(10 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v13-0001-Built-in-compression-method.patchapplication/octet-stream; name=v13-0001-Built-in-compression-method.patchDownload
From d40f2a5575e8d8daa4827fb912b32b1ccfcf1e4e Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v13 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 2 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 127 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 39 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 48 ++++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1761 insertions(+), 584 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index ace4ed5dec..d6a8b35632 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index bc59a2d77d..c559d1ffd5 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -967,6 +968,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 6774f597a4..7523e4142b 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Datum cvalue = toast_compress_datum(value, InvalidOid);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..edc8f5ed22
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,127 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index a7ed93fdc1..2bcdf69686 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 2519771210..5efb70bdea 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -76,7 +76,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/, $(CATALOG_H
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4cd7d76938..a2e2763a7c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b88b4a1f12..200c2d24c7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e3cfaf8b07..8c167cd098 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -560,7 +561,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -854,6 +856,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2706,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6234,6 +6278,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7673,6 +7729,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11750,6 +11814,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17628,3 +17705,27 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 29e07b7228..664e6d983c 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1918,6 +1921,79 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2123,6 +2199,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3031c52991..279757e63e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2929,6 +2929,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 9aa853748d..635b0ea10c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2581,6 +2581,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4504b1503b..1040311bf3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2833,6 +2833,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 95e256883b..11fa23cc29 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3378,11 +3380,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3391,8 +3394,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3437,6 +3440,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3666,6 +3677,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15185,6 +15197,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15703,6 +15716,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 254c0f65c2..7482461dbe 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1066,6 +1067,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1bd68011c..748cd540ba 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 809b27a038..b46d876965 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -30,6 +30,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -287,6 +288,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3c276f2bcb..01470f5eb1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -385,6 +385,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8551,6 +8552,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8636,6 +8638,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8654,7 +8665,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8681,6 +8697,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8709,6 +8726,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15739,6 +15757,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15763,6 +15782,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15798,6 +15820,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 317bb83970..2912e4c0dc 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 07d640021c..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5238a960f7..afa1aac594 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,11 +2093,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index cdf75a2380..c506d31029 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..8b3c7a1cc1
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,48 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c01da4bf01..d2344758f2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7219,6 +7228,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 21a467a7a7..fb6cbaa575 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -578,6 +578,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7ef9b0eac0..7818bd718f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -648,6 +648,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -686,6 +687,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..9029f76147 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 097ff5d111..917cadd4ef 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3016,11 +3016,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3036,11 +3036,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3067,11 +3067,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..a9f475b069 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v13-0002-alter-table-set-compression.patchapplication/octet-stream; name=v13-0002-alter-table-set-compression.patchDownload
From 64e92b8af566a30191438031b7175c5c1248bc84 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v13 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8c167cd098..366ffb792e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3863,6 +3865,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4390,6 +4393,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4798,6 +4802,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5422,6 +5430,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -15004,6 +15015,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 664e6d983c..0fb5a1bbe5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1927,7 +1927,7 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 11fa23cc29..8acf53d2ed 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2265,6 +2265,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..a70e68f155 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7818bd718f..f07825edf1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1857,7 +1857,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v13-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v13-0005-new-compression-method-extension-for-zlib.patchDownload
From b8d6292a3cbbe8b8f68bf52e6ad543ea86120d10 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v13 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
v13-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v13-0004-Create-custom-compression-methods.patchDownload
From 113284dd29456ce95c57e11bc4509965c001914f Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v13 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
doc/src/sgml/ref/allfiles.sgml | 4 +-
.../sgml/ref/create_compression_method.sgml | 110 ++++++++++++++++++
doc/src/sgml/ref/drop_compression_method.sgml | 110 ++++++++++++++++++
doc/src/sgml/reference.sgml | 2 +
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 104 +++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 ++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
30 files changed, 594 insertions(+), 47 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
create mode 100644 doc/src/sgml/ref/drop_compression_method.sgml
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 0f0064150c..354ebb4b75 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -62,6 +62,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
@@ -105,10 +106,11 @@ Complete list of usable sgml source files in this directory.
<!ENTITY delete SYSTEM "delete.sgml">
<!ENTITY discard SYSTEM "discard.sgml">
<!ENTITY do SYSTEM "do.sgml">
-<!ENTITY dropAccessMethod SYSTEM "drop_access_method.sgml">
+<!ENTITY dropCompressionMethod SYSTEM "drop_access_method.sgml">
<!ENTITY dropAggregate SYSTEM "drop_aggregate.sgml">
<!ENTITY dropCast SYSTEM "drop_cast.sgml">
<!ENTITY dropCollation SYSTEM "drop_collation.sgml">
+<!ENTITY dropAccessMethod SYSTEM "drop_compression_method.sgml">
<!ENTITY dropConversion SYSTEM "drop_conversion.sgml">
<!ENTITY dropDatabase SYSTEM "drop_database.sgml">
<!ENTITY dropDomain SYSTEM "drop_domain.sgml">
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..560a7f2942
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,110 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the compression method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ compression method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Create an compression method <literal>pglz2</literal> with
+ handler function <literal>pglzhandler</literal>:
+<programlisting>
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> is a
+ <productname>PostgreSQL</productname> extension.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-drop-compression-method"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/drop_compression_method.sgml b/doc/src/sgml/ref/drop_compression_method.sgml
new file mode 100644
index 0000000000..2a7399f6c2
--- /dev/null
+++ b/doc/src/sgml/ref/drop_compression_method.sgml
@@ -0,0 +1,110 @@
+<!--
+doc/src/sgml/ref/drop_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-drop-compression-method">
+ <indexterm zone="sql-drop-compression-method">
+ <primary>DROP COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>DROP COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>DROP COMPRESSION METHOD</refname>
+ <refpurpose>remove a compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+DROP COMPRESSION METHOD [ IF EXISTS ] <replaceable class="parameter">name</replaceable> [ CASCADE | RESTRICT ]
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>DROP COMPRESSION METHOD</command> removes an existing compression method.
+ Only superusers can drop compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>IF EXISTS</literal></term>
+ <listitem>
+ <para>
+ Do not throw an error if the compression method does not exist.
+ A notice is issued in this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of an existing compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>CASCADE</literal></term>
+ <listitem>
+ <para>
+ Automatically drop objects that depend on the compression method
+ and in turn all objects that depend on those objects
+ (see <xref linkend="ddl-depend"/>).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>RESTRICT</literal></term>
+ <listitem>
+ <para>
+ Refuse to drop the compression method if any objects depend on it.
+ This is the default.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Drop the compression method <literal>pglz2</literal>:
+<programlisting>
+DROP COMPRESSION METHOD pglz2;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ <command>DROP COMPRESSION METHOD</command> is a
+ <productname>PostgreSQL</productname> extension.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-create-compression-method"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index dd2bddab8c..381f52a359 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -90,6 +90,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
@@ -137,6 +138,7 @@
&dropAggregate;
&dropCast;
&dropCollation;
+ &dropCompressionMethod;
&dropConversion;
&dropDatabase;
&dropDomain;
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index edc8f5ed22..52c167a35d 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,10 +39,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -50,7 +50,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -60,7 +60,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -68,9 +68,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -86,17 +86,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 076ef56ba5..c928610c14 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1566,6 +1566,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1587,7 +1588,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8f95da758f..f80305cf44 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -189,6 +189,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1009,6 +1023,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1260,6 +1275,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2271,6 +2291,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2564,6 +2585,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a1be6ef7be..ce6f7f83f5 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,115 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, CompressionIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 172242ae37..b2a07bc977 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5257aeb66b..6653bd4d9a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1970,8 +1970,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5413bd92a4..707c6ea90b 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4312,6 +4312,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5474,6 +5485,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 09882c9375..0c17cec725 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2008,6 +2008,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3527,6 +3536,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index deb82ea660..d21cb7c94c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -881,6 +881,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5263,6 +5264,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6265,6 +6282,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index f398027fa6..dfd79d1007 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2562,6 +2567,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2969,6 +2977,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3573,6 +3585,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e0a51f0145..89478ddea3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1722,6 +1722,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2435,6 +2436,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v13-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v13-0006-Support-compression-methods-options.patchDownload
From a05eeb37274ccd5e848d137d6812a29103a1a710 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 24 Oct 2020 20:16:48 +0530
Subject: [PATCH v13 6/6] Support compression methods options
---
contrib/cmzlib/zlib.c | 75 +++++++++++++++--
doc/src/sgml/ref/alter_table.sgml | 18 ++--
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/brin/brin_tuple.c | 2 +-
src/backend/access/common/indextuple.c | 2 +-
src/backend/access/common/reloptions.c | 42 ++++++++++
src/backend/access/common/toast_internals.c | 13 ++-
src/backend/access/compression/lz4.c | 69 ++++++++++++++-
src/backend/access/compression/pglz.c | 93 ++++++++++++++++++++-
src/backend/access/table/toast_helper.c | 25 +++++-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 3 +-
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 18 +++-
src/backend/commands/foreigncmds.c | 44 ----------
src/backend/commands/tablecmds.c | 34 ++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/include/access/compressionapi.h | 14 +++-
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 3 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/create_cm.out | 14 ++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/create_cm.sql | 8 ++
32 files changed, 437 insertions(+), 97 deletions(-)
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
index f24fc1c936..fa7c657bfa 100644
--- a/contrib/cmzlib/zlib.c
+++ b/contrib/cmzlib/zlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -150,6 +205,8 @@ zlibhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
routine->cmcompress = zlib_cmcompress;
routine->cmdecompress = zlib_cmdecompress;
routine->cmdecompress_slice = NULL;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 733ee0cf20..58782153a6 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,17 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods. The PRESERVE list
- contains list of compression methods used on the column and determines
- which of them should be kept on the column. Without PRESERVE or if all
- the previous compression methods are not preserved then the table will
- be rewritten. If PRESERVE ALL is specified then all the previous methods
- will be preserved and the table will not be rewritten.
+ set from available built-in compression methods. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression methods used
+ on the column and determines which of them should be kept on the column.
+ Without PRESERVE or if all the previous compression methods are not
+ preserved then the table will be rewritten. If PRESERVE ALL is specified
+ then all the previous methods will be preserved and the table will not be
+ rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index c559d1ffd5..466945fcfe 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -969,13 +969,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
can be set from the available built-in compression methods. The available
options are <literal>pglz</literal> and <literal>lz4</literal>. The
- default compression method is <literal>pglz</literal>.
+ default compression method is <literal>pglz</literal>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 7523e4142b..785eb4d181 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,7 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value, InvalidOid);
+ Datum cvalue = toast_compress_datum(value, InvalidOid, NULL);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f81fc8ea66..616f31ba47 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -105,7 +105,7 @@ index_form_tuple(TupleDesc tupleDescriptor,
att->attstorage == TYPSTORAGE_MAIN))
{
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, NULL);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..3185082034 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,45 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d69dd907c9..de5d0f674e 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -62,10 +62,11 @@ toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
CompressionId cmid;
CompressionRoutine *cmroutine;
@@ -80,10 +81,18 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = GetCompressionRoutine(cmoid);
cmid = GetCompressionId(cmoid);
+ if (cmroutine->cminitstate)
+ options = cmroutine->cminitstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value,
IsCustomCompression(cmid) ?
- TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ,
+ options);
+
+ if (options != NULL)
+ pfree(options);
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index 52c167a35d..533bda5239 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -16,12 +16,70 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
#ifdef HAVE_LIBLZ4
#include "lz4.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -29,11 +87,12 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -41,9 +100,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -120,6 +179,8 @@ lz4handler(PG_FUNCTION_ARGS)
#else
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = lz4_cmcheck;
+ routine->cminitstate = lz4_cminitstate;
routine->cmcompress = lz4_cmcompress;
routine->cmdecompress = lz4_cmdecompress;
routine->cmdecompress_slice = lz4_cmdecompress_slice;
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 931394f779..607849a73d 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -15,11 +15,92 @@
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -122,6 +205,8 @@ pglzhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
routine->cmcompress = pglz_cmcompress;
routine->cmdecompress = pglz_cmdecompress;
routine->cmdecompress_slice = pglz_cmdecompress_slice;
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..1f842abfaf 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,12 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -43,6 +44,7 @@ toast_tuple_init(ToastTupleContext *ttc)
TupleDesc tupleDesc = ttc->ttc_rel->rd_att;
int numAttrs = tupleDesc->natts;
int i;
+ Datum attcmoptions;
ttc->ttc_flags = 0;
@@ -51,10 +53,28 @@ toast_tuple_init(ToastTupleContext *ttc)
Form_pg_attribute att = TupleDescAttr(tupleDesc, i);
struct varlena *old_value;
struct varlena *new_value;
+ HeapTuple attr_tuple;
+ bool isNull;
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ ttc->ttc_attr[i].tai_cmoptions = NULL;
+ else
+ ttc->ttc_attr[i].tai_cmoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +250,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index a2e2763a7c..490de64185 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1150,6 +1158,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1407,7 +1416,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 200c2d24c7..3aa4630263 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -507,7 +507,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, NULL, indstate);
CatalogCloseIndexes(indstate);
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index f1850436bd..80fcb1c313 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -255,6 +255,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 04d12a7ece..40e93bebd7 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -675,6 +675,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index ce6f7f83f5..c889a71eb4 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -229,7 +229,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
ListCell *cell;
@@ -248,6 +248,22 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionRoutine *routine = GetCompressionRoutine(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(compression->options);
+
+ pfree(routine);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e82b362bad..200fd90271 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -611,6 +611,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -815,6 +816,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -868,8 +871,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -923,8 +927,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -6152,6 +6159,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6314,7 +6322,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression, NULL);
+ colDef->compression, &acoptions, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6324,7 +6332,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -15078,9 +15086,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
ListCell *lc;
@@ -15114,7 +15124,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression,
+ &acoptions, &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15123,7 +15134,18 @@ ATExecSetCompression(AlteredTableInfo *tab,
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ /* update existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 707c6ea90b..9ea3707005 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2956,6 +2956,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 0c17cec725..861f6eea78 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2615,6 +2615,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 22757dfea9..fd748e9525 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2858,6 +2858,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d21cb7c94c..8b5c4ab5aa 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3461,11 +3462,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3473,14 +3480,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 4d4326ff57..fb8e44754e 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_compression_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -42,7 +43,12 @@ typedef enum CompressionId
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size,
+ void *options);
+typedef struct varlena *(*cmdecompress_function)(const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)(
const struct varlena *value,
@@ -61,11 +67,17 @@ struct CompressionRoutine
/* name of the compression method */
char cmname[64];
+ /* compression option check, can be NULL */
+ cmcheck_function cmcheck;
+
+ /* compression option intialize, can be NULL */
+ cminitstate_function cminitstate;
+
/* compression routine for the compression method */
cmcompress_function cmcompress;
/* decompression routine for the compression method */
- cmcompress_function cmdecompress;
+ cmdecompress_function cmdecompress;
/* slice decompression routine for the compression method */
cmdecompress_slice_function cmdecompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a88e3daa6a..9071e80714 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -33,6 +33,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 48ca172eae..53becd2057 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -67,7 +67,7 @@ do { \
#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
(((toast_compress_header_custom *) (ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index f94defff3c..bd192f457e 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index c506d31029..7e4e211153 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -171,6 +171,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8952b2b70d..fcbdbfa401 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -137,6 +137,7 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,7 +151,7 @@ extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
+ Datum *acoptions, bool *need_rewrite);
extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 89478ddea3..74ec86f313 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -636,6 +636,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 897a61689f..c9f324ed4c 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -121,4 +121,18 @@ SELECT length(f1) FROM cmmove2;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz2 | |
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 9029f76147..81be500a2e 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -98,6 +98,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -108,5 +109,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index a0a0aa13f0..9135ce5e5d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -58,4 +58,12 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
SELECT length(f1) FROM cmmove2;
\d+ cmmove2
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Sun, Nov 8, 2020 at 4:29 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 29, 2020 at 12:07 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Oct 29, 2020 at 12:31 AM Tomas Vondra
<tomas.vondra@2ndquadrant.com> wrote:On Wed, Oct 28, 2020 at 01:16:31PM +0530, Dilip Kumar wrote:
...
I have added the next patch to support the compression options. I am
storing the compression options only for the latest compression
method. Basically, based on this design we would be able to support
the options which are used only for compressions. As of now, the
compression option infrastructure is added and the compression options
for inbuilt method pglz and the external method zlib are added. Next,
I will work on adding the options for the lz4 method.In the attached patch set I have also included the compression option
support for lz4. As of now, I have only supported the acceleration
for LZ4_compress_fast. There is also support for the dictionary-based
compression but if we try to support that then we will need the
dictionary for decompression also. Since we are only keeping the
options for the current compression methods, we can not support
dictionary-based options as of now.OK, thanks. Do you have any other plans to improve this patch series? I
plan to do some testing and review, but if you're likely to post another
version soon then I'd wait a bit.There was some issue in create_compression_method.sgml and the
drop_compression_method.sgml was missing. I have fixed that in the
attached patch. Now I am not planning to change anything soon so you
can review. Thanks.The patches were not applying on the current head so I have re-based them.
There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v14-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v14-0005-new-compression-method-extension-for-zlib.patchDownload
From e6a62e3b70aca136b6a75ac53f8155d96a396c3f Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v14 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 45 ++++++++
contrib/cmzlib/sql/cmzlib.sql | 21 ++++
contrib/cmzlib/zlib.c | 158 +++++++++++++++++++++++++++++
8 files changed, 273 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
create mode 100644 contrib/cmzlib/zlib.c
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..d4281228a3
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ zlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..fcdc0e7980
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE COMPRESSION METHOD zlib HANDLER zlibhandler;
+COMMENT ON COMPRESSION METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..576d99bcfa
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,45 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..bdb59af4fc
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,21 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
new file mode 100644
index 0000000000..f24fc1c936
--- /dev/null
+++ b/contrib/cmzlib/zlib.c
@@ -0,0 +1,158 @@
+/*-------------------------------------------------------------------------
+ *
+ * zlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/zlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = zlib_cmcompress;
+ routine->cmdecompress = zlib_cmdecompress;
+ routine->cmdecompress_slice = NULL;
+
+ PG_RETURN_POINTER(routine);
+}
\ No newline at end of file
--
2.23.0
v14-0002-alter-table-set-compression.patchapplication/octet-stream; name=v14-0002-alter-table-set-compression.patchDownload
From be261fa03209ba42a87552c1d7a1e8625797e9d9 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 16:34:21 +0530
Subject: [PATCH v14 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
---
doc/src/sgml/ref/alter_table.sgml | 13 +++
src/backend/commands/tablecmds.c | 131 ++++++++++++++++++++++++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 9 ++
src/include/commands/event_trigger.h | 2 +-
src/include/executor/executor.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/create_cm.out | 15 +++
src/test/regress/sql/create_cm.sql | 7 ++
9 files changed, 180 insertions(+), 3 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..5c88c979af 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be
+ set from available built-in compression methods.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8c167cd098..366ffb792e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3863,6 +3865,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4390,6 +4393,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
+ case AT_SetCompression:
ATSimplePermissions(rel, ATT_TABLE);
/* These commands never recurse */
/* No command-specific prep needed */
@@ -4798,6 +4802,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5422,6 +5430,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -15004,6 +15015,126 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple atttuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+ ListCell *lc;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(atttuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ /* Prevent from altering untoastable attributes with PLAIN storage */
+ if (atttableform->attstorage == 'p' && !TypeIsToastable(atttableform->atttypid))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompressionMethod(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(atttuple);
+
+ /* Apply the change to indexes as well */
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ atttuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(atttuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
+ atttableform->attcompression = cmoid;
+
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(atttuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 664e6d983c..0fb5a1bbe5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1927,7 +1927,7 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
* of the compressed value is not supported in the target attribute the
* decompress the value.
*/
-static void
+void
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleDesc targetTupDesc)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 11fa23cc29..8acf53d2ed 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2265,6 +2265,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..a70e68f155 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot, TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7818bd718f..f07825edf1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1857,7 +1857,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 418c7de06c..3de5fab8c9 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -79,4 +79,19 @@ SELECT length(f1) FROM lz4test;
24096
(2 rows)
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 2ac2da8da8..8b75a0640d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -40,4 +40,11 @@ INSERT INTO lz4test VALUES(repeat('1234567890',1004));
INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
SELECT length(f1) FROM lz4test;
+-- alter compression method with rewrite
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmmove2
+
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v14-0001-Built-in-compression-method.patchapplication/octet-stream; name=v14-0001-Built-in-compression-method.patchDownload
From 5af64a9db1126cd3c4840a73cfce704189b3278e Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v14 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
---
configure | 80 ++++++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ref/create_table.sgml | 13 +
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 41 ++-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 70 +++--
src/backend/access/common/tupdesc.c | 4 +
src/backend/access/compression/Makefile | 17 ++
.../access/compression/compressionapi.c | 134 +++++++++
src/backend/access/compression/lz4.c | 127 +++++++++
src/backend/access/compression/pglz.c | 130 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 2 +
src/backend/catalog/Makefile | 4 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/commands/tablecmds.c | 103 ++++++-
src/backend/executor/nodeModifyTable.c | 84 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/cache/syscache.c | 23 ++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 39 ++-
src/bin/psql/tab-complete.c | 5 +-
src/include/access/compressionapi.h | 73 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 31 ++-
src/include/catalog/pg_attribute.h | 5 +-
src/include/catalog/pg_compression.dat | 20 ++
src/include/catalog/pg_compression.h | 48 ++++
src/include/catalog/pg_proc.dat | 16 ++
src/include/catalog/pg_type.dat | 5 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/include/utils/syscache.h | 2 +
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_cm.out | 82 ++++++
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 78 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/misc_sanity.out | 1 +
src/test/regress/expected/psql.out | 40 +--
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/sanity_check.out | 1 +
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 1 +
src/test/regress/sql/create_cm.sql | 43 +++
70 files changed, 1764 insertions(+), 584 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compressionapi.c
create mode 100644 src/backend/access/compression/lz4.c
create mode 100644 src/backend/access/compression/pglz.c
create mode 100644 src/include/access/compressionapi.h
create mode 100644 src/include/catalog/pg_compression.dat
create mode 100644 src/include/catalog/pg_compression.h
create mode 100644 src/test/regress/expected/create_cm.out
create mode 100644 src/test/regress/sql/create_cm.sql
diff --git a/configure b/configure
index ace4ed5dec..d6a8b35632 100755
--- a/configure
+++ b/configure
@@ -700,6 +700,7 @@ LD
LDFLAGS_SL
LDFLAGS_EX
with_zlib
+with_lz4
with_system_tzdata
with_libxslt
XML2_LIBS
@@ -867,6 +868,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1571,6 +1573,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8609,7 +8612,31 @@ CPPFLAGS="$CPPFLAGS $INCLUDES"
LDFLAGS="$LDFLAGS $LIBDIRS"
+#
+# Lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+fi
# Check whether --with-gnu-ld was given.
@@ -12092,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for inflate in -lz" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index bc59a2d77d..c559d1ffd5 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -967,6 +968,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. The
+ default compression method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 6774f597a4..cda7b061d2 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..9fa4e2da1d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -445,21 +446,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- return result;
+ return cmroutine->cmdecompress(attr);
}
@@ -473,22 +470,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ CompressionRoutine *cmroutine;
+ Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->cmdecompress_slice)
+ return cmroutine->cmdecompress_slice(attr, slicelength);
+ else
+ return cmroutine->cmdecompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..f81fc8ea66 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..7c603dd19a 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -30,6 +30,20 @@
static bool toastrel_valueid_exists(Relation toastrel, Oid valueid);
static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
+/* ---------
+ * toast_set_compressed_datum_info
+ *
+ * Save metadata in compressed datum. This information will required
+ * for decompression.
+ * --------
+ */
+void
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+{
+ TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
+ TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+}
+
/* ----------
* toast_compress_datum -
*
@@ -44,46 +58,44 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Get the compression routines for the compression method */
+ cmroutine = GetCompressionRoutine(cmoid);
+
+ /* Call the actual compression function */
+ tmp = cmroutine->cmcompress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
+ valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9abfe0971b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -663,6 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attalign = typeForm->typalign;
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ att->attcompression = InvalidOid;
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..981f708f66
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = pglz.o lz4.o compressionapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
new file mode 100644
index 0000000000..ee57261213
--- /dev/null
+++ b/src/backend/access/compression/compressionapi.c
@@ -0,0 +1,134 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressionapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressionapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "catalog/pg_compression.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * GetCompressionOid - given an compression method name, look up its OID.
+ */
+Oid
+GetCompressionOid(const char *cmname)
+{
+ HeapTuple tup;
+ Oid oid = InvalidOid;
+
+ tup = SearchSysCache1(CMNAME, CStringGetDatum(cmname));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ oid = cmform->oid;
+ ReleaseSysCache(tup);
+ }
+
+ return oid;
+}
+
+/*
+ * GetCompressionNameFromOid - given an compression method oid, look up its
+ * name.
+ */
+char *
+GetCompressionNameFromOid(Oid cmoid)
+{
+ HeapTuple tup;
+ char *cmname = NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ cmname = pstrdup(NameStr(cmform->cmname));
+ ReleaseSysCache(tup);
+ }
+
+ return cmname;
+}
+
+/*
+ * GetCompressionRoutine - given an compression method oid, look up
+ * its handler routines.
+ */
+CompressionRoutine *
+GetCompressionRoutine(Oid cmoid)
+{
+ Datum datum;
+ HeapTuple tup;
+ CompressionRoutine *routine = NULL;
+
+ if (!OidIsValid(cmoid))
+ return NULL;
+
+ tup = SearchSysCache1(CMOID, ObjectIdGetDatum(cmoid));
+ if (HeapTupleIsValid(tup))
+ {
+ Form_pg_compression cmform = (Form_pg_compression) GETSTRUCT(tup);
+
+ datum = OidFunctionCall0(cmform->cmhandler);
+ routine = (CompressionRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionRoutine))
+ elog(ERROR, "compression method handler function %u "
+ "did not return a CompressionRoutine struct",
+ cmform->cmhandler);
+ ReleaseSysCache(tup);
+ }
+
+ return routine;
+}
+
+/*
+ * GetCompressionMethodFromCompressionId - Get the compression method oid from
+ * the compression id.
+ */
+Oid
+GetCompressionOidFromCompressionId(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionId - Get the compression id from compression oid
+ */
+CompressionId
+GetCompressionId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method %d", cmoid);
+ }
+}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
new file mode 100644
index 0000000000..edc8f5ed22
--- /dev/null
+++ b/src/backend/access/compression/lz4.c
@@ -0,0 +1,127 @@
+/*-------------------------------------------------------------------------
+ *
+ * cm_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * contrib/cm_lz4/cm_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+#endif
+
+/* lz4 is the default compression method */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = lz4_cmcompress;
+ routine->cmdecompress = lz4_cmdecompress;
+ routine->cmdecompress_slice = lz4_cmdecompress_slice;
+ PG_RETURN_POINTER(routine);
+#endif
+}
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
new file mode 100644
index 0000000000..61dfe42004
--- /dev/null
+++ b/src/backend/access/compression/pglz.c
@@ -0,0 +1,130 @@
+/*-------------------------------------------------------------------------
+ *
+ * pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* pglz is the default compression method */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ CompressionRoutine *routine = makeNode(CompressionRoutine);
+
+ routine->cmcompress = pglz_cmcompress;
+ routine->cmdecompress = pglz_cmdecompress;
+ routine->cmdecompress_slice = pglz_cmdecompress_slice;
+
+ PG_RETURN_POINTER(routine);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index a7ed93fdc1..2bcdf69686 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/Makefile b/src/backend/catalog/Makefile
index 2519771210..5efb70bdea 100644
--- a/src/backend/catalog/Makefile
+++ b/src/backend/catalog/Makefile
@@ -54,7 +54,7 @@ include $(top_srcdir)/src/backend/common.mk
CATALOG_HEADERS := \
pg_proc.h pg_type.h pg_attribute.h pg_class.h \
pg_attrdef.h pg_constraint.h pg_inherits.h pg_index.h pg_operator.h \
- pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h \
+ pg_opfamily.h pg_opclass.h pg_am.h pg_amop.h pg_amproc.h pg_compression.h \
pg_language.h pg_largeobject_metadata.h pg_largeobject.h pg_aggregate.h \
pg_statistic_ext.h pg_statistic_ext_data.h \
pg_statistic.h pg_rewrite.h pg_trigger.h pg_event_trigger.h pg_description.h \
@@ -76,7 +76,7 @@ POSTGRES_BKI_SRCS := $(addprefix $(top_srcdir)/src/include/catalog/, $(CATALOG_H
# The .dat files we need can just be listed alphabetically.
POSTGRES_BKI_DATA = $(addprefix $(top_srcdir)/src/include/catalog/,\
- pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_authid.dat \
+ pg_aggregate.dat pg_am.dat pg_amop.dat pg_amproc.dat pg_compression.dat pg_authid.dat \
pg_cast.dat pg_class.dat pg_collation.dat pg_conversion.dat \
pg_database.dat pg_language.dat \
pg_namespace.dat pg_opclass.dat pg_operator.dat pg_opfamily.dat \
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4cd7d76938..a2e2763a7c 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b88b4a1f12..200c2d24c7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e3cfaf8b07..8c167cd098 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -560,7 +561,8 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
+ char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -854,6 +856,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+ GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression =
+ GetCompressionNameFromOid(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2706,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6234,6 +6278,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompressionMethod(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7673,6 +7729,14 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
errmsg("column data type %s can only have storage PLAIN",
format_type_be(attrtuple->atttypid))));
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;
+
CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -11750,6 +11814,19 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /* InvalidOid for the plain storage otherwise default compression id */
+ if (attTup->attstorage == TYPSTORAGE_PLAIN)
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17628,3 +17705,27 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method for the attribute from compression name.
+ */
+static Oid
+GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
+{
+ Oid cmoid;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression)));
+ return cmoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 29e07b7228..664e6d983c 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressionapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1918,6 +1921,79 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Comppare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute the
+ * decompress the value.
+ */
+static void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ /* FIXME : Is there a better way to handle this ?*/
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2123,6 +2199,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3031c52991..279757e63e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2929,6 +2929,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 9aa853748d..635b0ea10c 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2581,6 +2581,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4504b1503b..1040311bf3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2833,6 +2833,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 95e256883b..11fa23cc29 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,6 +601,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -636,9 +638,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3378,11 +3380,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3391,8 +3394,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3437,6 +3440,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3666,6 +3677,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15185,6 +15197,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15703,6 +15716,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 254c0f65c2..7482461dbe 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressionapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1066,6 +1067,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index c1bd68011c..748cd540ba 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..a1fa8ccefc 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -351,3 +351,4 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
PSEUDOTYPE_DUMMY_IO_FUNCS(anynonarray);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatible);
PSEUDOTYPE_DUMMY_IO_FUNCS(anycompatiblenonarray);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_handler);
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index 809b27a038..b46d876965 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -30,6 +30,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -287,6 +288,28 @@ static const struct cachedesc cacheinfo[] = {
},
8
},
+ {CompressionRelationId, /* CMNAME */
+ CompressionNameIndexId,
+ 1,
+ {
+ Anum_pg_compression_cmname,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
+ {CompressionRelationId, /* CMOID */
+ CompressionIndexId,
+ 1,
+ {
+ Anum_pg_compression_oid,
+ 0,
+ 0,
+ 0
+ },
+ 4
+ },
{CollationRelationId, /* COLLNAMEENCNSP */
CollationNameEncNspIndexId,
3,
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3c276f2bcb..01470f5eb1 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -385,6 +385,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8551,6 +8552,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8636,6 +8638,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "c.cmname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8654,7 +8665,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_compression c "
+ "ON a.attcompression = c.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8681,6 +8697,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8709,6 +8726,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15739,6 +15757,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15763,6 +15782,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15798,6 +15820,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 317bb83970..2912e4c0dc 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 07d640021c..9bc27a86ce 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,19 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT c.cmname "
+ " FROM pg_catalog.pg_compression c "
+ " WHERE c.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2032,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2093,6 +2109,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5238a960f7..afa1aac594 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,11 +2093,14 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("WITH (", "PRESERVE (");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
new file mode 100644
index 0000000000..6660285e84
--- /dev/null
+++ b/src/include/access/compressionapi.h
@@ -0,0 +1,73 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressionapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressionapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSIONAPI_H
+#define COMPRESSIONAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_compression_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid. And,
+ * using that oid we can get the compression handler routine by fetching the
+ * pg_compression catalog row.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+
+typedef struct CompressionRoutine CompressionRoutine;
+
+/* compresion handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression.
+ *
+ * 'cmcompress' and 'cmdecompress' - varlena compression functions.
+ */
+struct CompressionRoutine
+{
+ NodeTag type;
+
+ /* name of the compression method */
+ char cmname[64];
+
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;
+
+ /* slice decompression routine for the compression method */
+ cmdecompress_slice_function cmdecompress_slice;
+};
+
+/* access/compression/compresssionapi.c */
+extern Oid GetCompressionOid(const char *compression);
+extern char *GetCompressionNameFromOid(Oid cmoid);
+extern CompressionRoutine *GetCompressionRoutine(Oid cmoid);
+extern Oid GetCompressionOidFromCompressionId(CompressionId cmid);
+extern CompressionId GetCompressionId(Oid cmoid);
+
+#endif /* COMPRESSIONAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..a88e3daa6a 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..8ef477cfe0 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressionapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,36 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)
-extern Datum toast_compress_datum(Datum value);
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
@@ -52,4 +67,12 @@ extern void toast_close_indexes(Relation *toastidxs, int num_indexes,
LOCKMODE lock);
extern void init_toast_snapshot(Snapshot toast_snapshot);
+/*
+ * toast_set_compressed_datum_info -
+ *
+ * Save metadata in compressed datum
+ */
+extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ int32 rawsize);
+
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index cdf75a2380..c506d31029 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /* attribute's compression Oid */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +186,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_compression.dat b/src/include/catalog/pg_compression.dat
new file mode 100644
index 0000000000..ef41f15dfd
--- /dev/null
+++ b/src/include/catalog/pg_compression.dat
@@ -0,0 +1,20 @@
+#----------------------------------------------------------------------
+#
+# pg_compression.dat
+# Initial contents of the pg_compression system relation.
+#
+# Predefined builtin compression methods.
+#
+# Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group
+# Portions Copyright (c) 1994, Regents of the University of California
+#
+# src/include/catalog/pg_compression.dat
+#
+#----------------------------------------------------------------------
+
+[
+
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_OID', cmname => 'pglz', cmhandler => 'pglzhandler'},
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_OID', cmname => 'lz4', cmhandler => 'lz4handler'},
+
+]
diff --git a/src/include/catalog/pg_compression.h b/src/include/catalog/pg_compression.h
new file mode 100644
index 0000000000..8b3c7a1cc1
--- /dev/null
+++ b/src/include/catalog/pg_compression.h
@@ -0,0 +1,48 @@
+/*-------------------------------------------------------------------------
+ *
+ * pg_compression.h
+ * definition of the "trigger" system catalog (pg_compression)
+ *
+ *
+ * Portions Copyright (c) 1996-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ * src/include/catalog/pg_compression.h
+ *
+ * NOTES
+ * The Catalog.pm module reads this file and derives schema
+ * information.
+ *
+ *-------------------------------------------------------------------------
+ */
+#ifndef PG_COMPRESSION_H
+#define PG_COMPRESSION_H
+
+#include "catalog/genbki.h"
+#include "catalog/pg_compression_d.h"
+
+/* ----------------
+ * pg_compression definition. cpp turns this into
+ * typedef struct FormData_pg_compression
+ * ----------------
+ */
+CATALOG(pg_compression,5555,CompressionRelationId)
+{
+ Oid oid; /* oid */
+ NameData cmname; /* compression method name */
+ regproc cmhandler BKI_LOOKUP(pg_proc); /* handler function */
+} FormData_pg_compression;
+
+/* ----------------
+ * Form_pg_compression corresponds to a pointer to a tuple with
+ * the format of pg_compression relation.
+ * ----------------
+ */
+typedef FormData_pg_compression *Form_pg_compression;
+
+DECLARE_UNIQUE_INDEX(pg_compression_index, 2137, on pg_compression using btree(oid oid_ops));
+#define CompressionIndexId 2137
+DECLARE_UNIQUE_INDEX(pg_compression_cmnam_index, 2121, on pg_compression using btree(cmname name_ops));
+#define CompressionNameIndexId 2121
+
+#endif /* PG_COMPRESSION_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c01da4bf01..d2344758f2 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7219,6 +7228,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_handler_in', proisstrict => 'f',
+ prorettype => 'compression_handler', proargtypes => 'cstring',
+ prosrc => 'compression_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_handler', prosrc => 'compression_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 21a467a7a7..fb6cbaa575 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -578,6 +578,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_handler_in',
+ typoutput => 'compression_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 7ddd8c011b..5a1ca9fada 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -492,6 +492,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 7ef9b0eac0..7818bd718f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -648,6 +648,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -686,6 +687,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index fb270df678..e5f1b49938 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/include/utils/syscache.h b/src/include/utils/syscache.h
index f27b73d76d..04a08be856 100644
--- a/src/include/utils/syscache.h
+++ b/src/include/utils/syscache.h
@@ -46,6 +46,8 @@ enum SysCacheIdentifier
CASTSOURCETARGET,
CLAAMNAMENSP,
CLAOID,
+ CMNAME,
+ CMOID,
COLLNAMEENCNSP,
COLLOID,
CONDEFAULT,
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
new file mode 100644
index 0000000000..418c7de06c
--- /dev/null
+++ b/src/test/regress/expected/create_cm.out
@@ -0,0 +1,82 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | external | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | main | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ st1 | text | | | | plain | pglz | |
+ st2 | integer | | | | plain | | |
+
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+ Table "public.cmstoragetest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ st1 | text | | | | extended | pglz | |
+ st2 | integer | | | | plain | | |
+
+DROP TABLE cmstoragetest;
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+-- drop original compression information
+DROP TABLE cmdata;
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10010
+(2 rows)
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 912c73d351..bf19c3f091 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -309,32 +309,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -348,12 +348,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -364,12 +364,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -386,11 +386,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -424,11 +424,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 2a033a6e11..9e365264c0 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -272,10 +272,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -409,10 +409,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index 2238f896f9..d4aa824857 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -442,14 +442,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..9029f76147 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -73,6 +73,7 @@ loop
end if;
end loop;
end$$;
+NOTICE: pg_compression contains unpinned initdb-created object(s)
NOTICE: pg_constraint contains unpinned initdb-created object(s)
NOTICE: pg_database contains unpinned initdb-created object(s)
NOTICE: pg_extension contains unpinned initdb-created object(s)
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index daac0ff49d..8c320fdb16 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2809,34 +2809,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 097ff5d111..917cadd4ef 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3016,11 +3016,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3036,11 +3036,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3067,11 +3067,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/sanity_check.out b/src/test/regress/expected/sanity_check.out
index 192445878d..5f99db5acf 100644
--- a/src/test/regress/expected/sanity_check.out
+++ b/src/test/regress/expected/sanity_check.out
@@ -111,6 +111,7 @@ pg_authid|t
pg_cast|t
pg_class|t
pg_collation|t
+pg_compression|t
pg_constraint|t
pg_conversion|t
pg_database|t
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 4c3edd213f..afe2878b00 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..a9f475b069 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -36,6 +36,7 @@ test: create_function_1
test: create_type
test: create_table
test: create_function_2
+test: create_cm
# ----------
# Load huge amounts of data
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
new file mode 100644
index 0000000000..2ac2da8da8
--- /dev/null
+++ b/src/test/regress/sql/create_cm.sql
@@ -0,0 +1,43 @@
+-- test storages
+CREATE TABLE cmstoragetest(st1 TEXT, st2 INT);
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTERNAL;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE MAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE PLAIN;
+\d+ cmstoragetest
+ALTER TABLE cmstoragetest ALTER COLUMN st1 SET STORAGE EXTENDED;
+\d+ cmstoragetest
+DROP TABLE cmstoragetest;
+
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+INSERT INTO cmdata VALUES(repeat('1234567890',1001));
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+
+-- we update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+
+-- drop original compression information
+DROP TABLE cmdata;
+
+-- check data is okdd
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+-- lz4 compression
+CREATE TABLE lz4test(f1 TEXT COMPRESSION lz4);
+INSERT INTO lz4test VALUES(repeat('1234567890',1004));
+INSERT INTO lz4test VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM lz4test;
+
+DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v14-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v14-0003-Add-support-for-PRESERVE.patchDownload
From e05fb8a5c155d4e934cd9923a55a21941aedbcca Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:18:34 +0530
Subject: [PATCH v14 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/dependency.c | 8 +-
src/backend/catalog/objectaddress.c | 24 +++
src/backend/commands/Makefile | 1 +
src/backend/commands/alter.c | 1 +
src/backend/commands/compressioncmds.c | 222 +++++++++++++++++++++
src/backend/commands/event_trigger.c | 1 +
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 7 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/catalog/dependency.h | 5 +-
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/create_cm.out | 9 +
src/test/regress/expected/create_index.out | 6 +-
src/test/regress/sql/create_cm.sql | 4 +
21 files changed, 470 insertions(+), 70 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 5c88c979af..733ee0cf20 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,12 +386,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods.
+ set from available built-in compression methods. The PRESERVE list
+ contains list of compression methods used on the column and determines
+ which of them should be kept on the column. Without PRESERVE or if all
+ the previous compression methods are not preserved then the table will
+ be rewritten. If PRESERVE ALL is specified then all the previous methods
+ will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index b0d037600e..076ef56ba5 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -29,6 +29,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -181,7 +182,8 @@ static const Oid object_classes[] = {
PublicationRelationId, /* OCLASS_PUBLICATION */
PublicationRelRelationId, /* OCLASS_PUBLICATION_REL */
SubscriptionRelationId, /* OCLASS_SUBSCRIPTION */
- TransformRelationId /* OCLASS_TRANSFORM */
+ TransformRelationId, /* OCLASS_TRANSFORM */
+ CompressionRelationId /* OCLASS_COMPRESSION */
};
@@ -1585,6 +1587,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
+ case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
@@ -2977,6 +2980,9 @@ getObjectClass(const ObjectAddress *object)
case TransformRelationId:
return OCLASS_TRANSFORM;
+
+ case CompressionRelationId:
+ return OCLASS_COMPRESSION;
}
/* shouldn't get here */
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 3662a8ebb6..8f95da758f 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressionapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
@@ -29,6 +30,7 @@
#include "catalog/pg_authid.h"
#include "catalog/pg_cast.h"
#include "catalog/pg_collation.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_constraint.h"
#include "catalog/pg_conversion.h"
#include "catalog/pg_database.h"
@@ -3906,6 +3908,15 @@ getObjectDescription(const ObjectAddress *object, bool missing_ok)
break;
}
+ case OCLASS_COMPRESSION:
+ {
+ char *cmname = GetCompressionNameFromOid(object->objectId);
+
+ if (cmname)
+ appendStringInfo(&buffer, _("compression %s"), cmname);
+ break;
+ }
+
case OCLASS_TRANSFORM:
{
HeapTuple trfTup;
@@ -4478,6 +4489,10 @@ getObjectTypeDescription(const ObjectAddress *object, bool missing_ok)
appendStringInfoString(&buffer, "transform");
break;
+ case OCLASS_COMPRESSION:
+ appendStringInfoString(&buffer, "compression");
+ break;
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
@@ -5762,6 +5777,15 @@ getObjectIdentityParts(const ObjectAddress *object,
}
break;
+ case OCLASS_COMPRESSION:
+ {
+ appendStringInfo(&buffer, "%u",
+ object->objectId);
+ if (objname)
+ *objname = list_make1(psprintf("%u", object->objectId));
+ break;
+ }
+
/*
* There's intentionally no default: case here; we want the
* compiler to warn if a new OCLASS hasn't been handled above.
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index d4815d3ce6..770df27b90 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/alter.c b/src/backend/commands/alter.c
index b11ebf0f61..8192429360 100644
--- a/src/backend/commands/alter.c
+++ b/src/backend/commands/alter.c
@@ -663,6 +663,7 @@ AlterObjectNamespace_oid(Oid classId, Oid objid, Oid nspOid,
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
/* ignore object types that don't have schema-qualified names */
break;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a1be6ef7be
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,222 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressionapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/fmgroids.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == CompressionRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ ListCell *cell;
+
+ /* no compression for the plain storage */
+ if (att->attstorage == TYPSTORAGE_PLAIN)
+ return InvalidOid;
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = GetCompressionOid(compression->cmname);
+ if (!OidIsValid(cmoid))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", compression->cmname)));
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = GetCompressionOid(cmname_p);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = GetCompressionNameFromOid(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 3ffba4e63e..172242ae37 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -1053,6 +1053,7 @@ EventTriggerSupportsObjectClass(ObjectClass objclass)
case OCLASS_PUBLICATION_REL:
case OCLASS_SUBSCRIPTION:
case OCLASS_TRANSFORM:
+ case OCLASS_COMPRESSION:
return true;
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 366ffb792e..e82b362bad 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -563,8 +566,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompressionMethod(Form_pg_attribute att,
- char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -589,6 +590,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -867,7 +869,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
attr->attcompression =
- GetAttributeCompressionMethod(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -937,6 +939,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2415,17 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression =
- GetCompressionNameFromOid(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2462,7 +2478,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2713,12 +2729,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4803,7 +4819,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6296,8 +6313,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
- attribute.attcompression = GetAttributeCompressionMethod(&attribute,
- colDef->compression);
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6472,6 +6489,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6619,6 +6637,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, CompressionRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordDependencyOn(&attref, &acref, DEPENDENCY_NORMAL);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11623,7 +11663,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
getObjectDescription(&foundObject, false),
colName)));
break;
+ case OCLASS_COMPRESSION:
+ /* Just check that dependency is the right type */
+ Assert(foundDep->deptype == DEPENDENCY_NORMAL);
+ break;
case OCLASS_DEFAULT:
/*
@@ -11734,7 +11778,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != CompressionRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11846,6 +11891,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15019,24 +15069,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple atttuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
ListCell *lc;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
atttuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15067,9 +15114,12 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompressionMethod(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
@@ -17836,27 +17886,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method for the attribute from compression name.
- */
-static Oid
-GetAttributeCompressionMethod(Form_pg_attribute att, char *compression)
-{
- Oid cmoid;
-
- /* no compression for the plain storage */
- if (att->attstorage == TYPSTORAGE_PLAIN)
- return InvalidOid;
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- cmoid = GetCompressionOid(compression);
- if (!OidIsValid(cmoid))
- ereport(ERROR,
- (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
- errmsg("compression type \"%s\" not recognized", compression)));
- return cmoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0fb5a1bbe5..5257aeb66b 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1936,6 +1937,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1968,8 +1970,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = GetCompressionOidFromCompressionId(
+ TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 279757e63e..5413bd92a4 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2929,7 +2929,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2949,6 +2949,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5618,6 +5630,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 635b0ea10c..09882c9375 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2581,7 +2581,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2601,6 +2601,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3673,6 +3683,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 1040311bf3..22757dfea9 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2833,7 +2833,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2851,6 +2851,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4199,6 +4209,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8acf53d2ed..deb82ea660 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -601,7 +601,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2266,12 +2268,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3394,7 +3396,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3449,13 +3451,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 7482461dbe..5ff4b31703 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1070,7 +1070,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = GetCompressionNameFromOid(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/include/catalog/dependency.h b/src/include/catalog/dependency.h
index 901d5019cd..fcc834e371 100644
--- a/src/include/catalog/dependency.h
+++ b/src/include/catalog/dependency.h
@@ -125,10 +125,11 @@ typedef enum ObjectClass
OCLASS_PUBLICATION, /* pg_publication */
OCLASS_PUBLICATION_REL, /* pg_publication_rel */
OCLASS_SUBSCRIPTION, /* pg_subscription */
- OCLASS_TRANSFORM /* pg_transform */
+ OCLASS_TRANSFORM, /* pg_transform */
+ OCLASS_COMPRESSION /* pg_compression */
} ObjectClass;
-#define LAST_OCLASS OCLASS_TRANSFORM
+#define LAST_OCLASS OCLASS_COMPRESSION
/* flag bits for performDeletion/performMultipleDeletions: */
#define PERFORM_DELETION_INTERNAL 0x0001 /* internal action */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 7a079ef07f..936b072c76 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -145,6 +145,13 @@ extern Oid get_table_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 5a1ca9fada..65ef987d67 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f07825edf1..e0a51f0145 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -625,6 +625,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -648,7 +662,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 3de5fab8c9..1b7b5bd6e5 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -94,4 +94,13 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 93a8736a3f..e006c04c7c 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2064,6 +2064,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
@@ -2074,7 +2075,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(10 rows)
+(11 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2093,6 +2094,7 @@ WHERE classid = 'pg_class'::regclass AND
ORDER BY 1, 2;
obj | objref | deptype
------------------------------------------+------------------------------------------------------------+---------
+ column c2 of table concur_reindex_tab | compression pglz | n
index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
index concur_reindex_ind2 | collation "default" | n
index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
@@ -2103,7 +2105,7 @@ WHERE classid = 'pg_class'::regclass AND
index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
materialized view concur_reindex_matview | schema public | n
table concur_reindex_tab | schema public | n
-(10 rows)
+(11 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 8b75a0640d..3549fb35f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -46,5 +46,9 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION lz4;
\d+ cmmove2
+-- preserve old compression method
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+\d+ cmmove2
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v14-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v14-0004-Create-custom-compression-methods.patchDownload
From 93fea204914175edbae05fc4cbdcb817cd788b70 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 20 Oct 2020 17:57:00 +0530
Subject: [PATCH v14 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
doc/src/sgml/ref/allfiles.sgml | 4 +-
.../sgml/ref/create_compression_method.sgml | 110 ++++++++++++++++++
doc/src/sgml/ref/drop_compression_method.sgml | 110 ++++++++++++++++++
doc/src/sgml/reference.sgml | 2 +
src/backend/access/common/detoast.c | 53 ++++++++-
src/backend/access/common/toast_internals.c | 15 ++-
.../access/compression/compressionapi.c | 2 +-
src/backend/access/compression/lz4.c | 23 ++--
src/backend/access/compression/pglz.c | 20 ++--
src/backend/catalog/aclchk.c | 2 +
src/backend/catalog/dependency.c | 2 +-
src/backend/catalog/objectaddress.c | 22 ++++
src/backend/commands/compressioncmds.c | 104 +++++++++++++++++
src/backend/commands/event_trigger.c | 3 +
src/backend/commands/seclabel.c | 1 +
src/backend/executor/nodeModifyTable.c | 3 +-
src/backend/nodes/copyfuncs.c | 14 +++
src/backend/nodes/equalfuncs.c | 12 ++
src/backend/parser/gram.y | 20 +++-
src/backend/tcop/utility.c | 16 +++
src/include/access/compressionapi.h | 24 ++--
src/include/access/detoast.h | 8 ++
src/include/access/toast_internals.h | 19 ++-
src/include/commands/defrem.h | 2 +
src/include/nodes/nodes.h | 3 +-
src/include/nodes/parsenodes.h | 12 ++
src/include/postgres.h | 8 ++
src/include/tcop/cmdtaglist.h | 2 +
src/test/regress/expected/create_cm.out | 18 +++
src/test/regress/sql/create_cm.sql | 7 ++
30 files changed, 594 insertions(+), 47 deletions(-)
create mode 100644 doc/src/sgml/ref/create_compression_method.sgml
create mode 100644 doc/src/sgml/ref/drop_compression_method.sgml
diff --git a/doc/src/sgml/ref/allfiles.sgml b/doc/src/sgml/ref/allfiles.sgml
index 0f0064150c..354ebb4b75 100644
--- a/doc/src/sgml/ref/allfiles.sgml
+++ b/doc/src/sgml/ref/allfiles.sgml
@@ -62,6 +62,7 @@ Complete list of usable sgml source files in this directory.
<!ENTITY createAggregate SYSTEM "create_aggregate.sgml">
<!ENTITY createCast SYSTEM "create_cast.sgml">
<!ENTITY createCollation SYSTEM "create_collation.sgml">
+<!ENTITY createCompressionMethod SYSTEM "create_compression_method.sgml">
<!ENTITY createConversion SYSTEM "create_conversion.sgml">
<!ENTITY createDatabase SYSTEM "create_database.sgml">
<!ENTITY createDomain SYSTEM "create_domain.sgml">
@@ -105,10 +106,11 @@ Complete list of usable sgml source files in this directory.
<!ENTITY delete SYSTEM "delete.sgml">
<!ENTITY discard SYSTEM "discard.sgml">
<!ENTITY do SYSTEM "do.sgml">
-<!ENTITY dropAccessMethod SYSTEM "drop_access_method.sgml">
+<!ENTITY dropCompressionMethod SYSTEM "drop_access_method.sgml">
<!ENTITY dropAggregate SYSTEM "drop_aggregate.sgml">
<!ENTITY dropCast SYSTEM "drop_cast.sgml">
<!ENTITY dropCollation SYSTEM "drop_collation.sgml">
+<!ENTITY dropAccessMethod SYSTEM "drop_compression_method.sgml">
<!ENTITY dropConversion SYSTEM "drop_conversion.sgml">
<!ENTITY dropDatabase SYSTEM "drop_database.sgml">
<!ENTITY dropDomain SYSTEM "drop_domain.sgml">
diff --git a/doc/src/sgml/ref/create_compression_method.sgml b/doc/src/sgml/ref/create_compression_method.sgml
new file mode 100644
index 0000000000..560a7f2942
--- /dev/null
+++ b/doc/src/sgml/ref/create_compression_method.sgml
@@ -0,0 +1,110 @@
+<!--
+doc/src/sgml/ref/create_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-create-compression-method">
+ <indexterm zone="sql-create-compression-method">
+ <primary>CREATE COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>CREATE COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>CREATE COMPRESSION METHOD</refname>
+ <refpurpose>define a new compresion method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+CREATE COMPRESSION METHOD <replaceable class="parameter">name</replaceable>
+ HANDLER <replaceable class="parameter">handler_function</replaceable>
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> creates a new compression method.
+ </para>
+
+ <para>
+ The compression method name must be unique within the database.
+ </para>
+
+ <para>
+ Only superusers can define new compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of the compression method to be created.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">handler_function</replaceable></term>
+ <listitem>
+ <para>
+ <replaceable class="parameter">handler_function</replaceable> is the
+ name (possibly schema-qualified) of a previously registered function
+ that represents the compression method. The handler function must be declared
+ to accept a single argument of type <type>internal</type> and to return
+ the pseudo-type <type>compression_handler</type>. The argument is a
+ dummy value that simply serves to prevent handler functions from being
+ called directly from SQL commands. The result of the function must be a
+ palloc'd struct of type <structname>CompressionRoutine</structname>,
+ which contains everything that the core code needs to know to make use of
+ the compression method.
+ The <structname>CompressionRoutine</structname> struct, also called the
+ compression method's <firstterm>API struct</firstterm>, contains pointers to
+ support functions for the compression method. These support functions are
+ plain C functions and are not visible or callable at the SQL level.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Create an compression method <literal>pglz2</literal> with
+ handler function <literal>pglzhandler</literal>:
+<programlisting>
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ <command>CREATE COMPRESSION METHOD</command> is a
+ <productname>PostgreSQL</productname> extension.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-drop-compression-method"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/ref/drop_compression_method.sgml b/doc/src/sgml/ref/drop_compression_method.sgml
new file mode 100644
index 0000000000..2a7399f6c2
--- /dev/null
+++ b/doc/src/sgml/ref/drop_compression_method.sgml
@@ -0,0 +1,110 @@
+<!--
+doc/src/sgml/ref/drop_compression_method.sgml
+PostgreSQL documentation
+-->
+
+<refentry id="sql-drop-compression-method">
+ <indexterm zone="sql-drop-compression-method">
+ <primary>DROP COMPRESSION METHOD</primary>
+ </indexterm>
+
+ <refmeta>
+ <refentrytitle>DROP COMPRESSION METHOD</refentrytitle>
+ <manvolnum>7</manvolnum>
+ <refmiscinfo>SQL - Language Statements</refmiscinfo>
+ </refmeta>
+
+ <refnamediv>
+ <refname>DROP COMPRESSION METHOD</refname>
+ <refpurpose>remove a compression method</refpurpose>
+ </refnamediv>
+
+ <refsynopsisdiv>
+<synopsis>
+DROP COMPRESSION METHOD [ IF EXISTS ] <replaceable class="parameter">name</replaceable> [ CASCADE | RESTRICT ]
+</synopsis>
+ </refsynopsisdiv>
+
+ <refsect1>
+ <title>Description</title>
+
+ <para>
+ <command>DROP COMPRESSION METHOD</command> removes an existing compression method.
+ Only superusers can drop compression methods.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>Parameters</title>
+
+ <variablelist>
+ <varlistentry>
+ <term><literal>IF EXISTS</literal></term>
+ <listitem>
+ <para>
+ Do not throw an error if the compression method does not exist.
+ A notice is issued in this case.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><replaceable class="parameter">name</replaceable></term>
+ <listitem>
+ <para>
+ The name of an existing compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>CASCADE</literal></term>
+ <listitem>
+ <para>
+ Automatically drop objects that depend on the compression method
+ and in turn all objects that depend on those objects
+ (see <xref linkend="ddl-depend"/>).
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
+ <term><literal>RESTRICT</literal></term>
+ <listitem>
+ <para>
+ Refuse to drop the compression method if any objects depend on it.
+ This is the default.
+ </para>
+ </listitem>
+ </varlistentry>
+ </variablelist>
+ </refsect1>
+
+ <refsect1>
+ <title>Examples</title>
+
+ <para>
+ Drop the compression method <literal>pglz2</literal>:
+<programlisting>
+DROP COMPRESSION METHOD pglz2;
+</programlisting></para>
+ </refsect1>
+
+ <refsect1>
+ <title>Compatibility</title>
+
+ <para>
+ <command>DROP COMPRESSION METHOD</command> is a
+ <productname>PostgreSQL</productname> extension.
+ </para>
+ </refsect1>
+
+ <refsect1>
+ <title>See Also</title>
+
+ <simplelist type="inline">
+ <member><xref linkend="sql-create-compression-method"/></member>
+ </simplelist>
+ </refsect1>
+
+</refentry>
diff --git a/doc/src/sgml/reference.sgml b/doc/src/sgml/reference.sgml
index dd2bddab8c..381f52a359 100644
--- a/doc/src/sgml/reference.sgml
+++ b/doc/src/sgml/reference.sgml
@@ -90,6 +90,7 @@
&createAggregate;
&createCast;
&createCollation;
+ &createCompressionMethod;
&createConversion;
&createDatabase;
&createDomain;
@@ -137,6 +138,7 @@
&dropAggregate;
&dropCast;
&dropCollation;
+ &dropCompressionMethod;
&dropConversion;
&dropDatabase;
&dropDomain;
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 9fa4e2da1d..174e6f6d5c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,6 +438,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return GetCompressionOidFromCompressionId(cmid);
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -451,12 +482,16 @@ toast_decompress_datum(struct varlena *attr)
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
- return cmroutine->cmdecompress(attr);
+ /* call the decompression routine */
+ return cmroutine->cmdecompress(attr,
+ IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
}
@@ -470,24 +505,30 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
+ Oid cmoid;
+ int32 header_sz;
CompressionRoutine *cmroutine;
- Oid cmoid;
Assert(VARATT_IS_COMPRESSED(attr));
- cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));
+ /* get the compression method oid */
+ cmoid = toast_get_compression_oid(attr);
/* get compression method handler routines */
cmroutine = GetCompressionRoutine(cmoid);
+ /* get the compression header size */
+ header_sz = IsCustomCompression(GetCompressionId(cmoid)) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ;
+
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->cmdecompress_slice)
- return cmroutine->cmdecompress_slice(attr, slicelength);
+ return cmroutine->cmdecompress_slice(attr, header_sz, slicelength);
else
- return cmroutine->cmdecompress(attr);
+ return cmroutine->cmdecompress(attr, header_sz);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 7c603dd19a..d69dd907c9 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -38,10 +38,14 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* --------
*/
void
-toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid, int32 rawsize)
+toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+ Oid cmoid, int32 rawsize)
{
TOAST_COMPRESS_SET_RAWSIZE(val, rawsize);
TOAST_COMPRESS_SET_COMPRESSION_METHOD(val, cmid);
+
+ if (IsCustomCompression(cmid))
+ TOAST_COMPRESS_SET_CMID(val, cmoid);
}
/* ----------
@@ -62,6 +66,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ CompressionId cmid;
CompressionRoutine *cmroutine;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -73,9 +78,12 @@ toast_compress_datum(Datum value, Oid cmoid)
/* Get the compression routines for the compression method */
cmroutine = GetCompressionRoutine(cmoid);
+ cmid = GetCompressionId(cmoid);
/* Call the actual compression function */
- tmp = cmroutine->cmcompress((const struct varlena *) value);
+ tmp = cmroutine->cmcompress((const struct varlena *) value,
+ IsCustomCompression(cmid) ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -94,8 +102,7 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- toast_set_compressed_datum_info(tmp, GetCompressionId(cmoid),
- valsize);
+ toast_set_compressed_datum_info(tmp, cmid, cmoid, valsize);
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compressionapi.c b/src/backend/access/compression/compressionapi.c
index ee57261213..cf91b1c69b 100644
--- a/src/backend/access/compression/compressionapi.c
+++ b/src/backend/access/compression/compressionapi.c
@@ -129,6 +129,6 @@ GetCompressionId(Oid cmoid)
case LZ4_COMPRESSION_OID:
return LZ4_COMPRESSION_ID;
default:
- elog(ERROR, "Invalid compression method %d", cmoid);
+ return CUSTOM_COMPRESSION_ID;
}
}
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index edc8f5ed22..52c167a35d 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize;
int32 len;
@@ -39,10 +39,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -50,7 +50,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -60,7 +60,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
int32 rawsize;
struct varlena *result;
@@ -68,9 +68,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -86,17 +86,18 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
int32 rawsize;
struct varlena *result;
- result = (struct varlena *)palloc(slicelength + VARHDRSZ);
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial((char *) value + TOAST_COMPRESS_HDRSZ,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
slicelength);
if (rawsize < 0)
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 61dfe42004..931394f779 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/catalog/aclchk.c b/src/backend/catalog/aclchk.c
index c626161408..bf47b230b1 100644
--- a/src/backend/catalog/aclchk.c
+++ b/src/backend/catalog/aclchk.c
@@ -3410,6 +3410,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
@@ -3549,6 +3550,7 @@ aclcheck_error(AclResult aclerr, ObjectType objtype,
case OBJECT_AMPROC:
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
case OBJECT_DOMCONSTRAINT:
diff --git a/src/backend/catalog/dependency.c b/src/backend/catalog/dependency.c
index 076ef56ba5..c928610c14 100644
--- a/src/backend/catalog/dependency.c
+++ b/src/backend/catalog/dependency.c
@@ -1566,6 +1566,7 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_AM:
case OCLASS_AMOP:
case OCLASS_AMPROC:
+ case OCLASS_COMPRESSION:
case OCLASS_SCHEMA:
case OCLASS_TSPARSER:
case OCLASS_TSDICT:
@@ -1587,7 +1588,6 @@ doDeletion(const ObjectAddress *object, int flags)
case OCLASS_DATABASE:
case OCLASS_TBLSPACE:
case OCLASS_SUBSCRIPTION:
- case OCLASS_COMPRESSION:
elog(ERROR, "global objects cannot be deleted by doDeletion");
break;
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 8f95da758f..f80305cf44 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -189,6 +189,20 @@ static const ObjectPropertyType ObjectProperty[] =
OBJECT_COLLATION,
true
},
+ {
+ "compression method",
+ CompressionRelationId,
+ CompressionIndexId,
+ CMOID,
+ CMNAME,
+ Anum_pg_compression_oid,
+ Anum_pg_compression_cmname,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ InvalidAttrNumber,
+ -1,
+ true
+ },
{
"constraint",
ConstraintRelationId,
@@ -1009,6 +1023,7 @@ get_object_address(ObjectType objtype, Node *object,
case OBJECT_FOREIGN_SERVER:
case OBJECT_EVENT_TRIGGER:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_PUBLICATION:
case OBJECT_SUBSCRIPTION:
address = get_object_address_unqualified(objtype,
@@ -1260,6 +1275,11 @@ get_object_address_unqualified(ObjectType objtype,
address.objectId = get_am_oid(name, missing_ok);
address.objectSubId = 0;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ address.classId = CompressionRelationId;
+ address.objectId = get_cm_oid(name, missing_ok);
+ address.objectSubId = 0;
+ break;
case OBJECT_DATABASE:
address.classId = DatabaseRelationId;
address.objectId = get_database_oid(name, missing_ok);
@@ -2271,6 +2291,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
objnode = (Node *) name;
break;
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_DATABASE:
case OBJECT_EVENT_TRIGGER:
case OBJECT_EXTENSION:
@@ -2564,6 +2585,7 @@ check_object_ownership(Oid roleid, ObjectType objtype, ObjectAddress address,
case OBJECT_TSPARSER:
case OBJECT_TSTEMPLATE:
case OBJECT_ACCESS_METHOD:
+ case OBJECT_COMPRESSION_METHOD:
/* We treat these object types as being owned by superusers */
if (!superuser_arg(roleid))
ereport(ERROR,
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a1be6ef7be..ce6f7f83f5 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -20,11 +20,115 @@
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
+#include "catalog/objectaccess.h"
#include "catalog/pg_attribute.h"
+#include "catalog/pg_compression.h"
#include "catalog/pg_depend.h"
+#include "catalog/pg_proc.h"
#include "commands/defrem.h"
+#include "miscadmin.h"
#include "nodes/parsenodes.h"
+#include "parser/parse_func.h"
#include "utils/fmgroids.h"
+#include "utils/fmgrprotos.h"
+#include "utils/syscache.h"
+
+/*
+ * CreateCompressionMethod
+ * Registers a new compression method.
+ */
+ObjectAddress
+CreateCompressionMethod(CreateCmStmt *stmt)
+{
+ Relation rel;
+ ObjectAddress myself;
+ ObjectAddress referenced;
+ Oid cmoid;
+ Oid cmhandler;
+ bool nulls[Natts_pg_compression];
+ Datum values[Natts_pg_compression];
+ HeapTuple tup;
+ Oid funcargtypes[1] = {INTERNALOID};
+
+ rel = table_open(CompressionRelationId, RowExclusiveLock);
+
+ /* Must be super user */
+ if (!superuser())
+ ereport(ERROR,
+ (errcode(ERRCODE_INSUFFICIENT_PRIVILEGE),
+ errmsg("permission denied to create compression method \"%s\"",
+ stmt->cmname),
+ errhint("Must be superuser to create an access method.")));
+
+ /* Check if name is used */
+ cmoid = GetSysCacheOid1(CMNAME, Anum_pg_compression_oid,
+ CStringGetDatum(stmt->cmname));
+ if (OidIsValid(cmoid))
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_DUPLICATE_OBJECT),
+ errmsg("compression method \"%s\" already exists",
+ stmt->cmname)));
+ }
+
+ if (stmt->handler_name == NIL)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_FUNCTION),
+ errmsg("handler function is not specified")));
+
+ /* Get the handler function oid */
+ cmhandler = LookupFuncName(stmt->handler_name, 1, funcargtypes, false);
+
+ /*
+ * Insert tuple into pg_compression.
+ */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+
+ cmoid = GetNewOidWithIndex(rel, CompressionIndexId, Anum_pg_compression_oid);
+ values[Anum_pg_compression_oid - 1] = ObjectIdGetDatum(cmoid);
+ values[Anum_pg_compression_cmname - 1] =
+ DirectFunctionCall1(namein, CStringGetDatum(stmt->cmname));
+ values[Anum_pg_compression_cmhandler - 1] = ObjectIdGetDatum(cmhandler);
+
+ tup = heap_form_tuple(RelationGetDescr(rel), values, nulls);
+
+ CatalogTupleInsert(rel, tup);
+ heap_freetuple(tup);
+
+ myself.classId = CompressionRelationId;
+ myself.objectId = cmoid;
+ myself.objectSubId = 0;
+
+ /* Record dependency on handler function */
+ referenced.classId = ProcedureRelationId;
+ referenced.objectId = cmhandler;
+ referenced.objectSubId = 0;
+
+ recordDependencyOn(&myself, &referenced, DEPENDENCY_NORMAL);
+
+ recordDependencyOnCurrentExtension(&myself, false);
+
+ InvokeObjectPostCreateHook(CompressionRelationId, cmoid, 0);
+
+ table_close(rel, RowExclusiveLock);
+
+ return myself;
+}
+
+Oid
+get_cm_oid(const char *cmname, bool missing_ok)
+{
+ Oid cmoid;
+
+ cmoid = GetCompressionOid(cmname);
+ if (!OidIsValid(cmoid) && !missing_ok)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("compression type \"%s\" not recognized", cmname)));
+
+ return cmoid;
+}
/*
* get list of all supported compression methods for the given attribute.
diff --git a/src/backend/commands/event_trigger.c b/src/backend/commands/event_trigger.c
index 172242ae37..b2a07bc977 100644
--- a/src/backend/commands/event_trigger.c
+++ b/src/backend/commands/event_trigger.c
@@ -953,6 +953,7 @@ EventTriggerSupportsObjectType(ObjectType obtype)
case OBJECT_CAST:
case OBJECT_COLUMN:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFACL:
case OBJECT_DEFAULT:
@@ -2106,6 +2107,7 @@ stringify_grant_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
@@ -2188,6 +2190,7 @@ stringify_adefprivs_objtype(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/commands/seclabel.c b/src/backend/commands/seclabel.c
index ee036e9087..083ac78e11 100644
--- a/src/backend/commands/seclabel.c
+++ b/src/backend/commands/seclabel.c
@@ -67,6 +67,7 @@ SecLabelSupportsObjectType(ObjectType objtype)
case OBJECT_ATTRIBUTE:
case OBJECT_CAST:
case OBJECT_COLLATION:
+ case OBJECT_COMPRESSION_METHOD:
case OBJECT_CONVERSION:
case OBJECT_DEFAULT:
case OBJECT_DEFACL:
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5257aeb66b..6653bd4d9a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1970,8 +1970,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* Get the compression method stored in the toast header and
* compare with the compression method of the target.
*/
- cmoid = GetCompressionOidFromCompressionId(
- TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 5413bd92a4..707c6ea90b 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -4312,6 +4312,17 @@ _copyCreateAmStmt(const CreateAmStmt *from)
return newnode;
}
+static CreateCmStmt *
+_copyCreateCmStmt(const CreateCmStmt *from)
+{
+ CreateCmStmt *newnode = makeNode(CreateCmStmt);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_NODE_FIELD(handler_name);
+
+ return newnode;
+}
+
static CreateTrigStmt *
_copyCreateTrigStmt(const CreateTrigStmt *from)
{
@@ -5474,6 +5485,9 @@ copyObjectImpl(const void *from)
case T_CreateAmStmt:
retval = _copyCreateAmStmt(from);
break;
+ case T_CreateCmStmt:
+ retval = _copyCreateCmStmt(from);
+ break;
case T_CreateTrigStmt:
retval = _copyCreateTrigStmt(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 09882c9375..0c17cec725 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2008,6 +2008,15 @@ _equalCreateAmStmt(const CreateAmStmt *a, const CreateAmStmt *b)
return true;
}
+static bool
+_equalCreateCmStmt(const CreateCmStmt *a, const CreateCmStmt *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_NODE_FIELD(handler_name);
+
+ return true;
+}
+
static bool
_equalCreateTrigStmt(const CreateTrigStmt *a, const CreateTrigStmt *b)
{
@@ -3527,6 +3536,9 @@ equal(const void *a, const void *b)
case T_CreateAmStmt:
retval = _equalCreateAmStmt(a, b);
break;
+ case T_CreateCmStmt:
+ retval = _equalCreateCmStmt(a, b);
+ break;
case T_CreateTrigStmt:
retval = _equalCreateTrigStmt(a, b);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index deb82ea660..d21cb7c94c 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -289,7 +289,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
DeallocateStmt PrepareStmt ExecuteStmt
DropOwnedStmt ReassignOwnedStmt
AlterTSConfigurationStmt AlterTSDictionaryStmt
- CreateMatViewStmt RefreshMatViewStmt CreateAmStmt
+ CreateMatViewStmt RefreshMatViewStmt CreateAmStmt CreateCmStmt
CreatePublicationStmt AlterPublicationStmt
CreateSubscriptionStmt AlterSubscriptionStmt DropSubscriptionStmt
@@ -881,6 +881,7 @@ stmt :
| CreateAsStmt
| CreateAssertionStmt
| CreateCastStmt
+ | CreateCmStmt
| CreateConversionStmt
| CreateDomainStmt
| CreateExtensionStmt
@@ -5263,6 +5264,22 @@ am_type:
| TABLE { $$ = AMTYPE_TABLE; }
;
+/*****************************************************************************
+ *
+ * QUERY:
+ * CREATE COMPRESSION METHOD name HANDLER handler_name
+ *
+ *****************************************************************************/
+
+CreateCmStmt: CREATE COMPRESSION METHOD name HANDLER handler_name
+ {
+ CreateCmStmt *n = makeNode(CreateCmStmt);
+ n->cmname = $4;
+ n->handler_name = $6;
+ $$ = (Node *) n;
+ }
+ ;
+
/*****************************************************************************
*
* QUERIES :
@@ -6265,6 +6282,7 @@ object_type_name:
drop_type_name:
ACCESS METHOD { $$ = OBJECT_ACCESS_METHOD; }
+ | COMPRESSION METHOD { $$ = OBJECT_COMPRESSION_METHOD; }
| EVENT TRIGGER { $$ = OBJECT_EVENT_TRIGGER; }
| EXTENSION { $$ = OBJECT_EXTENSION; }
| FOREIGN DATA_P WRAPPER { $$ = OBJECT_FDW; }
diff --git a/src/backend/tcop/utility.c b/src/backend/tcop/utility.c
index f398027fa6..dfd79d1007 100644
--- a/src/backend/tcop/utility.c
+++ b/src/backend/tcop/utility.c
@@ -168,6 +168,7 @@ ClassifyUtilityCommandAsReadOnly(Node *parsetree)
case T_CompositeTypeStmt:
case T_CreateAmStmt:
case T_CreateCastStmt:
+ case T_CreateCmStmt:
case T_CreateConversionStmt:
case T_CreateDomainStmt:
case T_CreateEnumStmt:
@@ -1805,6 +1806,10 @@ ProcessUtilitySlow(ParseState *pstate,
address = CreateAccessMethod((CreateAmStmt *) parsetree);
break;
+ case T_CreateCmStmt:
+ address = CreateCompressionMethod((CreateCmStmt *) parsetree);
+ break;
+
case T_CreatePublicationStmt:
address = CreatePublication((CreatePublicationStmt *) parsetree);
break;
@@ -2562,6 +2567,9 @@ CreateCommandTag(Node *parsetree)
case OBJECT_ACCESS_METHOD:
tag = CMDTAG_DROP_ACCESS_METHOD;
break;
+ case OBJECT_COMPRESSION_METHOD:
+ tag = CMDTAG_DROP_COMPRESSION_METHOD;
+ break;
case OBJECT_PUBLICATION:
tag = CMDTAG_DROP_PUBLICATION;
break;
@@ -2969,6 +2977,10 @@ CreateCommandTag(Node *parsetree)
tag = CMDTAG_CREATE_ACCESS_METHOD;
break;
+ case T_CreateCmStmt:
+ tag = CMDTAG_CREATE_COMPRESSION_METHOD;
+ break;
+
case T_CreatePublicationStmt:
tag = CMDTAG_CREATE_PUBLICATION;
break;
@@ -3573,6 +3585,10 @@ GetCommandLogLevel(Node *parsetree)
lev = LOGSTMT_DDL;
break;
+ case T_CreateCmStmt:
+ lev = LOGSTMT_DDL;
+ break;
+
case T_CreatePublicationStmt:
lev = LOGSTMT_DDL;
break;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 6660285e84..4d4326ff57 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -23,23 +23,31 @@
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression method oid. And,
* using that oid we can get the compression handler routine by fetching the
- * pg_compression catalog row.
+ * pg_compression catalog row. If it is custome compression id then the
+ * compressed data will store special custom compression header wherein it will
+ * directly store the oid of the custom compression method.
*/
typedef enum CompressionId
{
- PGLZ_COMPRESSION_ID,
- LZ4_COMPRESSION_ID
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
-/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+/* Use default compression method if it is not specified */
+#define DefaultCompressionOid PGLZ_COMPRESSION_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_slice_function)(
+ const struct varlena *value,
+ int32 slicelength,
+ int32 toast_header_size);
/*
* API struct for a compression.
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 8ef477cfe0..48ca172eae 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -52,6 +64,9 @@ do { \
#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);
+#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
+ (((toast_compress_header_custom *) (ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
@@ -72,7 +87,9 @@ extern void init_toast_snapshot(Snapshot toast_snapshot);
*
* Save metadata in compressed datum
*/
-extern void toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
+extern void toast_set_compressed_datum_info(struct varlena *val,
+ CompressionId cmid,
+ Oid cmoid,
int32 rawsize);
#endif /* TOAST_INTERNALS_H */
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 936b072c76..8952b2b70d 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -146,6 +146,8 @@ extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
+extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
+extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
bool *need_rewrite);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 65ef987d67..ae297c9116 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -415,6 +415,7 @@ typedef enum NodeTag
T_AlterPolicyStmt,
T_CreateTransformStmt,
T_CreateAmStmt,
+ T_CreateCmStmt,
T_CreatePublicationStmt,
T_AlterPublicationStmt,
T_CreateSubscriptionStmt,
@@ -493,7 +494,7 @@ typedef enum NodeTag
T_TimeLineHistoryCmd,
T_SQLCmd,
- T_CompressionRoutine, /* in access/compressionapi.h */
+ T_CompressionRoutine, /* in access/compressionapi.h */
/*
* TAGS FOR RANDOM OTHER STUFF
*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index e0a51f0145..89478ddea3 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1722,6 +1722,7 @@ typedef enum ObjectType
OBJECT_CAST,
OBJECT_COLUMN,
OBJECT_COLLATION,
+ OBJECT_COMPRESSION_METHOD,
OBJECT_CONVERSION,
OBJECT_DATABASE,
OBJECT_DEFAULT,
@@ -2435,6 +2436,17 @@ typedef struct CreateAmStmt
char amtype; /* type of access method */
} CreateAmStmt;
+/*----------------------
+ * Create COMPRESSION METHOD Statement
+ *----------------------
+ */
+typedef struct CreateCmStmt
+{
+ NodeTag type;
+ char *cmname; /* compression method name */
+ List *handler_name; /* handler function name */
+} CreateCmStmt;
+
/* ----------------------
* Create TRIGGER Statement
* ----------------------
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 4e7cad3daf..a8893759cf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/include/tcop/cmdtaglist.h b/src/include/tcop/cmdtaglist.h
index be94852bbd..a9bc1c4bae 100644
--- a/src/include/tcop/cmdtaglist.h
+++ b/src/include/tcop/cmdtaglist.h
@@ -83,6 +83,7 @@ PG_CMDTAG(CMDTAG_COMMIT_PREPARED, "COMMIT PREPARED", false, false, false)
PG_CMDTAG(CMDTAG_COPY, "COPY", false, false, true)
PG_CMDTAG(CMDTAG_COPY_FROM, "COPY FROM", false, false, false)
PG_CMDTAG(CMDTAG_CREATE_ACCESS_METHOD, "CREATE ACCESS METHOD", true, false, false)
+PG_CMDTAG(CMDTAG_CREATE_COMPRESSION_METHOD, "CREATE COMPRESSION METHOD", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_AGGREGATE, "CREATE AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_CAST, "CREATE CAST", true, false, false)
PG_CMDTAG(CMDTAG_CREATE_COLLATION, "CREATE COLLATION", true, false, false)
@@ -138,6 +139,7 @@ PG_CMDTAG(CMDTAG_DROP_ACCESS_METHOD, "DROP ACCESS METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_AGGREGATE, "DROP AGGREGATE", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CAST, "DROP CAST", true, false, false)
PG_CMDTAG(CMDTAG_DROP_COLLATION, "DROP COLLATION", true, false, false)
+PG_CMDTAG(CMDTAG_DROP_COMPRESSION_METHOD, "DROP COMPRESSIOn METHOD", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONSTRAINT, "DROP CONSTRAINT", true, false, false)
PG_CMDTAG(CMDTAG_DROP_CONVERSION, "DROP CONVERSION", true, false, false)
PG_CMDTAG(CMDTAG_DROP_DATABASE, "DROP DATABASE", false, false, false)
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 1b7b5bd6e5..897a61689f 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -103,4 +103,22 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
+\d+ cmmove2
+ Table "public.cmmove2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index 3549fb35f0..a0a0aa13f0 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -51,4 +51,11 @@ ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
\d+ cmmove2
+-- create compression method
+CREATE COMPRESSION METHOD pglz2 HANDLER pglzhandler;
+ALTER TABLE cmmove2 ALTER COLUMN f1 SET COMPRESSION pglz2;
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT length(f1) FROM cmmove2;
+\d+ cmmove2
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
v14-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v14-0006-Support-compression-methods-options.patchDownload
From c3f81fde98470cedf372b6350133131f42767942 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sat, 24 Oct 2020 20:16:48 +0530
Subject: [PATCH v14 6/6] Support compression methods options
---
contrib/cmzlib/zlib.c | 75 +++++++++++++++--
doc/src/sgml/ref/alter_table.sgml | 18 ++--
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 42 ++++++++++
src/backend/access/common/toast_internals.c | 16 +++-
src/backend/access/compression/lz4.c | 69 ++++++++++++++-
src/backend/access/compression/pglz.c | 93 ++++++++++++++++++++-
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 43 ++++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 49 ++++++++++-
src/backend/commands/foreigncmds.c | 44 ----------
src/backend/commands/tablecmds.c | 51 +++++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/include/access/compressionapi.h | 14 +++-
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 4 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/create_cm.out | 14 ++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/create_cm.sql | 8 ++
32 files changed, 510 insertions(+), 104 deletions(-)
diff --git a/contrib/cmzlib/zlib.c b/contrib/cmzlib/zlib.c
index f24fc1c936..fa7c657bfa 100644
--- a/contrib/cmzlib/zlib.c
+++ b/contrib/cmzlib/zlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -150,6 +205,8 @@ zlibhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = zlib_cmcheck;
+ routine->cminitstate = zlib_cminitstate;
routine->cmcompress = zlib_cmcompress;
routine->cmdecompress = zlib_cmdecompress;
routine->cmdecompress_slice = NULL;
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 733ee0cf20..58782153a6 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,17 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be
- set from available built-in compression methods. The PRESERVE list
- contains list of compression methods used on the column and determines
- which of them should be kept on the column. Without PRESERVE or if all
- the previous compression methods are not preserved then the table will
- be rewritten. If PRESERVE ALL is specified then all the previous methods
- will be preserved and the table will not be rewritten.
+ set from available built-in compression methods. If compression
+ method has options they could be specified with <literal>WITH</literal>
+ parameter. The PRESERVE list contains list of compression methods used
+ on the column and determines which of them should be kept on the column.
+ Without PRESERVE or if all the previous compression methods are not
+ preserved then the table will be rewritten. If PRESERVE ALL is specified
+ then all the previous methods will be preserved and the table will not be
+ rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index c559d1ffd5..466945fcfe 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -969,13 +969,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
can be set from the available built-in compression methods. The available
options are <literal>pglz</literal> and <literal>lz4</literal>. The
- default compression method is <literal>pglz</literal>.
+ default compression method is <literal>pglz</literal>. If compression
+ method has options they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index cda7b061d2..4b1462463d 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index f81fc8ea66..d8e8b560d9 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..3185082034 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,45 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d69dd907c9..2907f210da 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -62,10 +62,11 @@ toast_set_compressed_datum_info(struct varlena *val, CompressionId cmid,
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
CompressionId cmid;
CompressionRoutine *cmroutine;
@@ -80,10 +81,21 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = GetCompressionRoutine(cmoid);
cmid = GetCompressionId(cmoid);
+ if (cmroutine->cminitstate)
+ options = cmroutine->cminitstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->cmcompress((const struct varlena *) value,
IsCustomCompression(cmid) ?
- TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ);
+ TOAST_CUSTOM_COMPRESS_HDRSZ : TOAST_COMPRESS_HDRSZ,
+ options);
+
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/lz4.c b/src/backend/access/compression/lz4.c
index 52c167a35d..533bda5239 100644
--- a/src/backend/access/compression/lz4.c
+++ b/src/backend/access/compression/lz4.c
@@ -16,12 +16,70 @@
#include "postgres.h"
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
#ifdef HAVE_LIBLZ4
#include "lz4.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -29,11 +87,12 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -41,9 +100,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -120,6 +179,8 @@ lz4handler(PG_FUNCTION_ARGS)
#else
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = lz4_cmcheck;
+ routine->cminitstate = lz4_cminitstate;
routine->cmcompress = lz4_cmcompress;
routine->cmdecompress = lz4_cmdecompress;
routine->cmdecompress_slice = lz4_cmdecompress_slice;
diff --git a/src/backend/access/compression/pglz.c b/src/backend/access/compression/pglz.c
index 931394f779..607849a73d 100644
--- a/src/backend/access/compression/pglz.c
+++ b/src/backend/access/compression/pglz.c
@@ -15,11 +15,92 @@
#include "access/compressionapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -122,6 +205,8 @@ pglzhandler(PG_FUNCTION_ARGS)
{
CompressionRoutine *routine = makeNode(CompressionRoutine);
+ routine->cmcheck = pglz_cmcheck;
+ routine->cminitstate = pglz_cminitstate;
routine->cmcompress = pglz_cmcompress;
routine->cmdecompress = pglz_cmdecompress;
routine->cmdecompress_slice = pglz_cmdecompress_slice;
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..7e399daaa1 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index a2e2763a7c..490de64185 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1150,6 +1158,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1407,7 +1416,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 200c2d24c7..9fe68e0731 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index f1850436bd..80fcb1c313 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -255,6 +255,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 04d12a7ece..40e93bebd7 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -675,6 +675,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index ce6f7f83f5..5a9bbca8c8 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -229,7 +229,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
ListCell *cell;
@@ -248,6 +248,22 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
(errcode(ERRCODE_INVALID_PARAMETER_VALUE),
errmsg("compression type \"%s\" not recognized", compression->cmname)));
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionRoutine *routine = GetCompressionRoutine(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->cmcheck != NULL)
+ routine->cmcheck(compression->options);
+
+ pfree(routine);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -324,3 +340,34 @@ MakeColumnCompression(Oid attcompression)
return node;
}
+
+/*
+ * Fetch atttributes compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e82b362bad..7ed878e176 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -611,6 +611,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -815,6 +816,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -868,8 +871,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (!IsBinaryUpgrade &&
(relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE))
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -923,8 +927,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -6152,6 +6159,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6314,7 +6322,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression, NULL);
+ colDef->compression, &acoptions, NULL);
else
attribute.attcompression = InvalidOid;
@@ -6324,7 +6332,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -15078,9 +15086,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
ListCell *lc;
@@ -15114,7 +15124,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression,
+ &acoptions, &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15123,7 +15134,18 @@ ATExecSetCompression(AlteredTableInfo *tab,
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+
+ /* update existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15162,7 +15184,22 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform = (Form_pg_attribute) GETSTRUCT(atttuple);
atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
+ /* update existing entry */
+ if (acoptions)
+ {
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(atttuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &atttuple->t_self, atttuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 707c6ea90b..9ea3707005 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2956,6 +2956,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 0c17cec725..861f6eea78 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2615,6 +2615,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 22757dfea9..fd748e9525 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2858,6 +2858,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index d21cb7c94c..8b5c4ab5aa 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
publication_name_list
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3461,11 +3462,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3473,14 +3480,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/include/access/compressionapi.h b/src/include/access/compressionapi.h
index 4d4326ff57..fb8e44754e 100644
--- a/src/include/access/compressionapi.h
+++ b/src/include/access/compressionapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_compression_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -42,7 +43,12 @@ typedef enum CompressionId
typedef struct CompressionRoutine CompressionRoutine;
/* compresion handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function)(const struct varlena *value,
+ int32 toast_header_size,
+ void *options);
+typedef struct varlena *(*cmdecompress_function)(const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)(
const struct varlena *value,
@@ -61,11 +67,17 @@ struct CompressionRoutine
/* name of the compression method */
char cmname[64];
+ /* compression option check, can be NULL */
+ cmcheck_function cmcheck;
+
+ /* compression option intialize, can be NULL */
+ cminitstate_function cminitstate;
+
/* compression routine for the compression method */
cmcompress_function cmcompress;
/* decompression routine for the compression method */
- cmcompress_function cmdecompress;
+ cmdecompress_function cmdecompress;
/* slice decompression routine for the compression method */
cmdecompress_slice_function cmdecompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a88e3daa6a..9071e80714 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -33,6 +33,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 48ca172eae..53becd2057 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -67,7 +67,7 @@ do { \
#define TOAST_COMPRESS_SET_CMID(ptr, oid) \
(((toast_compress_header_custom *) (ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index f94defff3c..bd192f457e 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index c506d31029..7e4e211153 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -171,6 +171,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 8952b2b70d..9c5831a185 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -137,6 +137,7 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,9 +151,10 @@ extern ObjectAddress CreateCompressionMethod(CreateCmStmt *stmt);
extern Oid get_cm_oid(const char *cmname, bool missing_ok);
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
+ Datum *acoptions, bool *need_rewrite);
extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 89478ddea3..74ec86f313 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -636,6 +636,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/create_cm.out b/src/test/regress/expected/create_cm.out
index 897a61689f..c9f324ed4c 100644
--- a/src/test/regress/expected/create_cm.out
+++ b/src/test/regress/expected/create_cm.out
@@ -121,4 +121,18 @@ SELECT length(f1) FROM cmmove2;
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz2 | |
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+ 10040
+ 10040
+(3 rows)
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 9029f76147..81be500a2e 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -98,6 +98,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -108,5 +109,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/create_cm.sql b/src/test/regress/sql/create_cm.sql
index a0a0aa13f0..9135ce5e5d 100644
--- a/src/test/regress/sql/create_cm.sql
+++ b/src/test/regress/sql/create_cm.sql
@@ -58,4 +58,12 @@ INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
SELECT length(f1) FROM cmmove2;
\d+ cmmove2
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+INSERT INTO cmmove1 VALUES (repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+SELECT length(f1) FROM cmmove2;
+
DROP TABLE cmmove1, cmmove2, cmmove3, lz4test;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.
Since the authorship history of this patch is complicated, it would be
nice if you would include authorship information and relevant
"Discussion" links in the patches.
Design level considerations and overall notes:
configure is autogenerated from configure.in, so the patch shouldn't
include changes only to the former.
Looking over the changes to src/include:
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_ID
I think that it would be good to assign values to these explicitly.
+/* compresion handler routines */
Spelling.
+ /* compression routine for the compression method */
+ cmcompress_function cmcompress;
+
+ /* decompression routine for the compression method */
+ cmcompress_function cmdecompress;Don't reuse cmcompress_function; that's confusing. Just have a typedef
per structure member, even if they end up being the same.
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+do { \
+ Assert(len > 0 && len <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = (len); \
+} while (0)Indentation.
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \
+ ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);What about making TOAST_COMPRESS_SET_RAWSIZE() take another argument?
And possibly also rename it to TEST_COMPRESS_SET_SIZE_AND_METHOD() or
something? It seems not great to have separate functions each setting
part of a 4-byte quantity. Too much chance of failing to set both
parts. I guess you've got a function called
toast_set_compressed_datum_info() for that, but it's just a wrapper
around two macros that could just be combined, which would reduce
complexity overall.
+ T_CompressionRoutine, /* in access/compressionapi.h */
This looks misplaced. I guess it should go just after these:
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
Looking over the regression test changes:
The tests at the top of create_cm.out that just test that we can
create tables with various storage types seem unrelated to the purpose
of the patch. And the file doesn't test creating a compression method
either, as the file name would suggest, so either the file name needs
to be changed (compression, compression_method?) or the tests don't go
here.
+-- check data is okdd
I guess whoever is responsible for this comment prefers vi to emacs.
I don't quite understand the purpose of all of these tests, and there
are some things that I feel like ought to be tested that seemingly
aren't. Like, you seem to test using an UPDATE to move a datum from a
table to another table with the same compression method, but not one
with a different compression method. Testing the former is nice and
everything, but that's the easy case: I think we also need to test the
latter. I think it would be good to verify not only that the data is
readable but that it's compressed the way we expect. I think it would
be a great idea to add a pg_column_compression() function in a similar
spirit to pg_column_size(). Perhaps it could return NULL when
compression is not in use or the data type is not varlena, and the
name of the compression method otherwise. That would allow for better
testing of this feature, and it would also be useful to users who are
switching methods, to see what data they still have that's using the
old method. It could be useful for debugging problems on customer
systems, too.
I wonder if we need a test that moves data between tables through an
intermediary. For instance, suppose a plpgsql function or DO block
fetches some data and stores it in a plpgsql variable and then uses
the variable to insert into another table. Hmm, maybe that would force
de-TOASTing. But perhaps there are other cases. Maybe a more general
way to approach the problem is: have you tried running a coverage
report and checked which parts of your code are getting exercised by
the existing tests and which parts are not? The stuff that isn't, we
should try to add more tests. It's easy to get corner cases wrong with
this kind of thing.
I notice that LIKE INCLUDING COMPRESSION doesn't seem to be tested, at
least not by 0001, which reinforces my feeling that the tests here are
not as thorough as they could be.
+NOTICE: pg_compression contains unpinned initdb-created object(s)
This seems wrong to me - why is it OK?
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
-
TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* get compression method handler routines */
+ cmroutine = GetCompressionRoutine(cmoid);- return result;
+ return cmroutine->cmdecompress(attr);I'm worried about how expensive this might be, and I think we could
make it cheaper. The reason why I think this might be expensive is:
currently, for every datum, you have a single direct function call.
Now, with this, you first have a direct function call to
GetCompressionOidFromCompressionId(). Then you have a call to
GetCompressionRoutine(), which does a syscache lookup and calls a
handler function, which is quite a lot more expensive than a single
function call. And the handler isn't even returning a statically
allocated structure, but is allocating new memory every time, which
involves more function calls and maybe memory leaks. Then you use the
results of all that to make an indirect function call.
I'm not sure exactly what combination of things we could use to make
this better, but it seems like there are a few possibilities:
(1) The handler function could return a pointer to the same
CompressionRoutine every time instead of constructing a new one every
time.
(2) The CompressionRoutine to which the handler function returns a
pointer could be statically allocated instead of being built at
runtime.
(3) GetCompressionRoutine could have an OID -> handler cache instead
of relying on syscache + calling the handler function all over again.
(4) For the compression types that have dedicated bit patterns in the
high bits of the compressed TOAST size, toast_compress_datum() could
just have hard-coded logic to use the correct handlers instead of
translating the bit pattern into an OID and then looking it up over
again.
(5) Going even further than #4 we could skip the handler layer
entirely for such methods, and just call the right function directly.
I think we should definitely do (1), and also (2) unless there's some
reason it's hard. (3) doesn't need to be part of this patch, but might
be something to consider later in the series. It's possible that it
doesn't have enough benefit to be worth the work, though. Also, I
think we should do either (4) or (5). I have a mild preference for (5)
unless it looks too ugly.
Note that I'm not talking about hard-coding a fast path for a
hard-coded list of OIDs - which would seem a little bit unprincipled -
but hard-coding a fast path for the bit patterns that are themselves
hard-coded. I don't think we lose anything in terms of extensibility
or even-handedness there; it's just avoiding a bunch of rigamarole
that doesn't really buy us anything.
All these points apply equally to toast_decompress_datum_slice() and
toast_compress_datum().
+ /* Fallback to default compression method, if not specified */
+ if (!OidIsValid(cmoid))
+ cmoid = DefaultCompressionOid;I think that the caller should be required to specify a legal value,
and this should be an elog(ERROR) or an Assert().
The change to equalTupleDescs() makes me wonder. Like, can we specify
the compression method for a function parameter, or a function return
value? I would think not. But then how are the tuple descriptors set
up in that case? Under what circumstances do we actually need the
tuple descriptors to compare unequal?
lz4.c's header comment calls it cm_lz4.c, and the pathname is wrong too.
I wonder if we should try to adopt a convention for the names of these
files that isn't just the compression method name, like cmlz4 or
compress_lz4. I kind of like the latter one. I am a little worried
that just calling it lz4.c will result in name collisions later - not
in this directory, of course, but elsewhere in the system. It's not a
disaster if that happens, but for example verbose error reports print
the file name, so it's nice if it's unambiguous.
+ if (!IsBinaryUpgrade &&
+ (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE))
+ attr->attcompression =
+
GetAttributeCompressionMethod(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;Storing InvalidOid in the IsBinaryUpgrade case looks wrong. If
upgrading from pre-v14, we need to store PGLZ_COMPRESSION_OID.
Otherwise, we need to preserve whatever value was present in the old
version. Or am I confused here?
I think there should be tests for the way this interacts with
partitioning, and I think the intended interaction should be
documented. Perhaps it should behave like TABLESPACE, where the parent
property has no effect on what gets stored because the parent has no
storage, but is inherited by each new child.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression? For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example). For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.
+ /*
+ * Use default compression method if the existing compression method is
+ * invalid but the new storage type is non plain storage.
+ */
+ if (!OidIsValid(attrtuple->attcompression) &&
+ (newstorage != TYPSTORAGE_PLAIN))
+ attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
I don't see a particularly good reason to treat plain and external
differently. More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure). However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.
I'm out of time for today, so I'll have to look at this more another
day. Hope this helps for a start.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.Since the authorship history of this patch is complicated, it would be
nice if you would include authorship information and relevant
"Discussion" links in the patches.Design level considerations and overall notes:
configure is autogenerated from configure.in, so the patch shouldn't
include changes only to the former.Looking over the changes to src/include:
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_IDI think that it would be good to assign values to these explicitly.
+/* compresion handler routines */
Spelling.
+ /* compression routine for the compression method */ + cmcompress_function cmcompress; + + /* decompression routine for the compression method */ + cmcompress_function cmdecompress;Don't reuse cmcompress_function; that's confusing. Just have a typedef
per structure member, even if they end up being the same.#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \ - (((toast_compress_header *) (ptr))->rawsize = (len)) +do { \ + Assert(len > 0 && len <= RAWSIZEMASK); \ + ((toast_compress_header *) (ptr))->info = (len); \ +} while (0)Indentation.
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \ + ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);What about making TOAST_COMPRESS_SET_RAWSIZE() take another argument?
And possibly also rename it to TEST_COMPRESS_SET_SIZE_AND_METHOD() or
something? It seems not great to have separate functions each setting
part of a 4-byte quantity. Too much chance of failing to set both
parts. I guess you've got a function called
toast_set_compressed_datum_info() for that, but it's just a wrapper
around two macros that could just be combined, which would reduce
complexity overall.+ T_CompressionRoutine, /* in access/compressionapi.h */
This looks misplaced. I guess it should go just after these:
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */Looking over the regression test changes:
The tests at the top of create_cm.out that just test that we can
create tables with various storage types seem unrelated to the purpose
of the patch. And the file doesn't test creating a compression method
either, as the file name would suggest, so either the file name needs
to be changed (compression, compression_method?) or the tests don't go
here.+-- check data is okdd
I guess whoever is responsible for this comment prefers vi to emacs.
I don't quite understand the purpose of all of these tests, and there
are some things that I feel like ought to be tested that seemingly
aren't. Like, you seem to test using an UPDATE to move a datum from a
table to another table with the same compression method, but not one
with a different compression method. Testing the former is nice and
everything, but that's the easy case: I think we also need to test the
latter. I think it would be good to verify not only that the data is
readable but that it's compressed the way we expect. I think it would
be a great idea to add a pg_column_compression() function in a similar
spirit to pg_column_size(). Perhaps it could return NULL when
compression is not in use or the data type is not varlena, and the
name of the compression method otherwise. That would allow for better
testing of this feature, and it would also be useful to users who are
switching methods, to see what data they still have that's using the
old method. It could be useful for debugging problems on customer
systems, too.I wonder if we need a test that moves data between tables through an
intermediary. For instance, suppose a plpgsql function or DO block
fetches some data and stores it in a plpgsql variable and then uses
the variable to insert into another table. Hmm, maybe that would force
de-TOASTing. But perhaps there are other cases. Maybe a more general
way to approach the problem is: have you tried running a coverage
report and checked which parts of your code are getting exercised by
the existing tests and which parts are not? The stuff that isn't, we
should try to add more tests. It's easy to get corner cases wrong with
this kind of thing.I notice that LIKE INCLUDING COMPRESSION doesn't seem to be tested, at
least not by 0001, which reinforces my feeling that the tests here are
not as thorough as they could be.+NOTICE: pg_compression contains unpinned initdb-created object(s)
This seems wrong to me - why is it OK?
- result = (struct varlena *) - palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); - SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr), - TOAST_COMPRESS_SIZE(attr), - VARDATA(result), - TOAST_COMPRESS_RAWSIZE(attr), true) < 0) - elog(ERROR, "compressed data is corrupted"); + /* get compression method handler routines */ + cmroutine = GetCompressionRoutine(cmoid);- return result; + return cmroutine->cmdecompress(attr);I'm worried about how expensive this might be, and I think we could
make it cheaper. The reason why I think this might be expensive is:
currently, for every datum, you have a single direct function call.
Now, with this, you first have a direct function call to
GetCompressionOidFromCompressionId(). Then you have a call to
GetCompressionRoutine(), which does a syscache lookup and calls a
handler function, which is quite a lot more expensive than a single
function call. And the handler isn't even returning a statically
allocated structure, but is allocating new memory every time, which
involves more function calls and maybe memory leaks. Then you use the
results of all that to make an indirect function call.I'm not sure exactly what combination of things we could use to make
this better, but it seems like there are a few possibilities:(1) The handler function could return a pointer to the same
CompressionRoutine every time instead of constructing a new one every
time.
(2) The CompressionRoutine to which the handler function returns a
pointer could be statically allocated instead of being built at
runtime.
(3) GetCompressionRoutine could have an OID -> handler cache instead
of relying on syscache + calling the handler function all over again.
(4) For the compression types that have dedicated bit patterns in the
high bits of the compressed TOAST size, toast_compress_datum() could
just have hard-coded logic to use the correct handlers instead of
translating the bit pattern into an OID and then looking it up over
again.
(5) Going even further than #4 we could skip the handler layer
entirely for such methods, and just call the right function directly.I think we should definitely do (1), and also (2) unless there's some
reason it's hard. (3) doesn't need to be part of this patch, but might
be something to consider later in the series. It's possible that it
doesn't have enough benefit to be worth the work, though. Also, I
think we should do either (4) or (5). I have a mild preference for (5)
unless it looks too ugly.Note that I'm not talking about hard-coding a fast path for a
hard-coded list of OIDs - which would seem a little bit unprincipled -
but hard-coding a fast path for the bit patterns that are themselves
hard-coded. I don't think we lose anything in terms of extensibility
or even-handedness there; it's just avoiding a bunch of rigamarole
that doesn't really buy us anything.All these points apply equally to toast_decompress_datum_slice() and
toast_compress_datum().+ /* Fallback to default compression method, if not specified */ + if (!OidIsValid(cmoid)) + cmoid = DefaultCompressionOid;I think that the caller should be required to specify a legal value,
and this should be an elog(ERROR) or an Assert().The change to equalTupleDescs() makes me wonder. Like, can we specify
the compression method for a function parameter, or a function return
value? I would think not. But then how are the tuple descriptors set
up in that case? Under what circumstances do we actually need the
tuple descriptors to compare unequal?lz4.c's header comment calls it cm_lz4.c, and the pathname is wrong too.
I wonder if we should try to adopt a convention for the names of these
files that isn't just the compression method name, like cmlz4 or
compress_lz4. I kind of like the latter one. I am a little worried
that just calling it lz4.c will result in name collisions later - not
in this directory, of course, but elsewhere in the system. It's not a
disaster if that happens, but for example verbose error reports print
the file name, so it's nice if it's unambiguous.+ if (!IsBinaryUpgrade && + (relkind == RELKIND_RELATION || + relkind == RELKIND_PARTITIONED_TABLE)) + attr->attcompression = + GetAttributeCompressionMethod(attr, colDef->compression); + else + attr->attcompression = InvalidOid;Storing InvalidOid in the IsBinaryUpgrade case looks wrong. If
upgrading from pre-v14, we need to store PGLZ_COMPRESSION_OID.
Otherwise, we need to preserve whatever value was present in the old
version. Or am I confused here?I think there should be tests for the way this interacts with
partitioning, and I think the intended interaction should be
documented. Perhaps it should behave like TABLESPACE, where the parent
property has no effect on what gets stored because the parent has no
storage, but is inherited by each new child.I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression? For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example). For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
I don't see a particularly good reason to treat plain and external
differently. More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure). However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.I'm out of time for today, so I'll have to look at this more another
day. Hope this helps for a start.
Thanks for the review Robert, I will work on these comments and
provide my analysis along with the updated patch in a couple of days.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
Most of the comments looks fine to me but I have a slightly different
opinion for one of the comment so replying only for that.
I'm worried about how expensive this might be, and I think we could
make it cheaper. The reason why I think this might be expensive is:
currently, for every datum, you have a single direct function call.
Now, with this, you first have a direct function call to
GetCompressionOidFromCompressionId(). Then you have a call to
GetCompressionRoutine(), which does a syscache lookup and calls a
handler function, which is quite a lot more expensive than a single
function call. And the handler isn't even returning a statically
allocated structure, but is allocating new memory every time, which
involves more function calls and maybe memory leaks. Then you use the
results of all that to make an indirect function call.I'm not sure exactly what combination of things we could use to make
this better, but it seems like there are a few possibilities:(1) The handler function could return a pointer to the same
CompressionRoutine every time instead of constructing a new one every
time.
(2) The CompressionRoutine to which the handler function returns a
pointer could be statically allocated instead of being built at
runtime.
(3) GetCompressionRoutine could have an OID -> handler cache instead
of relying on syscache + calling the handler function all over again.
(4) For the compression types that have dedicated bit patterns in the
high bits of the compressed TOAST size, toast_compress_datum() could
just have hard-coded logic to use the correct handlers instead of
translating the bit pattern into an OID and then looking it up over
again.
(5) Going even further than #4 we could skip the handler layer
entirely for such methods, and just call the right function directly.I think we should definitely do (1), and also (2) unless there's some
reason it's hard. (3) doesn't need to be part of this patch, but might
be something to consider later in the series. It's possible that it
doesn't have enough benefit to be worth the work, though. Also, I
think we should do either (4) or (5). I have a mild preference for (5)
unless it looks too ugly.Note that I'm not talking about hard-coding a fast path for a
hard-coded list of OIDs - which would seem a little bit unprincipled -
but hard-coding a fast path for the bit patterns that are themselves
hard-coded. I don't think we lose anything in terms of extensibility
or even-handedness there; it's just avoiding a bunch of rigamarole
that doesn't really buy us anything.All these points apply equally to toast_decompress_datum_slice() and
toast_compress_datum().
I agree that (1) and (2) we shall definitely do as part of the first
patch and (3) we might do in later patches. I think from (4) and (5)
I am more inclined to do (4) for a couple of reasons
a) If we bypass the handler function and directly calls the
compression and decompression routines then we need to check whether
the current executable is compiled with this particular compression
library or not for example in 'lz4handler' we have this below check,
now if we don't have the handler function we either need to put this
in each compression/decompression functions or we need to put is in
each caller.
Datum
lz4handler(PG_FUNCTION_ARGS)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
(errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
errmsg("not built with lz4 support")));
#else
b) Another reason is that once we start supporting the compression
options (0006-Support-compression-methods-options.patch) then we also
need to call 'cminitstate_function' for parsing the compression
options and then calling the compression function, so we need to
hardcode multiple function calls.
I think b) is still okay but because of a) I am more inclined to do
(4), what is your opinion on this?
About (4), one option is that we directly call the correct handler
function for the built-in type directly from
toast_(de)compress(_slice) functions but in that case, we are
duplicating the code, another option is that we call the
GetCompressionRoutine() a common function and in that, for the
built-in type, we can directly call the corresponding handler function
and get the routine. The only thing is to avoid duplicating in
decompression routine we need to convert CompressionId to Oid before
calling GetCompressionRoutine(), but now we can avoid sys cache lookup
for the built-in type.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
About (4), one option is that we directly call the correct handler
function for the built-in type directly from
toast_(de)compress(_slice) functions but in that case, we are
duplicating the code, another option is that we call the
GetCompressionRoutine() a common function and in that, for the
built-in type, we can directly call the corresponding handler function
and get the routine. The only thing is to avoid duplicating in
decompression routine we need to convert CompressionId to Oid before
calling GetCompressionRoutine(), but now we can avoid sys cache lookup
for the built-in type.
Suppose that we have a variable lz4_methods (like heapam_methods) that
is always defined, whether or not lz4 support is present. It's defined
like this:
const CompressionAmRoutine lz4_compress_methods = {
.datum_compress = lz4_datum_compress,
.datum_decompress = lz4_datum_decompress,
.datum_decompress_slice = lz4_datum_decompress_slice
};
(It would be good, I think, to actually name things something like
this - in particular why would we have TableAmRoutine and
IndexAmRoutine but not include "Am" in the one for compression? In
general I think tableam is a good pattern to adhere to and we should
try to make this patch hew closely to it.)
Then those functions are contingent on #ifdef HAVE_LIBLZ4: they either
do their thing, or complain that lz4 compression is not supported.
Then in this function you can just say, well, if we have the 01 bit
pattern, handler = &lz4_compress_methods and proceed from there.
BTW, I think the "not supported" message should probably use the 'by
this build' language we use in some places i.e.
[rhaas pgsql]$ git grep errmsg.*'this build' | grep -vF .po:
contrib/pg_prewarm/pg_prewarm.c: errmsg("prefetch is not supported by
this build")));
src/backend/libpq/be-secure-openssl.c: (errmsg("\"%s\" setting \"%s\"
not supported by this build",
src/backend/libpq/be-secure-openssl.c: (errmsg("\"%s\" setting \"%s\"
not supported by this build",
src/backend/libpq/hba.c: errmsg("local connections are not supported
by this build"),
src/backend/libpq/hba.c: errmsg("hostssl record cannot match because
SSL is not supported by this build"),
src/backend/libpq/hba.c: errmsg("hostgssenc record cannot match
because GSSAPI is not supported by this build"),
src/backend/libpq/hba.c: errmsg("invalid authentication method \"%s\":
not supported by this build",
src/backend/utils/adt/pg_locale.c: errmsg("ICU is not supported in
this build"), \
src/backend/utils/misc/guc.c: GUC_check_errmsg("Bonjour is not
supported by this build");
src/backend/utils/misc/guc.c: GUC_check_errmsg("SSL is not supported
by this build");
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Nov 24, 2020 at 7:14 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Tue, Nov 24, 2020 at 7:11 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
About (4), one option is that we directly call the correct handler
function for the built-in type directly from
toast_(de)compress(_slice) functions but in that case, we are
duplicating the code, another option is that we call the
GetCompressionRoutine() a common function and in that, for the
built-in type, we can directly call the corresponding handler function
and get the routine. The only thing is to avoid duplicating in
decompression routine we need to convert CompressionId to Oid before
calling GetCompressionRoutine(), but now we can avoid sys cache lookup
for the built-in type.Suppose that we have a variable lz4_methods (like heapam_methods) that
is always defined, whether or not lz4 support is present. It's defined
like this:const CompressionAmRoutine lz4_compress_methods = {
.datum_compress = lz4_datum_compress,
.datum_decompress = lz4_datum_decompress,
.datum_decompress_slice = lz4_datum_decompress_slice
};
Yeah, this makes sense.
(It would be good, I think, to actually name things something like
this - in particular why would we have TableAmRoutine and
IndexAmRoutine but not include "Am" in the one for compression? In
general I think tableam is a good pattern to adhere to and we should
try to make this patch hew closely to it.)
For the compression routine name, I did not include "Am" because
currently, we are storing the compression method in the new catalog
"pg_compression" not in the pg_am. So are you suggesting that we
should store the compression methods also in the pg_am instead of
creating a new catalog? IMHO, storing the compression methods in a
new catalog is a better option instead of storing them in pg_am
because actually, the compression methods are not the same as heap or
index AMs, I mean they are actually not the access methods. Am I
missing something?
Then those functions are contingent on #ifdef HAVE_LIBLZ4: they either
do their thing, or complain that lz4 compression is not supported.
Then in this function you can just say, well, if we have the 01 bit
pattern, handler = &lz4_compress_methods and proceed from there.
Okay
BTW, I think the "not supported" message should probably use the 'by
this build' language we use in some places i.e.
Okay
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Nov 24, 2020 at 10:47 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
For the compression routine name, I did not include "Am" because
currently, we are storing the compression method in the new catalog
"pg_compression" not in the pg_am. So are you suggesting that we
should store the compression methods also in the pg_am instead of
creating a new catalog? IMHO, storing the compression methods in a
new catalog is a better option instead of storing them in pg_am
because actually, the compression methods are not the same as heap or
index AMs, I mean they are actually not the access methods. Am I
missing something?
Oh, I thought it had been suggested in previous discussions that these
should be treated as access methods rather than inventing a whole new
concept just for this, and it seemed like a good idea to me. I guess I
missed the fact that the patch wasn't doing it that way. Hmm.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Robert Haas <robertmhaas@gmail.com> writes:
On Tue, Nov 24, 2020 at 10:47 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
For the compression routine name, I did not include "Am" because
currently, we are storing the compression method in the new catalog
"pg_compression" not in the pg_am. So are you suggesting that we
should store the compression methods also in the pg_am instead of
creating a new catalog? IMHO, storing the compression methods in a
new catalog is a better option instead of storing them in pg_am
because actually, the compression methods are not the same as heap or
index AMs, I mean they are actually not the access methods. Am I
missing something?
Oh, I thought it had been suggested in previous discussions that these
should be treated as access methods rather than inventing a whole new
concept just for this, and it seemed like a good idea to me. I guess I
missed the fact that the patch wasn't doing it that way. Hmm.
FWIW, I kind of agree with Robert's take on this. Heap and index AMs
are pretty fundamentally different animals, yet we don't have a problem
sticking them in the same catalog. I think anything that is related to
storage access could reasonably go into that catalog, rather than
inventing a new one.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Tue, Nov 24, 2020 at 1:21 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
FWIW, I kind of agree with Robert's take on this. Heap and index AMs
are pretty fundamentally different animals, yet we don't have a problem
sticking them in the same catalog. I think anything that is related to
storage access could reasonably go into that catalog, rather than
inventing a new one.
It's good to have your opinion on this since I wasn't totally sure
what was best, but for the record, I can't take credit. Looks like it
was Álvaro's suggestion originally:
/messages/by-id/20171130205155.7mgq2cuqv6zxi25a@alvherre.pgsql
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On 2020-Nov-24, Tom Lane wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Oh, I thought it had been suggested in previous discussions that these
should be treated as access methods rather than inventing a whole new
concept just for this, and it seemed like a good idea to me. I guess I
missed the fact that the patch wasn't doing it that way. Hmm.FWIW, I kind of agree with Robert's take on this. Heap and index AMs
are pretty fundamentally different animals, yet we don't have a problem
sticking them in the same catalog. I think anything that is related to
storage access could reasonably go into that catalog, rather than
inventing a new one.
Right -- Something like amname=lz4, amhandler=lz4handler, amtype=c.
The core code must of course know how to instantiate an AM of type
'c' and what to use it for.
Re: [HACKERS] Custom compression methods
On Wed, Nov 25, 2020 at 12:50 AM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
On 2020-Nov-24, Tom Lane wrote:
Robert Haas <robertmhaas@gmail.com> writes:
Oh, I thought it had been suggested in previous discussions that these
should be treated as access methods rather than inventing a whole new
concept just for this, and it seemed like a good idea to me. I guess I
missed the fact that the patch wasn't doing it that way. Hmm.FWIW, I kind of agree with Robert's take on this. Heap and index AMs
are pretty fundamentally different animals, yet we don't have a problem
sticking them in the same catalog. I think anything that is related to
storage access could reasonably go into that catalog, rather than
inventing a new one.Right -- Something like amname=lz4, amhandler=lz4handler, amtype=c.
The core code must of course know how to instantiate an AM of type
'c' and what to use it for.
I have changed this, I agree that using the access method for creating
compression has simplified the code. I will share the updated patch
set after fixing other review comments by Robert.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
While working on this comment I have doubts.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression? For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example).
Yeah for the toast table it doesn't matter, but I am not sure what do
you mean by altering the compression method for the toast table. Do you
mean manually update the pg_attribute tuple for the toast table and
set different compression methods? Or there is some direct way to
alter the toast table?
For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.
I see that we allow setting the STORAGE for the materialized view but
I am not sure what is the use case. Basically, the tuples are
directly getting selected from the host table and inserted in the
materialized view without checking target and source storage type.
The behavior is the same if you execute INSERT INTO dest_table SELECT
* FROM source_table. Basically, if the source_table attribute has
extended storage and the target table has plain storage, still the
value will be inserted directly into the target table without any
conversion. However, in the table, you can insert the new tuple and
that will be stored as per the new storage method so that is still
fine but I don't know any use case for the materialized view. Now I am
thinking what should be the behavior for the materialized view?
For the materialized view can we have the same behavior as storage? I
think for the built-in compression method that might not be a problem
but for the external compression method how can we handle the
dependency, I mean when the materialized view has created the table
was having an external compression method "cm1" and we have created
the materialized view based on that now if we alter table and set the
new compression method and force table rewrite then what will happen
to the tuple inside the materialized view, I mean tuple is still
compressed with "cm1" and there is no attribute is maintaining the
dependency on "cm1" because the materialized view can point to any
compression method. Now if we drop the cm1 it will be allowed to
drop. So I think for the compression method we can consider the
materialized view same as the table, I mean we can allow setting the
compression method for the materialized view and we can always ensure
that all the tuple in this view is compressed with the current or the
preserved compression methods. So whenever we are inserting in the
materialized view then we should compare the datum compression method
with the target compression method.
+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
I don't see a particularly good reason to treat plain and external
differently.
Yeah, I think they should be treated the same.
More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure).
I agree.
However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.
IIUC, even if we change the attstorage the existing tuples are stored
as it is without changing the tuple storage. So I think even if the
attstorage is changed the attcompression should not have any change.
After observing this behavior of storage I tend to think that for
built-in compression methods also we should have the same behavior, I mean
if the tuple is compressed with one of the built-in compression
methods and if we are altering the compression method or we are doing
INSERT INTO SELECT to the target field having a different compression
method then we should not rewrite/decompress those tuples. Basically,
I mean to say that the built-in compression methods can always be
treated as PRESERVE because those can not be dropped.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Dec 1, 2020 at 4:50 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
While working on this comment I have doubts.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression? For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example).Yeah for the toast table it doesn't matter, but I am not sure what do
you mean by altering the compression method for the toast table. Do you
mean manually update the pg_attribute tuple for the toast table and
set different compression methods? Or there is some direct way to
alter the toast table?For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.I see that we allow setting the STORAGE for the materialized view but
I am not sure what is the use case. Basically, the tuples are
directly getting selected from the host table and inserted in the
materialized view without checking target and source storage type.
The behavior is the same if you execute INSERT INTO dest_table SELECT
* FROM source_table. Basically, if the source_table attribute has
extended storage and the target table has plain storage, still the
value will be inserted directly into the target table without any
conversion. However, in the table, you can insert the new tuple and
that will be stored as per the new storage method so that is still
fine but I don't know any use case for the materialized view. Now I am
thinking what should be the behavior for the materialized view?For the materialized view can we have the same behavior as storage? I
think for the built-in compression method that might not be a problem
but for the external compression method how can we handle the
dependency, I mean when the materialized view has created the table
was having an external compression method "cm1" and we have created
the materialized view based on that now if we alter table and set the
new compression method and force table rewrite then what will happen
to the tuple inside the materialized view, I mean tuple is still
compressed with "cm1" and there is no attribute is maintaining the
dependency on "cm1" because the materialized view can point to any
compression method. Now if we drop the cm1 it will be allowed to
drop. So I think for the compression method we can consider the
materialized view same as the table, I mean we can allow setting the
compression method for the materialized view and we can always ensure
that all the tuple in this view is compressed with the current or the
preserved compression methods. So whenever we are inserting in the
materialized view then we should compare the datum compression method
with the target compression method.+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
I don't see a particularly good reason to treat plain and external
differently.Yeah, I think they should be treated the same.
More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure).I agree.
However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.IIUC, even if we change the attstorage the existing tuples are stored
as it is without changing the tuple storage. So I think even if the
attstorage is changed the attcompression should not have any change.
I have put some more thought into this and IMHO the rules should be as below
1. If attstorage is EXTENDED -> attcompression "must be valid"
2. if attstorage is PLAIN/EXTERNAL -> atttcompression "maybe valid"
3. if typstorage is PLAIN/EXTERNAL -> atttcompression "must be invalid"
I am a little bit confused about (2), basically, it will be valid in
the scenario u mentioned that change the atttstorege from EXTENDED to
PLAIN/EXTERNAL. But I think in this case also we can just set the
attcompression to invalid, however, we have to maintain the dependency
between attribute and compression method so that the old methods using
which we might have compressed a few tuples in the table doesn't get
dropped.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Dec 1, 2020 at 9:15 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Tue, Dec 1, 2020 at 4:50 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
While working on this comment I have doubts.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression? For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example).Yeah for the toast table it doesn't matter, but I am not sure what do
you mean by altering the compression method for the toast table. Do you
mean manually update the pg_attribute tuple for the toast table and
set different compression methods? Or there is some direct way to
alter the toast table?For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.I see that we allow setting the STORAGE for the materialized view but
I am not sure what is the use case. Basically, the tuples are
directly getting selected from the host table and inserted in the
materialized view without checking target and source storage type.
The behavior is the same if you execute INSERT INTO dest_table SELECT
* FROM source_table. Basically, if the source_table attribute has
extended storage and the target table has plain storage, still the
value will be inserted directly into the target table without any
conversion. However, in the table, you can insert the new tuple and
that will be stored as per the new storage method so that is still
fine but I don't know any use case for the materialized view. Now I am
thinking what should be the behavior for the materialized view?For the materialized view can we have the same behavior as storage? I
think for the built-in compression method that might not be a problem
but for the external compression method how can we handle the
dependency, I mean when the materialized view has created the table
was having an external compression method "cm1" and we have created
the materialized view based on that now if we alter table and set the
new compression method and force table rewrite then what will happen
to the tuple inside the materialized view, I mean tuple is still
compressed with "cm1" and there is no attribute is maintaining the
dependency on "cm1" because the materialized view can point to any
compression method. Now if we drop the cm1 it will be allowed to
drop. So I think for the compression method we can consider the
materialized view same as the table, I mean we can allow setting the
compression method for the materialized view and we can always ensure
that all the tuple in this view is compressed with the current or the
preserved compression methods. So whenever we are inserting in the
materialized view then we should compare the datum compression method
with the target compression method.
As per the offlist discussion with Robert, for materialized/table we
will always compress the value as per the target attribute compression
method. So if we are creating/refreshing the materialized view and
the attcompression for the target attribute is different than the
source table then we will decompress it and then compress it back as
per the target table/view.
+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
I don't see a particularly good reason to treat plain and external
differently.Yeah, I think they should be treated the same.
More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure).I agree.
However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.IIUC, even if we change the attstorage the existing tuples are stored
as it is without changing the tuple storage. So I think even if the
attstorage is changed the attcompression should not have any change.I have put some more thought into this and IMHO the rules should be as below
1. If attstorage is EXTENDED -> attcompression "must be valid"
2. if attstorage is PLAIN/EXTERNAL -> atttcompression "maybe valid"
3. if typstorage is PLAIN/EXTERNAL -> atttcompression "must be invalid"I am a little bit confused about (2), basically, it will be valid in
the scenario u mentioned that change the atttstorege from EXTENDED to
PLAIN/EXTERNAL. But I think in this case also we can just set the
attcompression to invalid, however, we have to maintain the dependency
between attribute and compression method so that the old methods using
which we might have compressed a few tuples in the table doesn't get
dropped.
For this also I had an offlist discussion with Robert and we decided
that it make sense to always have a valid compression method stored in
the attribute if the attribute type is compressible irrespective of
what is the current attribute storage. For example, if the attribute
type is varchar then it will always have a valid compression method,
it does not matter even if the att storage is plain or external.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.Since the authorship history of this patch is complicated, it would be
nice if you would include authorship information and relevant
"Discussion" links in the patches.
I have added that.
Design level considerations and overall notes:
configure is autogenerated from configure.in, so the patch shouldn't
include changes only to the former.
Yeah, I missed those changes. Done now.
Looking over the changes to src/include:
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_IDI think that it would be good to assign values to these explicitly.
Done
+/* compresion handler routines */
Spelling.
Done
+ /* compression routine for the compression method */ + cmcompress_function cmcompress; + + /* decompression routine for the compression method */ + cmcompress_function cmdecompress;Don't reuse cmcompress_function; that's confusing. Just have a typedef
per structure member, even if they end up being the same.
Fixed as suggested
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \ - (((toast_compress_header *) (ptr))->rawsize = (len)) +do { \ + Assert(len > 0 && len <= RAWSIZEMASK); \ + ((toast_compress_header *) (ptr))->info = (len); \ +} while (0)Indentation.
Done
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \ + ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);What about making TOAST_COMPRESS_SET_RAWSIZE() take another argument?
And possibly also rename it to TEST_COMPRESS_SET_SIZE_AND_METHOD() or
something? It seems not great to have separate functions each setting
part of a 4-byte quantity. Too much chance of failing to set both
parts. I guess you've got a function called
toast_set_compressed_datum_info() for that, but it's just a wrapper
around two macros that could just be combined, which would reduce
complexity overall.
Done that way
+ T_CompressionRoutine, /* in access/compressionapi.h */
This looks misplaced. I guess it should go just after these:
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
Done
Looking over the regression test changes:
The tests at the top of create_cm.out that just test that we can
create tables with various storage types seem unrelated to the purpose
of the patch. And the file doesn't test creating a compression method
either, as the file name would suggest, so either the file name needs
to be changed (compression, compression_method?) or the tests don't go
here.
Changed to "compression"
+-- check data is okdd
I guess whoever is responsible for this comment prefers vi to emacs.
Fixed
I don't quite understand the purpose of all of these tests, and there
are some things that I feel like ought to be tested that seemingly
aren't. Like, you seem to test using an UPDATE to move a datum from a
table to another table with the same compression method, but not one
with a different compression method.
Added test for this, and some other tests to improve overall coverage.
Testing the former is nice and
everything, but that's the easy case: I think we also need to test the
latter. I think it would be good to verify not only that the data is
readable but that it's compressed the way we expect. I think it would
be a great idea to add a pg_column_compression() function in a similar
spirit to pg_column_size(). Perhaps it could return NULL when
compression is not in use or the data type is not varlena, and the
name of the compression method otherwise. That would allow for better
testing of this feature, and it would also be useful to users who are
switching methods, to see what data they still have that's using the
old method. It could be useful for debugging problems on customer
systems, too.
This is a really great idea, I have added this function and used in my test.
I wonder if we need a test that moves data between tables through an
intermediary. For instance, suppose a plpgsql function or DO block
fetches some data and stores it in a plpgsql variable and then uses
the variable to insert into another table. Hmm, maybe that would force
de-TOASTing. But perhaps there are other cases. Maybe a more general
way to approach the problem is: have you tried running a coverage
report and checked which parts of your code are getting exercised by
the existing tests and which parts are not? The stuff that isn't, we
should try to add more tests. It's easy to get corner cases wrong with
this kind of thing.I notice that LIKE INCLUDING COMPRESSION doesn't seem to be tested, at
least not by 0001, which reinforces my feeling that the tests here are
not as thorough as they could be.
Added test for this as well.
+NOTICE: pg_compression contains unpinned initdb-created object(s)
This seems wrong to me - why is it OK?
Yeah, this is wrong, now fixed.
- result = (struct varlena *) - palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); - SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr), - TOAST_COMPRESS_SIZE(attr), - VARDATA(result), - TOAST_COMPRESS_RAWSIZE(attr), true) < 0) - elog(ERROR, "compressed data is corrupted"); + /* get compression method handler routines */ + cmroutine = GetCompressionRoutine(cmoid);- return result; + return cmroutine->cmdecompress(attr);I'm worried about how expensive this might be, and I think we could
make it cheaper. The reason why I think this might be expensive is:
currently, for every datum, you have a single direct function call.
Now, with this, you first have a direct function call to
GetCompressionOidFromCompressionId(). Then you have a call to
GetCompressionRoutine(), which does a syscache lookup and calls a
handler function, which is quite a lot more expensive than a single
function call. And the handler isn't even returning a statically
allocated structure, but is allocating new memory every time, which
involves more function calls and maybe memory leaks. Then you use the
results of all that to make an indirect function call.I'm not sure exactly what combination of things we could use to make
this better, but it seems like there are a few possibilities:(1) The handler function could return a pointer to the same
CompressionRoutine every time instead of constructing a new one every
time.
(2) The CompressionRoutine to which the handler function returns a
pointer could be statically allocated instead of being built at
runtime.
(3) GetCompressionRoutine could have an OID -> handler cache instead
of relying on syscache + calling the handler function all over again.
(4) For the compression types that have dedicated bit patterns in the
high bits of the compressed TOAST size, toast_compress_datum() could
just have hard-coded logic to use the correct handlers instead of
translating the bit pattern into an OID and then looking it up over
again.
(5) Going even further than #4 we could skip the handler layer
entirely for such methods, and just call the right function directly.
I think we should definitely do (1), and also (2) unless there's some
reason it's hard. (3) doesn't need to be part of this patch, but might
be something to consider later in the series. It's possible that it
doesn't have enough benefit to be worth the work, though. Also, I
think we should do either (4) or (5). I have a mild preference for (5)
unless it looks too ugly.
Note that I'm not talking about hard-coding a fast path for a
hard-coded list of OIDs - which would seem a little bit unprincipled -
but hard-coding a fast path for the bit patterns that are themselves
hard-coded. I don't think we lose anything in terms of extensibility
or even-handedness there; it's just avoiding a bunch of rigamarole
that doesn't really buy us anything.All these points apply equally to toast_decompress_datum_slice() and
toast_compress_datum().
Fixed as discussed at [1]/messages/by-id/CA+Tgmob3W8cnLgOQX+JQzeyGN3eKGmRrBkUY6WGfNyHa+t_qEw@mail.gmail.com
+ /* Fallback to default compression method, if not specified */ + if (!OidIsValid(cmoid)) + cmoid = DefaultCompressionOid;I think that the caller should be required to specify a legal value,
and this should be an elog(ERROR) or an Assert().The change to equalTupleDescs() makes me wonder. Like, can we specify
the compression method for a function parameter, or a function return
value? I would think not. But then how are the tuple descriptors set
up in that case? Under what circumstances do we actually need the
tuple descriptors to compare unequal?
If we alter the compression method then we check whether we need to
rebuild the tuple descriptor or not based on what value is changed so
if the attribute compression method is changed we need to rebuild the
compression method right. You might say that in the first patch we
are not allowing altering the compression method so we might move this
to the second patch but I thought since we added this field to
pg_attribute in this patch then better to add this check as well.
What am I missing?
lz4.c's header comment calls it cm_lz4.c, and the pathname is wrong too.
I wonder if we should try to adopt a convention for the names of these
files that isn't just the compression method name, like cmlz4 or
compress_lz4. I kind of like the latter one. I am a little worried
that just calling it lz4.c will result in name collisions later - not
in this directory, of course, but elsewhere in the system. It's not a
disaster if that happens, but for example verbose error reports print
the file name, so it's nice if it's unambiguous.
Changed to compress_lz4.
+ if (!IsBinaryUpgrade && + (relkind == RELKIND_RELATION || + relkind == RELKIND_PARTITIONED_TABLE)) + attr->attcompression = + GetAttributeCompressionMethod(attr, colDef->compression); + else + attr->attcompression = InvalidOid;Storing InvalidOid in the IsBinaryUpgrade case looks wrong. If
upgrading from pre-v14, we need to store PGLZ_COMPRESSION_OID.
Otherwise, we need to preserve whatever value was present in the old
version. Or am I confused here?
Okay, so I think we can simply remove the IsBinaryUpgrade check so it
will behave as expected. Basically, now it the compression method is
specified then it will take that compression method and if it is not
specified then it will take the PGLZ_COMPRESSION_OID.
I think there should be tests for the way this interacts with
partitioning, and I think the intended interaction should be
documented. Perhaps it should behave like TABLESPACE, where the parent
property has no effect on what gets stored because the parent has no
storage, but is inherited by each new child.
I have added the test for this and also documented the same.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression?
I have changed this to store the Invalid compression method always.
For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example).
You mean to update the pg_attribute table for the toasted field (e.g
chunk_data) and set the attcompression to something valid? Or there
is a better way to write this test?
For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.
Fixed as described as [2]/messages/by-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W+LE+3bqWPJUZj4GnHOJg@mail.gmail.com
+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
Fixed
I don't see a particularly good reason to treat plain and external
differently. More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure). However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.I'm out of time for today, so I'll have to look at this more another
day. Hope this helps for a start.
Fixed as I have described at [2]/messages/by-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W+LE+3bqWPJUZj4GnHOJg@mail.gmail.com, and the rules are documented in
pg_attribute.h (atop attcompression field)
[1]: /messages/by-id/CA+Tgmob3W8cnLgOQX+JQzeyGN3eKGmRrBkUY6WGfNyHa+t_qEw@mail.gmail.com
[2]: /messages/by-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W+LE+3bqWPJUZj4GnHOJg@mail.gmail.com
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v15-0002-alter-table-set-compression.patchapplication/octet-stream; name=v15-0002-alter-table-set-compression.patchDownload
From bb76a217a95dbf8e36c4a1f61152ca3acbe402ea Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Sun, 6 Dec 2020 17:53:43 +0530
Subject: [PATCH v15 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 +++++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/sql/compression.sql | 21 +++
8 files changed, 272 insertions(+), 46 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..bfa5c36fb7 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be set
+ from available built-in compression methods. The available built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b6c2c02824..7f619d1d59 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3861,6 +3863,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4388,7 +4391,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4796,6 +4800,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5420,6 +5428,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7660,6 +7671,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7675,7 +7751,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7739,47 +7814,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15000,6 +15036,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 39467aa2e0..9683062216 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2257,6 +2257,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 3a43c09bf6..1da16ebc0a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,7 +2093,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 78b7f76215..af79aa6534 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1856,7 +1856,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 83f1fb4d44..fbbb6bf9bc 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -133,6 +133,65 @@ SELECT pg_column_compression(f1) FROM cmpart;
pglz
(2 rows)
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 742d095c39..f889630d62 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -61,6 +61,27 @@ INSERT INTO cmpart VALUES (repeat('123456789',1004));
INSERT INTO cmpart VALUES (repeat('123456789',4004));
SELECT pg_column_compression(f1) FROM cmpart;
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v15-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v15-0005-new-compression-method-extension-for-zlib.patchDownload
From 6fa37dac3da12010b656a4eed31ce4aeb0ac03aa Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v15 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..bf36854ef3
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ lz4
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..aa7c57030d
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.23.0
v15-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v15-0003-Add-support-for-PRESERVE.patchDownload
From e21010ccc7073114816ad73e70d38960d370c110 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 7 Dec 2020 11:55:39 +0530
Subject: [PATCH v15 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/objectaddress.c | 1 +
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 231 +++++++++++++++++++++
src/backend/commands/tablecmds.c | 117 ++++++-----
src/backend/executor/nodeModifyTable.c | 10 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 +++-
src/test/regress/expected/create_index.out | 56 ++---
src/test/regress/sql/compression.sql | 9 +
19 files changed, 517 insertions(+), 95 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index bfa5c36fb7..2fb9cfaadc 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be set
from available built-in compression methods. The available built-in
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ methods are <literal>pglz</literal> and <literal>lz4</literal>. The
+ PRESERVE list contains list of compression methods used on the column and
+ determines which of them should be kept on the column. Without PRESERVE or
+ if all the previous compression methods are not preserved then the table
+ will be rewritten. If PRESERVE ALL is specified then all the previous
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index a5eccdffd0..636364995a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 25290c821f..b198bc9b00 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..91eb59ca17
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,231 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7f619d1d59..91a341fc3e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -391,6 +391,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -531,7 +532,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -563,7 +566,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -588,6 +590,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -866,7 +869,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +939,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2414,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2460,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2711,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4801,7 +4820,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6295,7 +6315,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6470,6 +6491,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6617,6 +6639,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11752,7 +11796,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11867,6 +11912,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15045,24 +15095,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15095,11 +15142,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17824,32 +17876,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 3e62668ffc..8f952f29f7 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1933,6 +1934,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1963,10 +1965,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * compare with the compression method of the target attribute. If
+ * the target compression method is not same then we need to
+ * decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ffb8e1a58c..2ac6d61320 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2931,7 +2931,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2951,6 +2951,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5622,6 +5634,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index cf86bdd428..51dee9e12f 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2585,7 +2585,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2605,6 +2605,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3677,6 +3687,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d2709df744..f458acae6d 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2852,7 +2852,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2870,6 +2870,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4214,6 +4224,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9683062216..22bd0e87a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -593,7 +593,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2258,12 +2260,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3386,7 +3388,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3441,13 +3443,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e5184933b7..4bbbf6370a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1da16ebc0a..8178a5124b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,6 +2098,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 2d76a85336..d1880dfc5e 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index c8d1f20b3b..885ff5ec5c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -478,6 +478,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index af79aa6534..c8135debbb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index fbbb6bf9bc..457e8bc4be 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -192,12 +192,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index fc6afab58a..d8300bdcdd 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index f889630d62..46da711c12 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -82,6 +82,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v15-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v15-0004-Create-custom-compression-methods.patchDownload
From fdf91141a0f26e51042eb608f4ba94ffba410e66 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 7 Dec 2020 15:26:57 +0530
Subject: [PATCH v15 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
doc/src/sgml/ref/alter_table.sgml | 17 +++---
doc/src/sgml/ref/create_access_method.sgml | 13 ++--
doc/src/sgml/ref/create_table.sgml | 11 ++--
src/backend/access/common/detoast.c | 59 +++++++++++++++++--
src/backend/access/common/toast_internals.c | 19 +++++-
src/backend/access/compression/compress_lz4.c | 21 +++----
.../access/compression/compress_pglz.c | 20 +++----
.../access/compression/compressamapi.c | 47 +++++++++++++++
src/backend/commands/amcmds.c | 5 ++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 +
src/bin/psql/tab-complete.c | 6 ++
src/include/access/compressamapi.h | 16 +++--
src/include/access/detoast.h | 8 +++
src/include/access/toast_internals.h | 15 +++++
src/test/regress/expected/compression.out | 41 ++++++++++++-
src/test/regress/sql/compression.sql | 11 ++++
19 files changed, 264 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 2fb9cfaadc..9efbc2352c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,14 +390,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. Compression method can be set
- from available built-in compression methods. The available built-in
- methods are <literal>pglz</literal> and <literal>lz4</literal>. The
- PRESERVE list contains list of compression methods used on the column and
- determines which of them should be kept on the column. Without PRESERVE or
- if all the previous compression methods are not preserved then the table
- will be rewritten. If PRESERVE ALL is specified then all the previous
- methods will be preserved and the table will not be rewritten.
+ This clause adds compression to a column. Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or it can
+ be set from the available built-in compression methods. The available
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ The PRESERVE list contains list of compression methods used on the column
+ and determines which of them should be kept on the column. Without
+ PRESERVE or if all the previous compression methods are not preserved then
+ the table will be rewritten. If PRESERVE ALL is specified then all the
+ previous methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index f404dd1088..ade3989d75 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -999,11 +999,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. Compression method
- can be set from the available built-in compression methods. The available
- options are <literal>pglz</literal> and <literal>lz4</literal>. If the
- compression method is not sepcified for the compressible type then it will
- have the default compression method. The default compression method is
- <literal>pglz</literal>.
+ could be created with <xref linkend="sql-create-access-method"/> or it can
+ be set from the available built-in compression methods. The available
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If the compression method is not sepcified for the compressible type then
+ it will have the default compression method. The default compression
+ method is <literal>pglz</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d66d733da6..f6c48ac955 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,13 +438,44 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -458,10 +489,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -477,9 +519,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -493,16 +537,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 72bf23f712..cd91aefef7 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index b455367be3..ce27616116 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -44,10 +44,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -55,7 +55,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -67,7 +67,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -80,9 +80,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -99,7 +99,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -111,9 +112,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 2a3ef17842..d693c880fd 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c8d2..9d96fad31f 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -59,3 +60,49 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ HeapTuple tuple;
+ Form_pg_am amform;
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for access method %u",
+ amoid);
+
+ amform = (Form_pg_am)GETSTRUCT(tuple);
+
+ /* Check if it's an index access method as opposed to some other AM */
+ if (amform->amtype != AMTYPE_COMPRESSION)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname), "INDEX")));
+
+ amhandler = amform->amhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("index access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ return routine;
+}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index fa8deda08b..bdd6d0a8ac 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -227,6 +227,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -264,6 +266,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 8f952f29f7..8fc6d27bbe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1969,7 +1969,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* the target compression method is not same then we need to
* decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 22bd0e87a3..54ff6fb937 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5254,6 +5254,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 65a6ef4ebe..6e8cfd2d9b 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5284,7 +5284,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5314,10 +5313,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dda8056ebf..75390d9239 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -12939,6 +12939,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8178a5124b..9bfa66ff09 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0596..913a633d83 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -26,18 +26,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -61,5 +68,6 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index e13e34cac3..81944912d2 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 457e8bc4be..2c336efaed 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -226,13 +226,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
@@ -261,3 +299,4 @@ SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP ACCESS METHOD pglz2;
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 46da711c12..b57f3d29ba 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -91,6 +91,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
@@ -100,3 +110,4 @@ SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP ACCESS METHOD pglz2;
--
2.23.0
v15-0001-Built-in-compression-method.patchapplication/octet-stream; name=v15-0001-Built-in-compression-method.patchDownload
From 4ba5464a3de7f63165cc35d2f17e01c7cbcd9310 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v15 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 2 +
doc/src/sgml/ref/create_table.sgml | 27 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++
.../access/compression/compress_pglz.c | 131 +++++++++
.../access/compression/compressamapi.c | 61 +++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 ++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 ++-
src/include/access/compressamapi.h | 65 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 169 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 72 +++++
src/tools/pgindent/typedefs.list | 1 +
77 files changed, 1976 insertions(+), 667 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index 2a03ed0a01..b88c7d5b29 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -866,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8601,6 +8604,35 @@ fi
+#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12092,6 +12124,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13295,6 +13380,20 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 466aa51dd6..71ff2cd16b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -999,6 +999,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1186,6 +1193,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1400,6 +1415,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index c12a32c8c7..5ec3d995f8 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,8 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ Partitions inherits the compression method of the parent for each column
+ however we can set different compression method for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..f404dd1088 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be coppied. The default
+ behavior is to exclude compression method, resulting in the copied
+ column will have the default compression method if the column type is
+ compressible.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +994,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method is not sepcified for the compressible type then it will
+ have the default compression method. The default compression method is
+ <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 17e50de530..2b8a6de6ed 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..d66d733da6 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -438,28 +439,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -473,22 +493,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..3024670414 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..72bf23f712 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9aac155e4a 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..c4814ef911
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..b455367be3
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+#endif
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..2a3ef17842
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000000..663102c8d2
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index a7ed93fdc1..8162ecd612 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 66fdaf67b1..94278786d9 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 4cd7d76938..c2189ef0ef 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1704,6 +1706,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 731610c701..7e6310b6f7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index f1850436bd..e5452614f7 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 6f05ee715b..fa8deda08b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 6bf6c5a310..f7c9ff0a3e 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -558,6 +558,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index cfc63915f3..2e1d990f9e 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -486,6 +486,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 46f1637e77..b6c2c02824 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -560,7 +561,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -854,6 +855,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2396,6 +2409,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2458,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2704,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6234,6 +6276,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11753,6 +11807,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17632,3 +17702,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index e0f24283b8..3e62668ffc 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1915,6 +1918,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2120,6 +2195,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 910906f639..ffb8e1a58c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2931,6 +2931,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 687609f59e..cf86bdd428 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2585,6 +2585,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 1dc873ed25..dc28492050 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3857,6 +3857,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8f5e4e71b2..d2709df744 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2852,6 +2852,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8f341ac006..39467aa2e0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -593,6 +593,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -628,9 +630,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3370,11 +3372,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3383,8 +3386,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3429,6 +3432,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15110,6 +15122,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15628,6 +15641,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 89ee990599..e5184933b7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 15dc51a94d..1facd7b9ba 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 3d6b2f9093..e87fc746fe 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -345,6 +345,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index ff9bf238f3..65a6ef4ebe 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "common/unicode_norm.h"
@@ -5274,6 +5276,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3b36335aa6..dda8056ebf 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -385,6 +385,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8545,6 +8546,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8630,6 +8632,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8648,7 +8659,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8675,6 +8691,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8703,6 +8720,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15700,6 +15718,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15724,6 +15743,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15759,6 +15781,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 317bb83970..2912e4c0dc 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -325,6 +325,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 14150d05a9..724000c48d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..8226ae0596
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..4e932bc067 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..e13e34cac3 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 0f051277a6..3a8d0ac09c 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index bc73e88a1a..0ffc844995 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index cdf75a2380..3e6ed706ac 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For the compressible atttypid this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible atttypid this must always be an invalid Oid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index fc2202b843..06dca7f583 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7047,6 +7056,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2228',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7217,6 +7230,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 21a467a7a7..9726ab159a 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -578,6 +578,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1133ae1143..2d76a85336 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..df439c8c38 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 3684f87a88..c8d1f20b3b 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -509,6 +509,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 48a79a7657..78b7f76215 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index de8f838e53..f894df3504 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..83f1fb4d44
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,169 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+DROP MATERIALIZED VIEW mv;
+DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 2c0760404d..e04e98ff85 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6293ab57bc..b79b0a2a7f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3019,11 +3019,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3039,11 +3039,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3070,11 +3070,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 7bfeaf85f0..197c312ed6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index ae89ed7f0b..e8aafd8d05 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,6 +114,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 525bdc804f..c38ba8a5d5 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -202,3 +202,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..742d095c39
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,72 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+DROP MATERIALIZED VIEW mv;
+DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index cf63acbf6f..e9845b66e6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -394,6 +394,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
v15-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v15-0006-Support-compression-methods-options.patchDownload
From ccf8862fcaed690752a14bc76288bd523dc7a2d9 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Dec 2020 16:46:53 +0530
Subject: [PATCH v15 6/6] Support compression methods options
---
contrib/cmzlib/cmzlib.c | 75 ++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 42 ++++++
src/backend/access/common/toast_internals.c | 14 +-
src/backend/access/compression/compress_lz4.c | 69 ++++++++-
.../access/compression/compress_pglz.c | 98 ++++++++++++-
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +-
src/backend/catalog/index.c | 43 +++++-
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 48 ++++++-
src/backend/commands/foreigncmds.c | 44 ------
src/backend/commands/tablecmds.c | 133 ++++++++++++++++--
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/include/access/compressamapi.h | 19 ++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 4 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 21 ++-
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 11 ++
32 files changed, 594 insertions(+), 108 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..cb5aa439e3 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 9efbc2352c..87e1a1acf3 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,7 +386,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
@@ -394,6 +394,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
could be created with <xref linkend="sql-create-access-method"/> or it can
be set from the available built-in compression methods. The available
built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If compression method has options they could be specified with
+ <literal>WITH</literal> parameter.
The PRESERVE list contains list of compression methods used on the column
and determines which of them should be kept on the column. Without
PRESERVE or if all the previous compression methods are not preserved then
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index ade3989d75..95e03f6379 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -995,7 +995,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
@@ -1004,7 +1004,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not sepcified for the compressible type then
it will have the default compression method. The default compression
- method is <literal>pglz</literal>.
+ method is <literal>pglz</literal>. If the compression method has options
+ they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 2b8a6de6ed..61fc6fbb30 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 3024670414..0cd7bdd730 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..3185082034 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,45 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index cd91aefef7..e9b5971928 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index ce27616116..d1ee63c8e9 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,12 +16,70 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
#ifdef HAVE_LIBLZ4
#include "lz4.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -29,7 +87,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -39,6 +97,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -46,9 +105,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d693c880fd..7cc5a22e37 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..7e399daaa1 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index c2189ef0ef..708a484880 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1150,6 +1158,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1407,7 +1416,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7e6310b6f7..b878b4af7f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index e5452614f7..6cf32fef6a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index fd5a6eec86..29a8145b1d 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -697,6 +697,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 91eb59ca17..934345be1f 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -27,6 +27,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* get list of all supported compression methods for the given attribute.
@@ -127,7 +128,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -153,6 +154,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -229,3 +244,34 @@ MakeColumnCompression(Oid attcompression)
return node;
}
+
+/*
+ * Fetch atttributes compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 91a341fc3e..2df9e52c49 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -611,6 +611,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -815,6 +816,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -868,8 +871,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -923,8 +927,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -6153,6 +6160,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6316,6 +6324,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6326,7 +6335,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7723,13 +7732,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option must be only valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7768,7 +7784,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7859,7 +7897,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15105,9 +15143,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15142,7 +15182,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15152,8 +15193,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15161,8 +15211,67 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+#if 0
+ foreach (lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ atttableform->attcompression = cmoid;
+
+ /* update existing entry */
+ if (acoptions)
+ {
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = PointerGetDatum(acoptions);
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+#endif
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 2ac6d61320..2436abd467 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2958,6 +2958,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 51dee9e12f..ae6cfb0266 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2610,6 +2610,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f458acae6d..323bccd345 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2877,6 +2877,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 54ff6fb937..65ea7a4d39 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -413,6 +413,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3452,11 +3453,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3464,14 +3471,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 913a633d83..8123cc8cc7 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -37,8 +38,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -49,14 +53,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 4e932bc067..1dc4aa9623 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 81944912d2..dc35064086 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index f94defff3c..bd192f457e 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3e6ed706ac..55ea9af6bd 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index d1880dfc5e..b593103367 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -138,6 +138,7 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,9 +151,10 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
+ Datum *acoptions, bool *need_rewrite);
extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c8135debbb..09e5e00d99 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 2c336efaed..0f977c5e92 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -263,6 +263,23 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata(f1);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1000));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1008));
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
@@ -282,7 +299,9 @@ SELECT length(f1) FROM cmmove1;
length
--------
10000
-(1 row)
+ 10040
+ 10080
+(3 rows)
SELECT length(f1) FROM cmmove2;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..27aab82462 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b57f3d29ba..1e47af121b 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -101,6 +101,17 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata(f1);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1000));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1008));
+SELECT pg_column_compression(f1) FROM cmmove1;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Wed, Dec 9, 2020 at 5:37 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.Since the authorship history of this patch is complicated, it would be
nice if you would include authorship information and relevant
"Discussion" links in the patches.I have added that.
Design level considerations and overall notes:
configure is autogenerated from configure.in, so the patch shouldn't
include changes only to the former.Yeah, I missed those changes. Done now.
Looking over the changes to src/include:
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_IDI think that it would be good to assign values to these explicitly.
Done
+/* compresion handler routines */
Spelling.
Done
+ /* compression routine for the compression method */ + cmcompress_function cmcompress; + + /* decompression routine for the compression method */ + cmcompress_function cmdecompress;Don't reuse cmcompress_function; that's confusing. Just have a typedef
per structure member, even if they end up being the same.Fixed as suggested
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \ - (((toast_compress_header *) (ptr))->rawsize = (len)) +do { \ + Assert(len > 0 && len <= RAWSIZEMASK); \ + ((toast_compress_header *) (ptr))->info = (len); \ +} while (0)Indentation.
Done
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \ + ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);What about making TOAST_COMPRESS_SET_RAWSIZE() take another argument?
And possibly also rename it to TEST_COMPRESS_SET_SIZE_AND_METHOD() or
something? It seems not great to have separate functions each setting
part of a 4-byte quantity. Too much chance of failing to set both
parts. I guess you've got a function called
toast_set_compressed_datum_info() for that, but it's just a wrapper
around two macros that could just be combined, which would reduce
complexity overall.Done that way
+ T_CompressionRoutine, /* in access/compressionapi.h */
This looks misplaced. I guess it should go just after these:
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */Done
Looking over the regression test changes:
The tests at the top of create_cm.out that just test that we can
create tables with various storage types seem unrelated to the purpose
of the patch. And the file doesn't test creating a compression method
either, as the file name would suggest, so either the file name needs
to be changed (compression, compression_method?) or the tests don't go
here.Changed to "compression"
+-- check data is okdd
I guess whoever is responsible for this comment prefers vi to emacs.
Fixed
I don't quite understand the purpose of all of these tests, and there
are some things that I feel like ought to be tested that seemingly
aren't. Like, you seem to test using an UPDATE to move a datum from a
table to another table with the same compression method, but not one
with a different compression method.Added test for this, and some other tests to improve overall coverage.
Testing the former is nice and
everything, but that's the easy case: I think we also need to test the
latter. I think it would be good to verify not only that the data is
readable but that it's compressed the way we expect. I think it would
be a great idea to add a pg_column_compression() function in a similar
spirit to pg_column_size(). Perhaps it could return NULL when
compression is not in use or the data type is not varlena, and the
name of the compression method otherwise. That would allow for better
testing of this feature, and it would also be useful to users who are
switching methods, to see what data they still have that's using the
old method. It could be useful for debugging problems on customer
systems, too.This is a really great idea, I have added this function and used in my test.
I wonder if we need a test that moves data between tables through an
intermediary. For instance, suppose a plpgsql function or DO block
fetches some data and stores it in a plpgsql variable and then uses
the variable to insert into another table. Hmm, maybe that would force
de-TOASTing. But perhaps there are other cases. Maybe a more general
way to approach the problem is: have you tried running a coverage
report and checked which parts of your code are getting exercised by
the existing tests and which parts are not? The stuff that isn't, we
should try to add more tests. It's easy to get corner cases wrong with
this kind of thing.I notice that LIKE INCLUDING COMPRESSION doesn't seem to be tested, at
least not by 0001, which reinforces my feeling that the tests here are
not as thorough as they could be.Added test for this as well.
+NOTICE: pg_compression contains unpinned initdb-created object(s)
This seems wrong to me - why is it OK?
Yeah, this is wrong, now fixed.
- result = (struct varlena *) - palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); - SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr), - TOAST_COMPRESS_SIZE(attr), - VARDATA(result), - TOAST_COMPRESS_RAWSIZE(attr), true) < 0) - elog(ERROR, "compressed data is corrupted"); + /* get compression method handler routines */ + cmroutine = GetCompressionRoutine(cmoid);- return result; + return cmroutine->cmdecompress(attr);I'm worried about how expensive this might be, and I think we could
make it cheaper. The reason why I think this might be expensive is:
currently, for every datum, you have a single direct function call.
Now, with this, you first have a direct function call to
GetCompressionOidFromCompressionId(). Then you have a call to
GetCompressionRoutine(), which does a syscache lookup and calls a
handler function, which is quite a lot more expensive than a single
function call. And the handler isn't even returning a statically
allocated structure, but is allocating new memory every time, which
involves more function calls and maybe memory leaks. Then you use the
results of all that to make an indirect function call.I'm not sure exactly what combination of things we could use to make
this better, but it seems like there are a few possibilities:(1) The handler function could return a pointer to the same
CompressionRoutine every time instead of constructing a new one every
time.
(2) The CompressionRoutine to which the handler function returns a
pointer could be statically allocated instead of being built at
runtime.
(3) GetCompressionRoutine could have an OID -> handler cache instead
of relying on syscache + calling the handler function all over again.
(4) For the compression types that have dedicated bit patterns in the
high bits of the compressed TOAST size, toast_compress_datum() could
just have hard-coded logic to use the correct handlers instead of
translating the bit pattern into an OID and then looking it up over
again.
(5) Going even further than #4 we could skip the handler layer
entirely for such methods, and just call the right function directly.
I think we should definitely do (1), and also (2) unless there's some
reason it's hard. (3) doesn't need to be part of this patch, but might
be something to consider later in the series. It's possible that it
doesn't have enough benefit to be worth the work, though. Also, I
think we should do either (4) or (5). I have a mild preference for (5)
unless it looks too ugly.
Note that I'm not talking about hard-coding a fast path for a
hard-coded list of OIDs - which would seem a little bit unprincipled -
but hard-coding a fast path for the bit patterns that are themselves
hard-coded. I don't think we lose anything in terms of extensibility
or even-handedness there; it's just avoiding a bunch of rigamarole
that doesn't really buy us anything.All these points apply equally to toast_decompress_datum_slice() and
toast_compress_datum().Fixed as discussed at [1]
+ /* Fallback to default compression method, if not specified */ + if (!OidIsValid(cmoid)) + cmoid = DefaultCompressionOid;I think that the caller should be required to specify a legal value,
and this should be an elog(ERROR) or an Assert().The change to equalTupleDescs() makes me wonder. Like, can we specify
the compression method for a function parameter, or a function return
value? I would think not. But then how are the tuple descriptors set
up in that case? Under what circumstances do we actually need the
tuple descriptors to compare unequal?If we alter the compression method then we check whether we need to
rebuild the tuple descriptor or not based on what value is changed so
if the attribute compression method is changed we need to rebuild the
compression method right. You might say that in the first patch we
are not allowing altering the compression method so we might move this
to the second patch but I thought since we added this field to
pg_attribute in this patch then better to add this check as well.
What am I missing?lz4.c's header comment calls it cm_lz4.c, and the pathname is wrong too.
I wonder if we should try to adopt a convention for the names of these
files that isn't just the compression method name, like cmlz4 or
compress_lz4. I kind of like the latter one. I am a little worried
that just calling it lz4.c will result in name collisions later - not
in this directory, of course, but elsewhere in the system. It's not a
disaster if that happens, but for example verbose error reports print
the file name, so it's nice if it's unambiguous.Changed to compress_lz4.
+ if (!IsBinaryUpgrade && + (relkind == RELKIND_RELATION || + relkind == RELKIND_PARTITIONED_TABLE)) + attr->attcompression = + GetAttributeCompressionMethod(attr, colDef->compression); + else + attr->attcompression = InvalidOid;Storing InvalidOid in the IsBinaryUpgrade case looks wrong. If
upgrading from pre-v14, we need to store PGLZ_COMPRESSION_OID.
Otherwise, we need to preserve whatever value was present in the old
version. Or am I confused here?Okay, so I think we can simply remove the IsBinaryUpgrade check so it
will behave as expected. Basically, now it the compression method is
specified then it will take that compression method and if it is not
specified then it will take the PGLZ_COMPRESSION_OID.I think there should be tests for the way this interacts with
partitioning, and I think the intended interaction should be
documented. Perhaps it should behave like TABLESPACE, where the parent
property has no effect on what gets stored because the parent has no
storage, but is inherited by each new child.I have added the test for this and also documented the same.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression?I have changed this to store the Invalid compression method always.
For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example).You mean to update the pg_attribute table for the toasted field (e.g
chunk_data) and set the attcompression to something valid? Or there
is a better way to write this test?For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.Fixed as described as [2]
+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
Fixed
I don't see a particularly good reason to treat plain and external
differently. More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure). However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.I'm out of time for today, so I'll have to look at this more another
day. Hope this helps for a start.Fixed as I have described at [2], and the rules are documented in
pg_attribute.h (atop attcompression field)[1] /messages/by-id/CA+Tgmob3W8cnLgOQX+JQzeyGN3eKGmRrBkUY6WGfNyHa+t_qEw@mail.gmail.com
[2] /messages/by-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W+LE+3bqWPJUZj4GnHOJg@mail.gmail.com
I was working on analyzing the behavior of how the attribute merging
should work for the compression method for an inherited child so for
that, I was analyzing the behavior for the storage method. I found
some behavior that doesn't seem right. Basically, while creating the
inherited child we don't allow the storage to be different than the
parent attribute's storage but later we are allowed to alter that, is
that correct behavior.
Here is the test case to demonstrate this.
postgres[12546]=# create table t (a varchar);
postgres[12546]=# alter table t ALTER COLUMN a SET STORAGE plain;
postgres[12546]=# create table t1 (a varchar);
postgres[12546]=# alter table t1 ALTER COLUMN a SET STORAGE external;
/* Not allowing to set the external because parent attribute has plain */
postgres[12546]=# create table t2 (LIKE t1 INCLUDING STORAGE) INHERITS ( t);
NOTICE: 00000: merging column "a" with inherited definition
LOCATION: MergeAttributes, tablecmds.c:2685
ERROR: 42804: column "a" has a storage parameter conflict
DETAIL: PLAIN versus EXTERNAL
LOCATION: MergeAttributes, tablecmds.c:2730
postgres[12546]=# create table t2 (LIKE t1 ) INHERITS (t);
/* But you can alter now */
postgres[12546]=# alter TABLE t2 ALTER COLUMN a SET STORAGE EXTERNAL ;
postgres[12546]=# \d+ t
Table "public.t"
Column | Type | Collation | Nullable | Default | Storage
| Compression | Stats target | Description
--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
a | character varying | | |
| plain | pglz | |
Child tables: t2
Access method: heap
postgres[12546]=# \d+ t2
Table "public.t2"
Column | Type | Collation | Nullable | Default | Storage
| Compression | Stats target | Description
--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
a | character varying | | |
| external | pglz | |
Inherits: t
Access method: heap
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Dec 17, 2020 at 10:55 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Dec 9, 2020 at 5:37 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Nov 21, 2020 at 3:50 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Nov 11, 2020 at 9:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
There were a few problems in this rebased version, basically, the
compression options were not passed while compressing values from the
brin_form_tuple, so I have fixed this.Since the authorship history of this patch is complicated, it would be
nice if you would include authorship information and relevant
"Discussion" links in the patches.I have added that.
Design level considerations and overall notes:
configure is autogenerated from configure.in, so the patch shouldn't
include changes only to the former.Yeah, I missed those changes. Done now.
Looking over the changes to src/include:
+ PGLZ_COMPRESSION_ID,
+ LZ4_COMPRESSION_IDI think that it would be good to assign values to these explicitly.
Done
+/* compresion handler routines */
Spelling.
Done
+ /* compression routine for the compression method */ + cmcompress_function cmcompress; + + /* decompression routine for the compression method */ + cmcompress_function cmdecompress;Don't reuse cmcompress_function; that's confusing. Just have a typedef
per structure member, even if they end up being the same.Fixed as suggested
#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \ - (((toast_compress_header *) (ptr))->rawsize = (len)) +do { \ + Assert(len > 0 && len <= RAWSIZEMASK); \ + ((toast_compress_header *) (ptr))->info = (len); \ +} while (0)Indentation.
Done
+#define TOAST_COMPRESS_SET_COMPRESSION_METHOD(ptr, cm_method) \ + ((toast_compress_header *) (ptr))->info |= ((cm_method) << 30);What about making TOAST_COMPRESS_SET_RAWSIZE() take another argument?
And possibly also rename it to TEST_COMPRESS_SET_SIZE_AND_METHOD() or
something? It seems not great to have separate functions each setting
part of a 4-byte quantity. Too much chance of failing to set both
parts. I guess you've got a function called
toast_set_compressed_datum_info() for that, but it's just a wrapper
around two macros that could just be combined, which would reduce
complexity overall.Done that way
+ T_CompressionRoutine, /* in access/compressionapi.h */
This looks misplaced. I guess it should go just after these:
T_FdwRoutine, /* in foreign/fdwapi.h */
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */Done
Looking over the regression test changes:
The tests at the top of create_cm.out that just test that we can
create tables with various storage types seem unrelated to the purpose
of the patch. And the file doesn't test creating a compression method
either, as the file name would suggest, so either the file name needs
to be changed (compression, compression_method?) or the tests don't go
here.Changed to "compression"
+-- check data is okdd
I guess whoever is responsible for this comment prefers vi to emacs.
Fixed
I don't quite understand the purpose of all of these tests, and there
are some things that I feel like ought to be tested that seemingly
aren't. Like, you seem to test using an UPDATE to move a datum from a
table to another table with the same compression method, but not one
with a different compression method.Added test for this, and some other tests to improve overall coverage.
Testing the former is nice and
everything, but that's the easy case: I think we also need to test the
latter. I think it would be good to verify not only that the data is
readable but that it's compressed the way we expect. I think it would
be a great idea to add a pg_column_compression() function in a similar
spirit to pg_column_size(). Perhaps it could return NULL when
compression is not in use or the data type is not varlena, and the
name of the compression method otherwise. That would allow for better
testing of this feature, and it would also be useful to users who are
switching methods, to see what data they still have that's using the
old method. It could be useful for debugging problems on customer
systems, too.This is a really great idea, I have added this function and used in my test.
I wonder if we need a test that moves data between tables through an
intermediary. For instance, suppose a plpgsql function or DO block
fetches some data and stores it in a plpgsql variable and then uses
the variable to insert into another table. Hmm, maybe that would force
de-TOASTing. But perhaps there are other cases. Maybe a more general
way to approach the problem is: have you tried running a coverage
report and checked which parts of your code are getting exercised by
the existing tests and which parts are not? The stuff that isn't, we
should try to add more tests. It's easy to get corner cases wrong with
this kind of thing.I notice that LIKE INCLUDING COMPRESSION doesn't seem to be tested, at
least not by 0001, which reinforces my feeling that the tests here are
not as thorough as they could be.Added test for this as well.
+NOTICE: pg_compression contains unpinned initdb-created object(s)
This seems wrong to me - why is it OK?
Yeah, this is wrong, now fixed.
- result = (struct varlena *) - palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); - SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ); + cmoid = GetCompressionOidFromCompressionId(TOAST_COMPRESS_METHOD(attr));- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr), - TOAST_COMPRESS_SIZE(attr), - VARDATA(result), - TOAST_COMPRESS_RAWSIZE(attr), true) < 0) - elog(ERROR, "compressed data is corrupted"); + /* get compression method handler routines */ + cmroutine = GetCompressionRoutine(cmoid);- return result; + return cmroutine->cmdecompress(attr);I'm worried about how expensive this might be, and I think we could
make it cheaper. The reason why I think this might be expensive is:
currently, for every datum, you have a single direct function call.
Now, with this, you first have a direct function call to
GetCompressionOidFromCompressionId(). Then you have a call to
GetCompressionRoutine(), which does a syscache lookup and calls a
handler function, which is quite a lot more expensive than a single
function call. And the handler isn't even returning a statically
allocated structure, but is allocating new memory every time, which
involves more function calls and maybe memory leaks. Then you use the
results of all that to make an indirect function call.I'm not sure exactly what combination of things we could use to make
this better, but it seems like there are a few possibilities:(1) The handler function could return a pointer to the same
CompressionRoutine every time instead of constructing a new one every
time.
(2) The CompressionRoutine to which the handler function returns a
pointer could be statically allocated instead of being built at
runtime.
(3) GetCompressionRoutine could have an OID -> handler cache instead
of relying on syscache + calling the handler function all over again.
(4) For the compression types that have dedicated bit patterns in the
high bits of the compressed TOAST size, toast_compress_datum() could
just have hard-coded logic to use the correct handlers instead of
translating the bit pattern into an OID and then looking it up over
again.
(5) Going even further than #4 we could skip the handler layer
entirely for such methods, and just call the right function directly.
I think we should definitely do (1), and also (2) unless there's some
reason it's hard. (3) doesn't need to be part of this patch, but might
be something to consider later in the series. It's possible that it
doesn't have enough benefit to be worth the work, though. Also, I
think we should do either (4) or (5). I have a mild preference for (5)
unless it looks too ugly.
Note that I'm not talking about hard-coding a fast path for a
hard-coded list of OIDs - which would seem a little bit unprincipled -
but hard-coding a fast path for the bit patterns that are themselves
hard-coded. I don't think we lose anything in terms of extensibility
or even-handedness there; it's just avoiding a bunch of rigamarole
that doesn't really buy us anything.All these points apply equally to toast_decompress_datum_slice() and
toast_compress_datum().Fixed as discussed at [1]
+ /* Fallback to default compression method, if not specified */ + if (!OidIsValid(cmoid)) + cmoid = DefaultCompressionOid;I think that the caller should be required to specify a legal value,
and this should be an elog(ERROR) or an Assert().The change to equalTupleDescs() makes me wonder. Like, can we specify
the compression method for a function parameter, or a function return
value? I would think not. But then how are the tuple descriptors set
up in that case? Under what circumstances do we actually need the
tuple descriptors to compare unequal?If we alter the compression method then we check whether we need to
rebuild the tuple descriptor or not based on what value is changed so
if the attribute compression method is changed we need to rebuild the
compression method right. You might say that in the first patch we
are not allowing altering the compression method so we might move this
to the second patch but I thought since we added this field to
pg_attribute in this patch then better to add this check as well.
What am I missing?lz4.c's header comment calls it cm_lz4.c, and the pathname is wrong too.
I wonder if we should try to adopt a convention for the names of these
files that isn't just the compression method name, like cmlz4 or
compress_lz4. I kind of like the latter one. I am a little worried
that just calling it lz4.c will result in name collisions later - not
in this directory, of course, but elsewhere in the system. It's not a
disaster if that happens, but for example verbose error reports print
the file name, so it's nice if it's unambiguous.Changed to compress_lz4.
+ if (!IsBinaryUpgrade && + (relkind == RELKIND_RELATION || + relkind == RELKIND_PARTITIONED_TABLE)) + attr->attcompression = + GetAttributeCompressionMethod(attr, colDef->compression); + else + attr->attcompression = InvalidOid;Storing InvalidOid in the IsBinaryUpgrade case looks wrong. If
upgrading from pre-v14, we need to store PGLZ_COMPRESSION_OID.
Otherwise, we need to preserve whatever value was present in the old
version. Or am I confused here?Okay, so I think we can simply remove the IsBinaryUpgrade check so it
will behave as expected. Basically, now it the compression method is
specified then it will take that compression method and if it is not
specified then it will take the PGLZ_COMPRESSION_OID.I think there should be tests for the way this interacts with
partitioning, and I think the intended interaction should be
documented. Perhaps it should behave like TABLESPACE, where the parent
property has no effect on what gets stored because the parent has no
storage, but is inherited by each new child.I have added the test for this and also documented the same.
I wonder in passing about TOAST tables and materialized views, which
are the other things that have storage. What gets stored for
attcompression?I have changed this to store the Invalid compression method always.
For a TOAST table it probably doesn't matter much
since TOAST table entries shouldn't ever be toasted themselves, so
anything that doesn't crash is fine (but maybe we should test that
trying to alter the compression properties of a TOAST table doesn't
crash, for example).You mean to update the pg_attribute table for the toasted field (e.g
chunk_data) and set the attcompression to something valid? Or there
is a better way to write this test?For a materialized view it seems reasonable to
want to set column properties, but I'm not quite sure how that works
today for things like STORAGE anyway. If we do allow setting STORAGE
or COMPRESSION for materialized view columns then dump-and-reload
needs to preserve the values.Fixed as described as [2]
+ /* + * Use default compression method if the existing compression method is + * invalid but the new storage type is non plain storage. + */ + if (!OidIsValid(attrtuple->attcompression) && + (newstorage != TYPSTORAGE_PLAIN)) + attrtuple->attcompression = DefaultCompressionOid;You have a few too many parens in there.
Fixed
I don't see a particularly good reason to treat plain and external
differently. More generally, I think there's a question here about
when we need an attribute to have a valid compression type and when we
don't. If typstorage is plan or external, then there's no point in
ever having a compression type and maybe we should even reject
attempts to set one (but I'm not sure). However, the attstorage is a
different case. Suppose the column is created with extended storage
and then later it's changed to plain. That's only a hint, so there may
still be toasted values in that column, so the compression setting
must endure. At any rate, we need to make sure we have clear and
sensible rules for when attcompression (a) must be valid, (b) may be
valid, and (c) must be invalid. And those rules need to at least be
documented in the comments, and maybe in the SGML docs.I'm out of time for today, so I'll have to look at this more another
day. Hope this helps for a start.Fixed as I have described at [2], and the rules are documented in
pg_attribute.h (atop attcompression field)[1] /messages/by-id/CA+Tgmob3W8cnLgOQX+JQzeyGN3eKGmRrBkUY6WGfNyHa+t_qEw@mail.gmail.com
[2] /messages/by-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W+LE+3bqWPJUZj4GnHOJg@mail.gmail.comI was working on analyzing the behavior of how the attribute merging
should work for the compression method for an inherited child so for
that, I was analyzing the behavior for the storage method. I found
some behavior that doesn't seem right. Basically, while creating the
inherited child we don't allow the storage to be different than the
parent attribute's storage but later we are allowed to alter that, is
that correct behavior.Here is the test case to demonstrate this.
postgres[12546]=# create table t (a varchar);
postgres[12546]=# alter table t ALTER COLUMN a SET STORAGE plain;
postgres[12546]=# create table t1 (a varchar);
postgres[12546]=# alter table t1 ALTER COLUMN a SET STORAGE external;/* Not allowing to set the external because parent attribute has plain */
postgres[12546]=# create table t2 (LIKE t1 INCLUDING STORAGE) INHERITS ( t);
NOTICE: 00000: merging column "a" with inherited definition
LOCATION: MergeAttributes, tablecmds.c:2685
ERROR: 42804: column "a" has a storage parameter conflict
DETAIL: PLAIN versus EXTERNAL
LOCATION: MergeAttributes, tablecmds.c:2730
On further analysis, IMHO the reason for this error is not that it can
not allow different storage methods for inherited child's attributes
but it is reporting error because of conflicting storage between child
and parent. For example, if we inherit a child from two-parent who
have the same attribute name with a different storage type then also
it will conflict. I know that if it conflicts between parent and
child we might give preference to the child's storage but I don't see
much problem with the current behavior also. So as of now, I have
kept the same behavior for the compression as well. I have added a
test case for the same.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v16-0002-alter-table-set-compression.patchapplication/octet-stream; name=v16-0002-alter-table-set-compression.patchDownload
From 82237b45791326cfad5a2e4908ac8720f059312c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v16 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 +++++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/sql/compression.sql | 21 +++
8 files changed, 272 insertions(+), 46 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..bfa5c36fb7 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. Compression method can be set
+ from available built-in compression methods. The available built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fe33248589..8944ebc586 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3860,6 +3862,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4387,7 +4390,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4795,6 +4799,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5419,6 +5427,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7659,6 +7670,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7674,7 +7750,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7738,47 +7813,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -14999,6 +15035,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 39467aa2e0..9683062216 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2257,6 +2257,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 3a43c09bf6..1da16ebc0a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,7 +2093,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..281509e432 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
* when a function is called by the event trigger manager.
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 78b7f76215..af79aa6534 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1856,7 +1856,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 701a24a6c7..06ac4161ae 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -142,6 +142,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 78f8726a29..aa8e7492df 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -65,6 +65,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v16-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v16-0004-Create-custom-compression-methods.patchDownload
From da304477b42e84f5af59c8fedf0e7bdf4442fbc3 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 7 Dec 2020 15:26:57 +0530
Subject: [PATCH v16 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
---
doc/src/sgml/ref/alter_table.sgml | 17 +++---
doc/src/sgml/ref/create_access_method.sgml | 13 ++--
doc/src/sgml/ref/create_table.sgml | 11 ++--
src/backend/access/common/detoast.c | 59 +++++++++++++++++--
src/backend/access/common/toast_internals.c | 19 +++++-
src/backend/access/compression/compress_lz4.c | 21 +++----
.../access/compression/compress_pglz.c | 20 +++----
.../access/compression/compressamapi.c | 47 +++++++++++++++
src/backend/commands/amcmds.c | 5 ++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 +
src/bin/psql/tab-complete.c | 6 ++
src/include/access/compressamapi.h | 16 +++--
src/include/access/detoast.h | 8 +++
src/include/access/toast_internals.h | 15 +++++
src/test/regress/expected/compression.out | 41 ++++++++++++-
src/test/regress/sql/compression.sql | 11 ++++
19 files changed, 264 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 2fb9cfaadc..9efbc2352c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,14 +390,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. Compression method can be set
- from available built-in compression methods. The available built-in
- methods are <literal>pglz</literal> and <literal>lz4</literal>. The
- PRESERVE list contains list of compression methods used on the column and
- determines which of them should be kept on the column. Without PRESERVE or
- if all the previous compression methods are not preserved then the table
- will be rewritten. If PRESERVE ALL is specified then all the previous
- methods will be preserved and the table will not be rewritten.
+ This clause adds compression to a column. Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or it can
+ be set from the available built-in compression methods. The available
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ The PRESERVE list contains list of compression methods used on the column
+ and determines which of them should be kept on the column. Without
+ PRESERVE or if all the previous compression methods are not preserved then
+ the table will be rewritten. If PRESERVE ALL is specified then all the
+ previous methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index f404dd1088..ade3989d75 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -999,11 +999,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. Compression method
- can be set from the available built-in compression methods. The available
- options are <literal>pglz</literal> and <literal>lz4</literal>. If the
- compression method is not sepcified for the compressible type then it will
- have the default compression method. The default compression method is
- <literal>pglz</literal>.
+ could be created with <xref linkend="sql-create-access-method"/> or it can
+ be set from the available built-in compression methods. The available
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If the compression method is not sepcified for the compressible type then
+ it will have the default compression method. The default compression
+ method is <literal>pglz</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d66d733da6..f6c48ac955 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,13 +438,44 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -458,10 +489,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -477,9 +519,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -493,16 +537,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 72bf23f712..cd91aefef7 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index b455367be3..ce27616116 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -29,7 +29,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -44,10 +44,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -55,7 +55,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -67,7 +67,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -80,9 +80,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -99,7 +99,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -111,9 +112,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 2a3ef17842..d693c880fd 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c8d2..9d96fad31f 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -59,3 +60,49 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ HeapTuple tuple;
+ Form_pg_am amform;
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ tuple = SearchSysCache1(AMOID, ObjectIdGetDatum(amoid));
+ if (!HeapTupleIsValid(tuple))
+ elog(ERROR, "cache lookup failed for access method %u",
+ amoid);
+
+ amform = (Form_pg_am)GETSTRUCT(tuple);
+
+ /* Check if it's an index access method as opposed to some other AM */
+ if (amform->amtype != AMTYPE_COMPRESSION)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname), "INDEX")));
+
+ amhandler = amform->amhandler;
+
+ /* Complain if handler OID is invalid */
+ if (!RegProcedureIsValid(amhandler))
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("index access method \"%s\" does not have a handler",
+ NameStr(amform->amname))));
+
+ ReleaseSysCache(tuple);
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ return routine;
+}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index fa8deda08b..bdd6d0a8ac 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -227,6 +227,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -264,6 +266,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 53a4e3131c..248b27f564 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1969,7 +1969,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* the target compression method is not same then we need to
* decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 22bd0e87a3..54ff6fb937 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5254,6 +5254,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index d6a36f7e97..ed8169a01f 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5285,7 +5285,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5315,10 +5314,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 201955fc3d..923dfa6451 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13035,6 +13035,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8178a5124b..9bfa66ff09 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0596..913a633d83 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -26,18 +26,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -61,5 +68,6 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index e13e34cac3..81944912d2 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9d87ec4ad7..82fd2d1a99 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -235,13 +235,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
@@ -270,3 +308,4 @@ SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP ACCESS METHOD pglz2;
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 9a9c3d95e7..8075fb7696 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -95,6 +95,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
@@ -104,3 +114,4 @@ SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP ACCESS METHOD pglz2;
--
2.23.0
v16-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v16-0005-new-compression-method-extension-for-zlib.patchDownload
From 83d3483cdf7c7053686fac865d5b3b02c087f906 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v16 5/6] new compression method extension for zlib
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..bf36854ef3
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ lz4
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..aa7c57030d
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.23.0
v16-0001-Built-in-compression-method.patchapplication/octet-stream; name=v16-0001-Built-in-compression-method.patchDownload
From bc826835e2ff58350d2e295297f2c761780b4b42 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v16 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 2 +
doc/src/sgml/ref/create_table.sgml | 27 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++
.../access/compression/compress_pglz.c | 131 +++++++++
.../access/compression/compressamapi.c | 61 +++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 ++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 40 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 ++-
src/include/access/compressamapi.h | 65 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 178 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 76 ++++++
src/tools/pgindent/typedefs.list | 1 +
77 files changed, 1989 insertions(+), 667 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index 11a4284e5b..9a0c7ba652 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -866,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8601,6 +8604,35 @@ fi
+#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12092,6 +12124,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13295,6 +13380,20 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index fc523c6aeb..4bf18c9938 100644
--- a/configure.ac
+++ b/configure.ac
@@ -999,6 +999,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1186,6 +1193,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1400,6 +1415,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index c12a32c8c7..5ec3d995f8 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,8 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ Partitions inherits the compression method of the parent for each column
+ however we can set different compression method for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..f404dd1088 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be coppied. The default
+ behavior is to exclude compression method, resulting in the copied
+ column will have the default compression method if the column type is
+ compressible.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +994,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method is not sepcified for the compressible type then it will
+ have the default compression method. The default compression method is
+ <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 17e50de530..2b8a6de6ed 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..d66d733da6 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -438,28 +439,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -473,22 +493,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..3024670414 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..72bf23f712 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9aac155e4a 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..c4814ef911
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..b455367be3
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+#endif
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..2a3ef17842
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compression
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000000..663102c8d2
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index a7ed93fdc1..8162ecd612 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 66fdaf67b1..94278786d9 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 51b5c4f7f6..e5554e2e33 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1706,6 +1708,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 731610c701..7e6310b6f7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index f1850436bd..e5452614f7 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 6f05ee715b..fa8deda08b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 6bf6c5a310..f7c9ff0a3e 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -558,6 +558,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index cfc63915f3..2e1d990f9e 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -486,6 +486,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1fa9f19f08..fe33248589 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2408,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2457,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2703,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6233,6 +6275,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11752,6 +11806,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17631,3 +17701,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ab3d655e60..64a396a218 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1915,6 +1918,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2120,6 +2195,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 70f8b718e0..7d63cee5a5 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,6 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 541e0e6b48..de26dce6d6 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2586,6 +2586,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 963f71e99d..d7a93f9360 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3848,6 +3848,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d78b16ed1d..55b35e6f13 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2853,6 +2853,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8f341ac006..39467aa2e0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -593,6 +593,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -628,9 +630,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3370,11 +3372,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3383,8 +3386,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3429,6 +3432,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15110,6 +15122,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15628,6 +15641,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 89ee990599..e5184933b7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6b0a59efaf..89b4a9bf60 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 99a93271fe..e177ed77d8 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 9300d19e0c..d6a36f7e97 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "common/hex_decode.h"
@@ -5275,6 +5277,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1ab98a2286..201955fc3d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -386,6 +386,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -8611,6 +8612,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8696,6 +8698,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 120000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8714,7 +8725,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8741,6 +8757,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8769,6 +8786,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15796,6 +15814,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15820,6 +15839,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15855,6 +15877,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index d7f77f1d3e..c08b1df75d 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 14150d05a9..724000c48d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 120000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..8226ae0596
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..4e932bc067 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..e13e34cac3 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 0f051277a6..3a8d0ac09c 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index bc73e88a1a..0ffc844995 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index cdf75a2380..3e6ed706ac 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For the compressible atttypid this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible atttypid this must always be an invalid Oid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 22970f46cd..d851619543 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '4388', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '4389', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7047,6 +7056,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2228',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7217,6 +7230,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 62018f063a..d8b1cbba8c 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -619,6 +619,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1133ae1143..2d76a85336 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..df439c8c38 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 3684f87a88..c8d1f20b3b 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -509,6 +509,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 48a79a7657..78b7f76215 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index de8f838e53..f894df3504 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..701a24a6c7
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,178 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+DROP MATERIALIZED VIEW mv;
+DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 2c0760404d..e04e98ff85 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6293ab57bc..b79b0a2a7f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3019,11 +3019,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3039,11 +3039,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3070,11 +3070,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 7bfeaf85f0..197c312ed6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e0e1ef71dd..2407ca945b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,6 +116,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 081fce32e7..152c8cb5b7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -203,3 +203,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..78f8726a29
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,76 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+DROP MATERIALIZED VIEW mv;
+DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bca37c536e..f60734f64e 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -394,6 +394,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
v16-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v16-0003-Add-support-for-PRESERVE.patchDownload
From 8a79522520466b409f26e9d18247c276b7b09a78 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 7 Dec 2020 11:55:39 +0530
Subject: [PATCH v16 3/6] Add support for PRESERVE
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 11 +-
src/backend/catalog/objectaddress.c | 1 +
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 231 +++++++++++++++++++++
src/backend/commands/tablecmds.c | 117 ++++++-----
src/backend/executor/nodeModifyTable.c | 10 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 +++-
src/test/regress/expected/create_index.out | 56 ++---
src/test/regress/sql/compression.sql | 9 +
19 files changed, 517 insertions(+), 95 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index bfa5c36fb7..2fb9cfaadc 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method can be set
from available built-in compression methods. The available built-in
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ methods are <literal>pglz</literal> and <literal>lz4</literal>. The
+ PRESERVE list contains list of compression methods used on the column and
+ determines which of them should be kept on the column. Without PRESERVE or
+ if all the previous compression methods are not preserved then the table
+ will be rewritten. If PRESERVE ALL is specified then all the previous
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index a5eccdffd0..636364995a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 429791694f..3c7aea0df4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..91eb59ca17
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,231 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * get list of all supported compression methods for the given attribute.
+ *
+ * If oldcmoids list is passed then it will delete the attribute dependency
+ * on the compression methods passed in the oldcmoids, otherwise it will
+ * return the list of all the compression method on which the attribute has
+ * dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ {
+ if (oldcmoids && list_member_oid(oldcmoids, depform->refobjid))
+ CatalogTupleDelete(rel, &tup->t_self);
+ else if (oldcmoids == NULL)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods given in the
+ * cmoids list.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ lookup_attribute_compression(attrelid, attnum, cmoids);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from multiple
+ * mentions of one access method in PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8944ebc586..5907d7b2b8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -390,6 +390,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -530,7 +531,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Specify relation Oid in attribute compression tuples, and create
+ * dependencies.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2413,16 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2459,7 +2477,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2710,12 +2729,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4800,7 +4819,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6294,7 +6314,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6469,6 +6490,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6616,6 +6638,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used to determine connection between column and builtin
+ * compression in ALTER SET COMPRESSION command.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11751,7 +11795,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11866,6 +11911,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15044,24 +15094,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15094,11 +15141,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17823,32 +17875,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 64a396a218..53a4e3131c 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1933,6 +1934,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1963,10 +1965,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * compare with the compression method of the target attribute. If
+ * the target compression method is not same then we need to
+ * decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7d63cee5a5..321da2accd 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,7 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2952,6 +2952,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5623,6 +5635,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index de26dce6d6..ade226efdc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2586,7 +2586,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2606,6 +2606,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3678,6 +3688,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 55b35e6f13..2e60bb750e 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2853,7 +2853,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2871,6 +2871,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4215,6 +4225,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9683062216..22bd0e87a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -593,7 +593,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2258,12 +2260,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3386,7 +3388,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3441,13 +3443,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e5184933b7..4bbbf6370a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1da16ebc0a..8178a5124b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,6 +2098,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 2d76a85336..d1880dfc5e 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index c8d1f20b3b..885ff5ec5c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -478,6 +478,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index af79aa6534..c8135debbb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 06ac4161ae..9d87ec4ad7 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -201,12 +201,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index fc6afab58a..d8300bdcdd 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index aa8e7492df..9a9c3d95e7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -86,6 +86,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v16-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v16-0006-Support-compression-methods-options.patchDownload
From 10e39d80ac8db5120a4ee8631ff78f4976a221c6 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Dec 2020 16:46:53 +0530
Subject: [PATCH v16 6/6] Support compression methods options
---
contrib/cmzlib/cmzlib.c | 75 +++++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 +++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 69 +++++++++++-
.../access/compression/compress_pglz.c | 98 +++++++++++++++--
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 ++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++++-
src/backend/commands/foreigncmds.c | 44 --------
src/backend/commands/tablecmds.c | 102 +++++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/include/access/compressamapi.h | 19 +++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 21 +++-
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 11 ++
33 files changed, 601 insertions(+), 130 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..cb5aa439e3 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 9efbc2352c..87e1a1acf3 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,7 +386,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
@@ -394,6 +394,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
could be created with <xref linkend="sql-create-access-method"/> or it can
be set from the available built-in compression methods. The available
built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If compression method has options they could be specified with
+ <literal>WITH</literal> parameter.
The PRESERVE list contains list of compression methods used on the column
and determines which of them should be kept on the column. Without
PRESERVE or if all the previous compression methods are not preserved then
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index ade3989d75..95e03f6379 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -995,7 +995,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
@@ -1004,7 +1004,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not sepcified for the compressible type then
it will have the default compression method. The default compression
- method is <literal>pglz</literal>.
+ method is <literal>pglz</literal>. If the compression method has options
+ they could be specified by <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 2b8a6de6ed..61fc6fbb30 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 3024670414..0cd7bdd730 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..9f0389a994 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index cd91aefef7..e9b5971928 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index ce27616116..d1ee63c8e9 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,12 +16,70 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
#ifdef HAVE_LIBLZ4
#include "lz4.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -29,7 +87,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -39,6 +97,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -46,9 +105,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d693c880fd..7cc5a22e37 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unexpected parameter for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is out of the allowed
* range for compression
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..7e399daaa1 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e5554e2e33..89899ba382 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1151,6 +1159,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1409,7 +1418,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7e6310b6f7..b878b4af7f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index e5452614f7..6cf32fef6a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index fd5a6eec86..29a8145b1d 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -697,6 +697,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 91eb59ca17..46a598add8 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -27,6 +27,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* get list of all supported compression methods for the given attribute.
@@ -127,7 +128,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -153,6 +154,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -217,15 +232,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttributes compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5907d7b2b8..2016e975c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -610,6 +610,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -814,6 +815,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -867,8 +870,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -922,8 +926,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2431,16 +2438,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2477,8 +2482,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2728,14 +2732,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6152,6 +6151,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6315,6 +6315,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6325,7 +6326,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7722,13 +7723,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option must be only valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7767,7 +7775,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7858,7 +7888,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15104,9 +15134,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15141,7 +15173,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15151,8 +15184,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15160,8 +15202,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 321da2accd..5955a67058 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2959,6 +2959,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ade226efdc..982f1673e8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2611,6 +2611,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 2e60bb750e..74faf0f997 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2878,6 +2878,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 54ff6fb937..65ea7a4d39 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -413,6 +413,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3452,11 +3453,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3464,14 +3471,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4bbbf6370a..62ef275392 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 913a633d83..8123cc8cc7 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -37,8 +38,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -49,14 +53,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 4e932bc067..1dc4aa9623 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 81944912d2..dc35064086 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index f94defff3c..bd192f457e 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3e6ed706ac..55ea9af6bd 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index d1880dfc5e..138ed6976f 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -138,6 +138,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,9 +152,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c8135debbb..09e5e00d99 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 82fd2d1a99..51ee61cf63 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -272,6 +272,23 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata(f1);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1000));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1008));
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
@@ -291,7 +308,9 @@ SELECT length(f1) FROM cmmove1;
length
--------
10000
-(1 row)
+ 10040
+ 10080
+(3 rows)
SELECT length(f1) FROM cmmove2;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..27aab82462 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 8075fb7696..314fb4d4b6 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,17 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+DROP TABLE cmmove1;
+CREATE TABLE cmmove1(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata(f1);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1000));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1004));
+ALTER TABLE cmmove1 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmmove1 VALUES(repeat('1234567890',1008));
+SELECT pg_column_compression(f1) FROM cmmove1;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
Re: [HACKERS] Custom compression methods
25 дек. 2020 г., в 14:34, Dilip Kumar <dilipbalaut@gmail.com> написал(а):
<v16-0002-alter-table-set-compression.patch> <v16-0004-Create-custom-compression-methods.patch> <v16-0005-new-compression-method-extension-for-zlib.patch> <v16-0001-Built-in-compression-method.patch> <v16-0003-Add-support-for-PRESERVE.patch> <v16-0006-Support-compression-methods-options.patch>
Maybe add Lz4\Zlib WAL FPI compression on top of this patchset? I'm not insisting on anything, it just would be so cool to have it...
BTW currently there are Oid collisions in original patchset.
Best regards, Andrey Borodin.
Attachments:
v16-0007-Add-Lz4-compression-to-WAL-FPIs.patchapplication/octet-stream; name=v16-0007-Add-Lz4-compression-to-WAL-FPIs.patch; x-unix-mode=0644Download
From b26154cd972bca454ed3e21a8f66b47600a07313 Mon Sep 17 00:00:00 2001
From: Andrey Borodin <amborodin@acm.org>
Date: Sun, 27 Dec 2020 12:01:35 +0500
Subject: [PATCH v16 7/7] Add Lz4 compression to WAL FPIs
---
src/backend/access/transam/xlog.c | 1 +
src/backend/access/transam/xloginsert.c | 41 ++++++++++++++++++++++---
src/backend/access/transam/xlogreader.c | 32 +++++++++++++++++--
src/backend/utils/misc/guc.c | 17 ++++++++++
src/include/access/xlog.h | 2 ++
src/include/access/xlogreader.h | 1 +
src/include/access/xlogrecord.h | 1 +
src/include/catalog/pg_am.dat | 4 +--
src/include/catalog/pg_proc.dat | 6 ++--
9 files changed, 93 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 48ca46a941..8e4f31819e 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -112,6 +112,7 @@ int CommitDelay = 0; /* precommit delay in microseconds */
int CommitSiblings = 5; /* # concurrent xacts needed to sleep */
int wal_retrieve_retry_interval = 5000;
int max_slot_wal_keep_size_mb = -1;
+int wal_compression_method = LZ4_COMPRESSION_ID;
#ifdef WAL_DEBUG
bool XLOG_DEBUG = false;
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1f0e4e01e6..a7f4accb1c 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "access/xlog_internal.h"
@@ -33,6 +34,10 @@
#include "storage/proc.h"
#include "utils/memutils.h"
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
/* Buffer size required to store a compressed version of backup block image */
#define PGLZ_MAX_BLCKSZ PGLZ_MAX_OUTPUT(BLCKSZ)
@@ -113,7 +118,8 @@ static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info,
XLogRecPtr RedoRecPtr, bool doPageWrites,
XLogRecPtr *fpw_lsn, int *num_fpi);
static bool XLogCompressBackupBlock(char *page, uint16 hole_offset,
- uint16 hole_length, char *dest, uint16 *dlen);
+ uint16 hole_length, char *dest,
+ uint16 *dlen, CompressionId compression);
/*
* Begin constructing a WAL record. This must be called before the
@@ -630,11 +636,15 @@ XLogRecordAssemble(RmgrId rmid, uint8 info,
*/
if (wal_compression)
{
+ bimg.compression_method = PGLZ_COMPRESSION_ID;
+#ifdef HAVE_LIBLZ4
+ bimg.compression_method = wal_compression_method;
+#endif
is_compressed =
XLogCompressBackupBlock(page, bimg.hole_offset,
cbimg.hole_length,
regbuf->compressed_page,
- &compressed_len);
+ &compressed_len, bimg.compression_method);
}
/*
@@ -827,7 +837,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info,
*/
static bool
XLogCompressBackupBlock(char *page, uint16 hole_offset, uint16 hole_length,
- char *dest, uint16 *dlen)
+ char *dest, uint16 *dlen, CompressionId compression)
{
int32 orig_len = BLCKSZ - hole_length;
int32 len;
@@ -853,12 +863,33 @@ XLogCompressBackupBlock(char *page, uint16 hole_offset, uint16 hole_length,
else
source = page;
+ if (compression == LZ4_COMPRESSION_ID)
+ {
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ len = LZ4_compress_fast(source, dest, orig_len, PGLZ_MAX_BLCKSZ, 1);
+#endif
+ }
+ else if (compression == PGLZ_COMPRESSION_ID)
+ {
+ len = pglz_compress(source, orig_len, dest, PGLZ_strategy_default);
+ }
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("unknown compression method requested")));
+ }
+
+
/*
- * We recheck the actual size even if pglz_compress() reports success and
+ * We recheck the actual size even if compression reports success and
* see if the number of bytes saved by compression is larger than the
* length of extra data needed for the compressed version of block image.
*/
- len = pglz_compress(source, orig_len, dest, PGLZ_strategy_default);
if (len >= 0 &&
len + extra_bytes < orig_len)
{
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index a63ad8cfd0..d1da11b79b 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -33,6 +33,10 @@
#include "utils/memutils.h"
#endif
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
static void report_invalid_record(XLogReaderState *state, const char *fmt,...)
pg_attribute_printf(2, 3);
static bool allocate_recordbuf(XLogReaderState *state, uint32 reclength);
@@ -1291,6 +1295,7 @@ DecodeXLogRecord(XLogReaderState *state, XLogRecord *record, char **errormsg)
{
COPY_HEADER_FIELD(&blk->bimg_len, sizeof(uint16));
COPY_HEADER_FIELD(&blk->hole_offset, sizeof(uint16));
+ COPY_HEADER_FIELD(&blk->compression_method, sizeof(uint8));
COPY_HEADER_FIELD(&blk->bimg_info, sizeof(uint8));
blk->apply_image = ((blk->bimg_info & BKPIMAGE_APPLY) != 0);
@@ -1565,8 +1570,31 @@ RestoreBlockImage(XLogReaderState *record, uint8 block_id, char *page)
if (bkpb->bimg_info & BKPIMAGE_IS_COMPRESSED)
{
/* If a backup block image is compressed, decompress it */
- if (pglz_decompress(ptr, bkpb->bimg_len, tmp.data,
- BLCKSZ - bkpb->hole_length, true) < 0)
+ int32 decomp_result = -1;
+ if (bkpb->compression_method == PGLZ_COMPRESSION_ID)
+ {
+ decomp_result = pglz_decompress(ptr, bkpb->bimg_len, tmp.data,
+ BLCKSZ - bkpb->hole_length, true);
+ }
+ else if (bkpb->compression_method == LZ4_COMPRESSION_ID)
+ {
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ decomp_result = LZ4_decompress_safe(ptr, tmp.data, bkpb->bimg_len, BLCKSZ);
+#endif
+ }
+ else
+ {
+#ifndef FRONTEND
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("unknown compression method requested")));
+#endif
+ }
+ if ( decomp_result < 0)
{
report_invalid_record(record, "invalid compressed image at %X/%X, block %d",
(uint32) (record->ReadRecPtr >> 32),
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index bbaf037bc6..290fd00687 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -483,6 +484,12 @@ const struct config_enum_entry ssl_protocol_versions_info[] = {
{NULL, 0, false}
};
+const struct config_enum_entry wal_compression_options[] = {
+ {"pglz", PGLZ_COMPRESSION_ID, false},
+ {"lz4", LZ4_COMPRESSION_ID, false},
+ {NULL, 0, false}
+};
+
StaticAssertDecl(lengthof(ssl_protocol_versions_info) == (PG_TLS1_3_VERSION + 2),
"array length mismatch");
@@ -4679,6 +4686,16 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"wal_compression_method", PGC_SIGHUP, WAL_SETTINGS,
+ gettext_noop("Set the method used to compress full page images in the WAL."),
+ NULL
+ },
+ &wal_compression_method,
+ LZ4_COMPRESSION_ID, wal_compression_options,
+ NULL, NULL, NULL
+ },
+
{
{"dynamic_shared_memory_type", PGC_POSTMASTER, RESOURCES_MEM,
gettext_noop("Selects the dynamic shared memory implementation used."),
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e71..5099c07b63 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -11,6 +11,7 @@
#ifndef XLOG_H
#define XLOG_H
+#include "access/compressamapi.h"
#include "access/rmgr.h"
#include "access/xlogdefs.h"
#include "access/xloginsert.h"
@@ -175,6 +176,7 @@ typedef enum RecoveryState
} RecoveryState;
extern PGDLLIMPORT int wal_level;
+extern PGDLLIMPORT int wal_compression_method;
/* Is WAL archiving enabled (always or only while server is running normally)? */
#define XLogArchivingActive() \
diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h
index 0b6d00dd7d..791b6f435c 100644
--- a/src/include/access/xlogreader.h
+++ b/src/include/access/xlogreader.h
@@ -133,6 +133,7 @@ typedef struct
bool apply_image; /* has image that should be restored */
char *bkp_image;
uint16 hole_offset;
+ uint8 compression_method;
uint16 hole_length;
uint16 bimg_len;
uint8 bimg_info;
diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h
index 2f0c8bf589..7eee5d42d0 100644
--- a/src/include/access/xlogrecord.h
+++ b/src/include/access/xlogrecord.h
@@ -131,6 +131,7 @@ typedef struct XLogRecordBlockImageHeader
{
uint16 length; /* number of page image bytes */
uint16 hole_offset; /* number of bytes before "hole" */
+ uint8 compression_method;
uint8 bimg_info; /* flag bits, see below */
/*
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 3a8d0ac09c..a984eeeadf 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,10 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+{ oid => '8022', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
-{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+{ oid => '8023', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
descr => 'lz4 compression access method',
amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d851619543..3148285f45 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,11 +941,11 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
-{ oid => '4388', descr => 'pglz compression access method handler',
+{ oid => '8024', descr => 'pglz compression access method handler',
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
-{ oid => '4389', descr => 'lz4 compression access method handler',
+{ oid => '8021', descr => 'lz4 compression access method handler',
proname => 'lz4handler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'lz4handler' },
@@ -7056,7 +7056,7 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
-{ oid => '2228',
+{ oid => '8025',
descr => 'compression method for the compressed datum',
proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
proargtypes => 'any', prosrc => 'pg_column_compression' },
--
2.24.3 (Apple Git-128)
Re: [HACKERS] Custom compression methods
On Sun, Dec 27, 2020 at 12:40 PM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
25 дек. 2020 г., в 14:34, Dilip Kumar <dilipbalaut@gmail.com> написал(а):
<v16-0002-alter-table-set-compression.patch> <v16-0004-Create-custom-compression-methods.patch> <v16-0005-new-compression-method-extension-for-zlib.patch> <v16-0001-Built-in-compression-method.patch> <v16-0003-Add-support-for-PRESERVE.patch> <v16-0006-Support-compression-methods-options.patch>
Maybe add Lz4\Zlib WAL FPI compression on top of this patchset? I'm not insisting on anything, it just would be so cool to have it...
BTW currently there are Oid collisions in original patchset.
Thanks for the patch. Maybe we can allow setting custom compression
methods for wal compression as well.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
28 дек. 2020 г., в 10:20, Dilip Kumar <dilipbalaut@gmail.com> написал(а):
On Sun, Dec 27, 2020 at 12:40 PM Andrey Borodin <x4mmm@yandex-team.ru> wrote:
25 дек. 2020 г., в 14:34, Dilip Kumar <dilipbalaut@gmail.com> написал(а):
<v16-0002-alter-table-set-compression.patch> <v16-0004-Create-custom-compression-methods.patch> <v16-0005-new-compression-method-extension-for-zlib.patch> <v16-0001-Built-in-compression-method.patch> <v16-0003-Add-support-for-PRESERVE.patch> <v16-0006-Support-compression-methods-options.patch>
Maybe add Lz4\Zlib WAL FPI compression on top of this patchset? I'm not insisting on anything, it just would be so cool to have it...
BTW currently there are Oid collisions in original patchset.
Thanks for the patch. Maybe we can allow setting custom compression
methods for wal compression as well.
No, unfortunately, we can't use truly custom methods. Custom compression handlers are WAL-logged. So we can use only static set of hardcoded compression methods.
Thanks!
Best regards, Andrey Borodin.
Re: [HACKERS] Custom compression methods
28 дек. 2020 г., в 11:14, Andrey Borodin <x4mmm@yandex-team.ru> написал(а):
Thanks for the patch. Maybe we can allow setting custom compression
methods for wal compression as well.No, unfortunately, we can't use truly custom methods. Custom compression handlers are WAL-logged. So we can use only static set of hardcoded compression methods.
So, I've made some very basic benchmarks on my machine [0]https://yadi.sk/d/6y5YiROXQRkoEw.
With pglz after checkpoint I observe 1146 and 1225 tps.
With lz4 I observe 1485 and 1524 tps.
Without wal_compression I see 1529 tps.
These observations can be explained with plain statement: pglz is bottleneck on my machine, lz4 is not.
While this effect can be reached with other means [1]/messages/by-id/25991595-1848-4178-AA57-872B10309DA2@yandex-team.ru I believe having lz4 for WAL FPIs would be much more CPU efficient.
FPA lz4 for WAL FPI patch v17. Changes: fixed some frontend issues, added some comments.
Best regards, Andrey Borodin.
[0]: https://yadi.sk/d/6y5YiROXQRkoEw
[1]: /messages/by-id/25991595-1848-4178-AA57-872B10309DA2@yandex-team.ru
Attachments:
v17-0007-Add-Lz4-compression-to-WAL-FPIs.patchapplication/octet-stream; name=v17-0007-Add-Lz4-compression-to-WAL-FPIs.patch; x-unix-mode=0644Download
From e0ef41560d8f9a4abd2da5b0c6033836aa096ecc Mon Sep 17 00:00:00 2001
From: Andrey Borodin <amborodin@acm.org>
Date: Sun, 27 Dec 2020 12:01:35 +0500
Subject: [PATCH v17 7/7] Add Lz4 compression to WAL FPIs
Introduce wal_compression_method GUC to control compression
codec used to compress WAL FPI. This GUC defaults to pglz
to avoid incompatibility between builds. We cannot use custom
compression methods in WAL because custom methods are registred
in WAL-logged catalog.
NB: maybe bump WAL file magic.
---
src/backend/access/transam/xlog.c | 1 +
src/backend/access/transam/xloginsert.c | 41 ++++++++++++++++++++++---
src/backend/access/transam/xlogreader.c | 35 +++++++++++++++++++--
src/backend/utils/misc/guc.c | 17 ++++++++++
src/include/access/xlog.h | 2 ++
src/include/access/xlogreader.h | 1 +
src/include/access/xlogrecord.h | 7 +++--
src/include/catalog/pg_am.dat | 4 +--
src/include/catalog/pg_proc.dat | 6 ++--
9 files changed, 99 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/transam/xlog.c b/src/backend/access/transam/xlog.c
index 48ca46a941..eb31a167d1 100644
--- a/src/backend/access/transam/xlog.c
+++ b/src/backend/access/transam/xlog.c
@@ -112,6 +112,7 @@ int CommitDelay = 0; /* precommit delay in microseconds */
int CommitSiblings = 5; /* # concurrent xacts needed to sleep */
int wal_retrieve_retry_interval = 5000;
int max_slot_wal_keep_size_mb = -1;
+int wal_compression_method = PGLZ_COMPRESSION_ID;
#ifdef WAL_DEBUG
bool XLOG_DEBUG = false;
diff --git a/src/backend/access/transam/xloginsert.c b/src/backend/access/transam/xloginsert.c
index 1f0e4e01e6..a7f4accb1c 100644
--- a/src/backend/access/transam/xloginsert.c
+++ b/src/backend/access/transam/xloginsert.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "access/xlog_internal.h"
@@ -33,6 +34,10 @@
#include "storage/proc.h"
#include "utils/memutils.h"
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
/* Buffer size required to store a compressed version of backup block image */
#define PGLZ_MAX_BLCKSZ PGLZ_MAX_OUTPUT(BLCKSZ)
@@ -113,7 +118,8 @@ static XLogRecData *XLogRecordAssemble(RmgrId rmid, uint8 info,
XLogRecPtr RedoRecPtr, bool doPageWrites,
XLogRecPtr *fpw_lsn, int *num_fpi);
static bool XLogCompressBackupBlock(char *page, uint16 hole_offset,
- uint16 hole_length, char *dest, uint16 *dlen);
+ uint16 hole_length, char *dest,
+ uint16 *dlen, CompressionId compression);
/*
* Begin constructing a WAL record. This must be called before the
@@ -630,11 +636,15 @@ XLogRecordAssemble(RmgrId rmid, uint8 info,
*/
if (wal_compression)
{
+ bimg.compression_method = PGLZ_COMPRESSION_ID;
+#ifdef HAVE_LIBLZ4
+ bimg.compression_method = wal_compression_method;
+#endif
is_compressed =
XLogCompressBackupBlock(page, bimg.hole_offset,
cbimg.hole_length,
regbuf->compressed_page,
- &compressed_len);
+ &compressed_len, bimg.compression_method);
}
/*
@@ -827,7 +837,7 @@ XLogRecordAssemble(RmgrId rmid, uint8 info,
*/
static bool
XLogCompressBackupBlock(char *page, uint16 hole_offset, uint16 hole_length,
- char *dest, uint16 *dlen)
+ char *dest, uint16 *dlen, CompressionId compression)
{
int32 orig_len = BLCKSZ - hole_length;
int32 len;
@@ -853,12 +863,33 @@ XLogCompressBackupBlock(char *page, uint16 hole_offset, uint16 hole_length,
else
source = page;
+ if (compression == LZ4_COMPRESSION_ID)
+ {
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ len = LZ4_compress_fast(source, dest, orig_len, PGLZ_MAX_BLCKSZ, 1);
+#endif
+ }
+ else if (compression == PGLZ_COMPRESSION_ID)
+ {
+ len = pglz_compress(source, orig_len, dest, PGLZ_strategy_default);
+ }
+ else
+ {
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("unknown compression method requested")));
+ }
+
+
/*
- * We recheck the actual size even if pglz_compress() reports success and
+ * We recheck the actual size even if compression reports success and
* see if the number of bytes saved by compression is larger than the
* length of extra data needed for the compressed version of block image.
*/
- len = pglz_compress(source, orig_len, dest, PGLZ_strategy_default);
if (len >= 0 &&
len + extra_bytes < orig_len)
{
diff --git a/src/backend/access/transam/xlogreader.c b/src/backend/access/transam/xlogreader.c
index a63ad8cfd0..392ffb0343 100644
--- a/src/backend/access/transam/xlogreader.c
+++ b/src/backend/access/transam/xlogreader.c
@@ -33,6 +33,10 @@
#include "utils/memutils.h"
#endif
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
static void report_invalid_record(XLogReaderState *state, const char *fmt,...)
pg_attribute_printf(2, 3);
static bool allocate_recordbuf(XLogReaderState *state, uint32 reclength);
@@ -1291,6 +1295,7 @@ DecodeXLogRecord(XLogReaderState *state, XLogRecord *record, char **errormsg)
{
COPY_HEADER_FIELD(&blk->bimg_len, sizeof(uint16));
COPY_HEADER_FIELD(&blk->hole_offset, sizeof(uint16));
+ COPY_HEADER_FIELD(&blk->compression_method, sizeof(uint8));
COPY_HEADER_FIELD(&blk->bimg_info, sizeof(uint8));
blk->apply_image = ((blk->bimg_info & BKPIMAGE_APPLY) != 0);
@@ -1565,8 +1570,34 @@ RestoreBlockImage(XLogReaderState *record, uint8 block_id, char *page)
if (bkpb->bimg_info & BKPIMAGE_IS_COMPRESSED)
{
/* If a backup block image is compressed, decompress it */
- if (pglz_decompress(ptr, bkpb->bimg_len, tmp.data,
- BLCKSZ - bkpb->hole_length, true) < 0)
+ int32 decomp_result = -1;
+ if (bkpb->compression_method == PGLZ_COMPRESSION_ID)
+ {
+ decomp_result = pglz_decompress(ptr, bkpb->bimg_len, tmp.data,
+ BLCKSZ - bkpb->hole_length, true);
+ }
+ else if (bkpb->compression_method == LZ4_COMPRESSION_ID)
+ {
+#ifndef HAVE_LIBLZ4
+ report_invalid_record(record, "image at %X/%X is compressed with "
+ "lz4 not compiled into this build, block %d",
+ (uint32) (record->ReadRecPtr >> 32),
+ (uint32) record->ReadRecPtr,
+ block_id);
+ return false;
+#else
+ decomp_result = LZ4_decompress_safe(ptr, tmp.data, bkpb->bimg_len, BLCKSZ);
+#endif
+ }
+ else
+ {
+ report_invalid_record(record, "image at %X/%X is compressed with unknown codec, block %d",
+ (uint32) (record->ReadRecPtr >> 32),
+ (uint32) record->ReadRecPtr,
+ block_id);
+ return false;
+ }
+ if ( decomp_result < 0)
{
report_invalid_record(record, "invalid compressed image at %X/%X, block %d",
(uint32) (record->ReadRecPtr >> 32),
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index bbaf037bc6..8ad0267f30 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -483,6 +484,12 @@ const struct config_enum_entry ssl_protocol_versions_info[] = {
{NULL, 0, false}
};
+const struct config_enum_entry wal_compression_options[] = {
+ {"pglz", PGLZ_COMPRESSION_ID, false},
+ {"lz4", LZ4_COMPRESSION_ID, false},
+ {NULL, 0, false}
+};
+
StaticAssertDecl(lengthof(ssl_protocol_versions_info) == (PG_TLS1_3_VERSION + 2),
"array length mismatch");
@@ -4679,6 +4686,16 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"wal_compression_method", PGC_SUSET, WAL_SETTINGS,
+ gettext_noop("Set the method used to compress full page images in the WAL."),
+ NULL
+ },
+ &wal_compression_method,
+ PGLZ_COMPRESSION_ID, wal_compression_options,
+ NULL, NULL, NULL
+ },
+
{
{"dynamic_shared_memory_type", PGC_POSTMASTER, RESOURCES_MEM,
gettext_noop("Selects the dynamic shared memory implementation used."),
diff --git a/src/include/access/xlog.h b/src/include/access/xlog.h
index 221af87e71..5099c07b63 100644
--- a/src/include/access/xlog.h
+++ b/src/include/access/xlog.h
@@ -11,6 +11,7 @@
#ifndef XLOG_H
#define XLOG_H
+#include "access/compressamapi.h"
#include "access/rmgr.h"
#include "access/xlogdefs.h"
#include "access/xloginsert.h"
@@ -175,6 +176,7 @@ typedef enum RecoveryState
} RecoveryState;
extern PGDLLIMPORT int wal_level;
+extern PGDLLIMPORT int wal_compression_method;
/* Is WAL archiving enabled (always or only while server is running normally)? */
#define XLogArchivingActive() \
diff --git a/src/include/access/xlogreader.h b/src/include/access/xlogreader.h
index 0b6d00dd7d..791b6f435c 100644
--- a/src/include/access/xlogreader.h
+++ b/src/include/access/xlogreader.h
@@ -133,6 +133,7 @@ typedef struct
bool apply_image; /* has image that should be restored */
char *bkp_image;
uint16 hole_offset;
+ uint8 compression_method;
uint16 hole_length;
uint16 bimg_len;
uint8 bimg_info;
diff --git a/src/include/access/xlogrecord.h b/src/include/access/xlogrecord.h
index 2f0c8bf589..41fe34c17c 100644
--- a/src/include/access/xlogrecord.h
+++ b/src/include/access/xlogrecord.h
@@ -129,9 +129,10 @@ typedef struct XLogRecordBlockHeader
*/
typedef struct XLogRecordBlockImageHeader
{
- uint16 length; /* number of page image bytes */
- uint16 hole_offset; /* number of bytes before "hole" */
- uint8 bimg_info; /* flag bits, see below */
+ uint16 length; /* number of page image bytes */
+ uint16 hole_offset; /* number of bytes before "hole" */
+ uint8 compression_method; /* compression method used for image */
+ uint8 bimg_info; /* flag bits, see below */
/*
* If BKPIMAGE_HAS_HOLE and BKPIMAGE_IS_COMPRESSED, an
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 3a8d0ac09c..a984eeeadf 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,10 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-{ oid => '4225', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+{ oid => '8022', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
descr => 'pglz compression access method',
amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
-{ oid => '4226', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+{ oid => '8023', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
descr => 'lz4 compression access method',
amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d851619543..3148285f45 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,11 +941,11 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
-{ oid => '4388', descr => 'pglz compression access method handler',
+{ oid => '8024', descr => 'pglz compression access method handler',
proname => 'pglzhandler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'pglzhandler' },
-{ oid => '4389', descr => 'lz4 compression access method handler',
+{ oid => '8021', descr => 'lz4 compression access method handler',
proname => 'lz4handler', provolatile => 'v',
prorettype => 'compression_am_handler', proargtypes => 'internal',
prosrc => 'lz4handler' },
@@ -7056,7 +7056,7 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
-{ oid => '2228',
+{ oid => '8025',
descr => 'compression method for the compressed datum',
proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
proargtypes => 'any', prosrc => 'pg_column_compression' },
--
2.24.3 (Apple Git-128)
Re: [HACKERS] Custom compression methods
The most recent patch doesn't compile --without-lz4:
compress_lz4.c:191:17: error: ‘lz4_cmcheck’ undeclared here (not in a function)
.datum_check = lz4_cmcheck,
...
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 (
- f1 text COMPRESSION lz4
+ f1 text
);You added pg_dump --no-compression, but the --help isn't updated. I think
there should also be an option for pg_restore, like --no-tablespaces. And I
think there should be a GUC for default_compression, like
default_table_access_method, so one can restore into an alternate compression
by setting PGOPTIONS=-cdefault_compression=lz4.
I'd like to be able to make all compressible columns of a table use a
non-default compression (except those which cannot), without having to use
\gexec... We have tables with up to 1600 columns. So a GUC would allow that.
Previously (on separate threads) I wondered whether pg_dump
--no-table-access-method was needed - maybe that be sufficient for this case,
too, but I think it should be possible to separately avoid restoring
compression AM and AM "proper". So maybe it'd be like --no-tableam=compress
--no-tableam=storage or --no-tableam-all.
Some language fixes:
Subject: [PATCH v16 1/6] Built-in compression method
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,8 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ Partitions inherits the compression method of the parent for each column
+ however we can set different compression method for each partition.Should say:
+ By default, each column in a partition inherits the compression method from its parent table,
+ however a different compression method can be set for each partition.+++ b/doc/src/sgml/ref/create_table.sgml+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be coppied. The default
+ behavior is to exclude compression method, resulting in the copied
+ column will have the default compression method if the column type is
+ compressible.Say:
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the
+ columns having the default compression method.+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available built-in compression methods. The available
+ options are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method is not sepcified for the compressible type then it will
+ have the default compression method. The default compression method is
+ <literal>pglz</literal>.Say "The compression method can be set from available compression methods" (or
remove this sentence).
Say "The available BUILT-IN methods are ..."
sepcified => specified
+
+ /*
+ * No point in wasting a palloc cycle if value size is out of the allowed
+ * range for compressionsay "outside the allowed range"
+ if (pset.sversion >= 120000 &&
+ if (pset.sversion >= 120000 &&A couple places that need to say >= 14
Subject: [PATCH v16 2/6] alter table set compression
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ This clause adds compression to a column. Compression method can be set
+ from available built-in compression methods. The available built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.Should say "The compression method can be set to any available method. The
built in methods are >PGLZ< or >LZ<"
That fixes grammar, and correction that it's possible to set to an available
method other than what's "built-in".
+++ b/src/include/commands/event_trigger.h
@@ -32,7 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
-
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*This is losing a useful newline.
Subject: [PATCH v16 4/6] Create custom compression methods
+ This clause adds compression to a column. Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or it can
+ be set from the available built-in compression methods. The available
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ The PRESERVE list contains list of compression methods used on the column
+ and determines which of them should be kept on the column. Without
+ PRESERVE or if all the previous compression methods are not preserved then
+ the table will be rewritten. If PRESERVE ALL is specified then all the
+ previous methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index f404dd1088..ade3989d75 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -999,11 +999,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
+ could be created with <xref linkend="sql-create-access-method"/> or it can
+ be set from the available built-in compression methods. The availableremove this first "built-in" ?
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+GetCompressionAmRoutineByAmId(Oid amoid)
...
+ /* Check if it's an index access method as opposed to some other AM */
+ if (amform->amtype != AMTYPE_COMPRESSION)
+ ereport(ERROR,
+ (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
+ errmsg("access method \"%s\" is not of type %s",
+ NameStr(amform->amname), "INDEX")));
...
+ errmsg("index access method \"%s\" does not have a handler",In 3 places, the comment and code should say "COMPRESSION" right ?
Subject: [PATCH v16 6/6] Support compression methods options
+ If compression method has options they could be specified with
+ <literal>WITH</literal> parameter.If *the* compression method has options, they *can* be specified with *the* ...
@@ -1004,7 +1004,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
+ method is <literal>pglz</literal>. If the compression method has options
+ they could be specified by <literal>WITH</literal>
+ parameter.same
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);Don't you need to say "else: error: unknown compression option" ?
+ /*
+ * Compression option must be only valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+should say "need be valid only if .."
--
Justin
--
Justin
Re: [HACKERS] Custom compression methods
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The most recent patch doesn't compile --without-lz4:
compress_lz4.c:191:17: error: ‘lz4_cmcheck’ undeclared here (not in a function)
.datum_check = lz4_cmcheck,
...And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );You added pg_dump --no-compression, but the --help isn't updated. I think
there should also be an option for pg_restore, like --no-tablespaces. And I
think there should be a GUC for default_compression, like
default_table_access_method, so one can restore into an alternate compression
by setting PGOPTIONS=-cdefault_compression=lz4.I'd like to be able to make all compressible columns of a table use a
non-default compression (except those which cannot), without having to use
\gexec... We have tables with up to 1600 columns. So a GUC would allow that.Previously (on separate threads) I wondered whether pg_dump
--no-table-access-method was needed - maybe that be sufficient for this case,
too, but I think it should be possible to separately avoid restoring
compression AM and AM "proper". So maybe it'd be like --no-tableam=compress
--no-tableam=storage or --no-tableam-all.Some language fixes:
Subject: [PATCH v16 1/6] Built-in compression method
+++ b/doc/src/sgml/ddl.sgml @@ -3762,6 +3762,8 @@ CREATE TABLE measurement ( <productname>PostgreSQL</productname> tables (or, possibly, foreign tables). It is possible to specify a tablespace and storage parameters for each partition separately. + Partitions inherits the compression method of the parent for each column + however we can set different compression method for each partition.Should say: + By default, each column in a partition inherits the compression method from its parent table, + however a different compression method can be set for each partition.+++ b/doc/src/sgml/ref/create_table.sgml+ <varlistentry> + <term><literal>INCLUDING COMPRESSION</literal></term> + <listitem> + <para> + Compression method of the columns will be coppied. The default + behavior is to exclude compression method, resulting in the copied + column will have the default compression method if the column type is + compressible.Say: + Compression method of the columns will be copied. The default + behavior is to exclude compression methods, resulting in the + columns having the default compression method.+ <varlistentry> + <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term> + <listitem> + <para> + This clause adds the compression method to a column. Compression method + can be set from the available built-in compression methods. The available + options are <literal>pglz</literal> and <literal>lz4</literal>. If the + compression method is not sepcified for the compressible type then it will + have the default compression method. The default compression method is + <literal>pglz</literal>.Say "The compression method can be set from available compression methods" (or
remove this sentence).
Say "The available BUILT-IN methods are ..."
sepcified => specified+ + /* + * No point in wasting a palloc cycle if value size is out of the allowed + * range for compressionsay "outside the allowed range"
+ if (pset.sversion >= 120000 && + if (pset.sversion >= 120000 &&A couple places that need to say >= 14
Subject: [PATCH v16 2/6] alter table set compression
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> + This clause adds compression to a column. Compression method can be set + from available built-in compression methods. The available built-in + methods are <literal>pglz</literal> and <literal>lz4</literal>.Should say "The compression method can be set to any available method. The
built in methods are >PGLZ< or >LZ<"
That fixes grammar, and correction that it's possible to set to an available
method other than what's "built-in".+++ b/src/include/commands/event_trigger.h @@ -32,7 +32,7 @@ typedef struct EventTriggerData #define AT_REWRITE_ALTER_PERSISTENCE 0x01 #define AT_REWRITE_DEFAULT_VAL 0x02 #define AT_REWRITE_COLUMN_REWRITE 0x04 - +#define AT_REWRITE_ALTER_COMPRESSION 0x08 /*This is losing a useful newline.
Subject: [PATCH v16 4/6] Create custom compression methods
+ This clause adds compression to a column. Compression method + could be created with <xref linkend="sql-create-access-method"/> or it can + be set from the available built-in compression methods. The available + built-in methods are <literal>pglz</literal> and <literal>lz4</literal>. + The PRESERVE list contains list of compression methods used on the column + and determines which of them should be kept on the column. Without + PRESERVE or if all the previous compression methods are not preserved then + the table will be rewritten. If PRESERVE ALL is specified then all the + previous methods will be preserved and the table will not be rewritten. </para> </listitem> </varlistentry>diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index f404dd1088..ade3989d75 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -999,11 +999,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM + could be created with <xref linkend="sql-create-access-method"/> or it can + be set from the available built-in compression methods. The availableremove this first "built-in" ?
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+GetCompressionAmRoutineByAmId(Oid amoid) ... + /* Check if it's an index access method as opposed to some other AM */ + if (amform->amtype != AMTYPE_COMPRESSION) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("access method \"%s\" is not of type %s", + NameStr(amform->amname), "INDEX"))); ... + errmsg("index access method \"%s\" does not have a handler",In 3 places, the comment and code should say "COMPRESSION" right ?
Subject: [PATCH v16 6/6] Support compression methods options
+ If compression method has options they could be specified with + <literal>WITH</literal> parameter.If *the* compression method has options, they *can* be specified with *the* ...
@@ -1004,7 +1004,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM + method is <literal>pglz</literal>. If the compression method has options + they could be specified by <literal>WITH</literal> + parameter.same
+static void * +lz4_cminitstate(List *options) +{ + int32 *acceleration = palloc(sizeof(int32)); + + /* initialize with the default acceleration */ + *acceleration = 1; + + if (list_length(options) > 0) + { + ListCell *lc; + + foreach(lc, options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "acceleration") == 0) + *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);Don't you need to say "else: error: unknown compression option" ?
+ /* + * Compression option must be only valid if we are updating the compression + * method. + */ + Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression)); +should say "need be valid only if .."
Thanks for the review, I will work on these and respond along with the
updated patches.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The most recent patch doesn't compile --without-lz4:
compress_lz4.c:191:17: error: ‘lz4_cmcheck’ undeclared here (not in a function)
.datum_check = lz4_cmcheck,
...
My bad, fixed this.
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );
I did not get this? pg_upgrade check is passing for me.
You added pg_dump --no-compression, but the --help isn't updated.
Fixed.
I think
there should also be an option for pg_restore, like --no-tablespaces. And I
think there should be a GUC for default_compression, like
default_table_access_method, so one can restore into an alternate compression
by setting PGOPTIONS=-cdefault_compression=lz4.I'd like to be able to make all compressible columns of a table use a
non-default compression (except those which cannot), without having to use
\gexec... We have tables with up to 1600 columns. So a GUC would allow that.Previously (on separate threads) I wondered whether pg_dump
--no-table-access-method was needed - maybe that be sufficient for this case,
too, but I think it should be possible to separately avoid restoring
compression AM and AM "proper". So maybe it'd be like --no-tableam=compress
--no-tableam=storage or --no-tableam-all.
I will put more thought into this and respond separately.
Some language fixes:
Subject: [PATCH v16 1/6] Built-in compression method
+++ b/doc/src/sgml/ddl.sgml @@ -3762,6 +3762,8 @@ CREATE TABLE measurement ( <productname>PostgreSQL</productname> tables (or, possibly, foreign tables). It is possible to specify a tablespace and storage parameters for each partition separately. + Partitions inherits the compression method of the parent for each column + however we can set different compression method for each partition.Should say: + By default, each column in a partition inherits the compression method from its parent table, + however a different compression method can be set for each partition.
Done
+++ b/doc/src/sgml/ref/create_table.sgml+ <varlistentry> + <term><literal>INCLUDING COMPRESSION</literal></term> + <listitem> + <para> + Compression method of the columns will be coppied. The default + behavior is to exclude compression method, resulting in the copied + column will have the default compression method if the column type is + compressible.Say: + Compression method of the columns will be copied. The default + behavior is to exclude compression methods, resulting in the + columns having the default compression method.
Done
+ <varlistentry> + <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term> + <listitem> + <para> + This clause adds the compression method to a column. Compression method + can be set from the available built-in compression methods. The available + options are <literal>pglz</literal> and <literal>lz4</literal>. If the + compression method is not sepcified for the compressible type then it will + have the default compression method. The default compression method is + <literal>pglz</literal>.Say "The compression method can be set from available compression methods" (or
remove this sentence).
Say "The available BUILT-IN methods are ..."
sepcified => specified
Done
+ + /* + * No point in wasting a palloc cycle if value size is out of the allowed + * range for compressionsay "outside the allowed range"
+ if (pset.sversion >= 120000 && + if (pset.sversion >= 120000 &&A couple places that need to say >= 14
Fixed
Subject: [PATCH v16 2/6] alter table set compression
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> + This clause adds compression to a column. Compression method can be set + from available built-in compression methods. The available built-in + methods are <literal>pglz</literal> and <literal>lz4</literal>.Should say "The compression method can be set to any available method. The
built in methods are >PGLZ< or >LZ<"
That fixes grammar, and correction that it's possible to set to an available
method other than what's "built-in".
Done
+++ b/src/include/commands/event_trigger.h @@ -32,7 +32,7 @@ typedef struct EventTriggerData #define AT_REWRITE_ALTER_PERSISTENCE 0x01 #define AT_REWRITE_DEFAULT_VAL 0x02 #define AT_REWRITE_COLUMN_REWRITE 0x04 - +#define AT_REWRITE_ALTER_COMPRESSION 0x08 /*This is losing a useful newline.
Fixed
Subject: [PATCH v16 4/6] Create custom compression methods
+ This clause adds compression to a column. Compression method + could be created with <xref linkend="sql-create-access-method"/> or it can + be set from the available built-in compression methods. The available + built-in methods are <literal>pglz</literal> and <literal>lz4</literal>. + The PRESERVE list contains list of compression methods used on the column + and determines which of them should be kept on the column. Without + PRESERVE or if all the previous compression methods are not preserved then + the table will be rewritten. If PRESERVE ALL is specified then all the + previous methods will be preserved and the table will not be rewritten. </para> </listitem> </varlistentry>diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml index f404dd1088..ade3989d75 100644 --- a/doc/src/sgml/ref/create_table.sgml +++ b/doc/src/sgml/ref/create_table.sgml @@ -999,11 +999,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM + could be created with <xref linkend="sql-create-access-method"/> or it can + be set from the available built-in compression methods. The availableremove this first "built-in" ?
Done
+ built-in methods are <literal>pglz</literal> and <literal>lz4</literal>.
+GetCompressionAmRoutineByAmId(Oid amoid) ... + /* Check if it's an index access method as opposed to some other AM */ + if (amform->amtype != AMTYPE_COMPRESSION) + ereport(ERROR, + (errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE), + errmsg("access method \"%s\" is not of type %s", + NameStr(amform->amname), "INDEX"))); ... + errmsg("index access method \"%s\" does not have a handler",In 3 places, the comment and code should say "COMPRESSION" right ?
Fixed, along with some other refactoring around this code.
Subject: [PATCH v16 6/6] Support compression methods options
+ If compression method has options they could be specified with + <literal>WITH</literal> parameter.If *the* compression method has options, they *can* be specified with *the* ...
Done
@@ -1004,7 +1004,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM + method is <literal>pglz</literal>. If the compression method has options + they could be specified by <literal>WITH</literal> + parameter.same
Done
+static void * +lz4_cminitstate(List *options) +{ + int32 *acceleration = palloc(sizeof(int32)); + + /* initialize with the default acceleration */ + *acceleration = 1; + + if (list_length(options) > 0) + { + ListCell *lc; + + foreach(lc, options) + { + DefElem *def = (DefElem *) lfirst(lc); + + if (strcmp(def->defname, "acceleration") == 0) + *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);Don't you need to say "else: error: unknown compression option" ?
Done
+ /* + * Compression option must be only valid if we are updating the compression + * method. + */ + Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression)); +should say "need be valid only if .."
Changed.
Apart from this, I have also done some refactoring and comment improvement.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v17-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v17-0003-Add-support-for-PRESERVE.patchDownload
From fb0d6204542c37cbd73778dc91ea96231d61992b Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:15:20 +0530
Subject: [PATCH v17 3/6] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. For
supporting this the column will maintain the dependency with all
the supported compression methods. So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 9 +-
src/backend/catalog/objectaddress.c | 1 +
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 266 +++++++++++++++++++++
src/backend/commands/tablecmds.c | 117 +++++----
src/backend/executor/nodeModifyTable.c | 13 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/create_index.out | 56 +++--
src/test/regress/sql/compression.sql | 9 +
19 files changed, 552 insertions(+), 96 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c4b53ec1c5..a2f3aff235 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. The Compression method can be
set to any available compression method. The built-in methods are
<literal>pglz</literal> and <literal>lz4</literal>.
+ The PRESERVE list contains list of compression methods used on the column
+ and determines which of them should be kept on the column. Without
+ PRESERVE or if all the previous compression methods are not preserved then
+ the table will be rewritten. If PRESERVE ALL is specified then all the
+ previous methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index a5eccdffd0..636364995a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 429791694f..3c7aea0df4 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a4d11d2dc3
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,266 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency upon.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8944ebc586..340ddb913c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -390,6 +390,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -530,7 +531,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2413,16 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2459,7 +2477,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2710,12 +2729,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4800,7 +4819,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6294,7 +6314,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6469,6 +6490,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6616,6 +6638,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11751,7 +11795,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11866,6 +11911,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15044,24 +15094,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15094,11 +15141,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17823,32 +17875,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 64a396a218..0b474e81e6 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1933,6 +1934,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1941,7 +1943,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop for all the attributes in the tuple and check if any of the
* attribute is compressed in the source tuple and its compression method
- * is not same as the target compression method then we need to decompress
+ * is not supported by the target attribute then we need to decompress
* it.
*/
for (i = 0; i < natts; i++)
@@ -1962,11 +1964,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 7d63cee5a5..321da2accd 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,7 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2952,6 +2952,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5623,6 +5635,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index de26dce6d6..ade226efdc 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2586,7 +2586,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2606,6 +2606,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3678,6 +3688,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 55b35e6f13..2e60bb750e 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2853,7 +2853,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2871,6 +2871,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4215,6 +4225,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9683062216..22bd0e87a3 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -593,7 +593,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2258,12 +2260,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3386,7 +3388,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3441,13 +3443,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e5184933b7..4bbbf6370a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1da16ebc0a..8178a5124b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,6 +2098,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 2d76a85336..d1880dfc5e 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index c8d1f20b3b..885ff5ec5c 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -478,6 +478,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index af79aa6534..c8135debbb 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3f47e5b576..50b9463955 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -202,12 +202,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index fc6afab58a..d8300bdcdd 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index bdb85a650b..2ee1027ac6 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -87,6 +87,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v17-0001-Built-in-compression-method.patchapplication/octet-stream; name=v17-0001-Built-in-compression-method.patchDownload
From 4dbf0bb9d5277c4f1cfa2ae7179ac0967c061ab9 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v17 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 2 +
doc/src/sgml/ref/create_table.sgml | 27 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++
.../access/compression/compress_pglz.c | 131 +++++++++
.../access/compression/compressamapi.c | 61 +++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 ++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 41 ++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 ++-
src/include/access/compressamapi.h | 65 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 179 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 77 ++++++
src/tools/pgindent/typedefs.list | 1 +
77 files changed, 1992 insertions(+), 667 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index 07529825d1..46b4946325 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -866,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8601,6 +8604,35 @@ fi
+#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12092,6 +12124,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13295,6 +13380,20 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 7f855783f4..c13fa5c0a0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -999,6 +999,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1186,6 +1193,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1400,6 +1415,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index c12a32c8c7..5ec3d995f8 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,8 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ Partitions inherits the compression method of the parent for each column
+ however we can set different compression method for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..43c417b809 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,18 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be coppied. The default
+ behavior is to exclude compression method, resulting in the copied
+ column will have the default compression method if the column type is
+ compressible.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +994,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. Compression method
+ can be set from the available compression methods. The available BUILT-IN
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If the compression method is not sepcified for the compressible type then
+ it will have the default compression method. The default compression
+ method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 17e50de530..2b8a6de6ed 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 44c37edcbb..d66d733da6 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -438,28 +439,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -473,22 +493,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 634016b9b7..3024670414 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 25a81e5ec6..72bf23f712 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 30c30cf3a2..9aac155e4a 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..c4814ef911
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..e8b035d138
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..578be6f1dd
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000000..663102c8d2
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 739b6ae990..b33c925a4b 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index a7ed93fdc1..8162ecd612 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 66fdaf67b1..94278786d9 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 51b5c4f7f6..e5554e2e33 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1706,6 +1708,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 731610c701..7e6310b6f7 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index f1850436bd..e5452614f7 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 6f05ee715b..fa8deda08b 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 009896bcee..6150f24135 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -581,6 +581,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index cfc63915f3..2e1d990f9e 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -486,6 +486,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1fa9f19f08..fe33248589 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2408,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2457,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2703,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6233,6 +6275,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11752,6 +11806,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17631,3 +17701,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ab3d655e60..64a396a218 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1915,6 +1918,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2120,6 +2195,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 70f8b718e0..7d63cee5a5 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,6 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 541e0e6b48..de26dce6d6 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2586,6 +2586,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 963f71e99d..d7a93f9360 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3848,6 +3848,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index d78b16ed1d..55b35e6f13 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2853,6 +2853,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 8f341ac006..39467aa2e0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -593,6 +593,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -628,9 +630,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3370,11 +3372,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3383,8 +3386,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3429,6 +3432,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3659,6 +3670,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15110,6 +15122,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15628,6 +15641,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 89ee990599..e5184933b7 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 6b0a59efaf..89b4a9bf60 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index 99a93271fe..e177ed77d8 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 9300d19e0c..d6a36f7e97 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "common/hex_decode.h"
@@ -5275,6 +5277,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 1ab98a2286..f17222edaa 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -386,6 +386,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1046,6 +1047,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8611,6 +8613,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8696,6 +8699,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (!dopt->binary_upgrade && fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8714,7 +8726,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8741,6 +8758,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8769,6 +8787,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15796,6 +15815,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15820,6 +15840,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15855,6 +15878,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods && !dopt->binary_upgrade &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index d7f77f1d3e..c08b1df75d 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 14150d05a9..e94dceb28d 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..8226ae0596
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 0e92acc546..4e932bc067 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 71e3ca2ef2..e13e34cac3 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 0f051277a6..a9282a8804 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index bc73e88a1a..0ffc844995 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index cdf75a2380..3e6ed706ac 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For the compressible atttypid this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible atttypid this must always be an invalid Oid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 139f4a08bd..d97b2d8846 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7047,6 +7056,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7217,6 +7230,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 62018f063a..d8b1cbba8c 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -619,6 +619,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1133ae1143..2d76a85336 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 0c48d2a519..df439c8c38 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 3684f87a88..c8d1f20b3b 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -509,6 +509,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 48a79a7657..78b7f76215 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 71dcdf2889..b8113673e7 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index ddaa9e8e18..0749dc4bf3 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index c48f47e930..4e7cad3daf 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..2f53c0e489
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,179 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+DROP MATERIALIZED VIEW mv;
+DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..e078818496 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6293ab57bc..b79b0a2a7f 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3019,11 +3019,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3039,11 +3039,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3070,11 +3070,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 7bfeaf85f0..197c312ed6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e0e1ef71dd..2407ca945b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,6 +116,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 081fce32e7..152c8cb5b7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -203,3 +203,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..292c7258ba
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,77 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+DROP MATERIALIZED VIEW mv;
+DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 9cd047ba25..d9ef8f4136 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -394,6 +394,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
v17-0002-alter-table-set-compression.patchapplication/octet-stream; name=v17-0002-alter-table-set-compression.patchDownload
From ea7985a5e9589de86b7e680ac80fd4156aa16ee5 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v17 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 +++++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/sql/compression.sql | 21 +++
8 files changed, 272 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..c4b53ec1c5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. The Compression method can be
+ set to any available compression method. The built-in methods are
+ <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fe33248589..8944ebc586 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3860,6 +3862,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4387,7 +4390,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4795,6 +4799,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5419,6 +5427,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7659,6 +7670,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7674,7 +7750,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7738,47 +7813,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -14999,6 +15035,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 39467aa2e0..9683062216 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2257,6 +2257,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 3a43c09bf6..1da16ebc0a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,7 +2093,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index 407fd6a978..1800fd8d0c 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 78b7f76215..af79aa6534 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1856,7 +1856,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 2f53c0e489..3f47e5b576 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -143,6 +143,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 292c7258ba..bdb85a650b 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -66,6 +66,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v17-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v17-0004-Create-custom-compression-methods.patchDownload
From 3bedb10bcb3b5903f97340cd254adce0b9bc2352 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:21:30 +0530
Subject: [PATCH v17 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
---
doc/src/sgml/ref/alter_table.sgml | 9 +--
doc/src/sgml/ref/create_access_method.sgml | 13 ++--
doc/src/sgml/ref/create_table.sgml | 3 +-
src/backend/access/common/detoast.c | 59 +++++++++++++++++--
src/backend/access/common/toast_internals.c | 19 +++++-
src/backend/access/compression/compress_lz4.c | 21 +++----
.../access/compression/compress_pglz.c | 20 +++----
.../access/compression/compressamapi.c | 27 +++++++++
src/backend/access/index/amapi.c | 50 ++++++++++++----
src/backend/commands/amcmds.c | 5 ++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 +
src/bin/psql/tab-complete.c | 6 ++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 16 +++--
src/include/access/detoast.h | 8 +++
src/include/access/toast_internals.h | 15 +++++
src/test/regress/expected/compression.out | 41 ++++++++++++-
src/test/regress/sql/compression.sql | 11 ++++
21 files changed, 275 insertions(+), 62 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index a2f3aff235..104bae26f6 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,14 +390,15 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. The Compression method can be
- set to any available compression method. The built-in methods are
- <literal>pglz</literal> and <literal>lz4</literal>.
+ This clause adds compression to a column. Compression method could be
+ created with <xref linkend="sql-create-access-method"/> or it can
+ be set to any available compression method. The available built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
The PRESERVE list contains list of compression methods used on the column
and determines which of them should be kept on the column. Without
PRESERVE or if all the previous compression methods are not preserved then
the table will be rewritten. If PRESERVE ALL is specified then all the
- previous methods will be preserved and the table will not be rewritten.
+ previous methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 43c417b809..1bf7d4a674 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -999,7 +999,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. Compression method
- can be set from the available compression methods. The available BUILT-IN
+ could be created with <xref linkend="sql-create-access-method"/> or can be
+ set from the available compression methods. The available BUILT-IN
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not sepcified for the compressible type then
it will have the default compression method. The default compression
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d66d733da6..f6c48ac955 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -438,13 +438,44 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -458,10 +489,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -477,9 +519,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -493,16 +537,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 72bf23f712..cd91aefef7 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index e8b035d138..34b0dc1002 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -30,7 +30,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -45,10 +45,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -56,7 +56,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -68,7 +68,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -81,9 +81,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -100,7 +100,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -112,9 +113,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 578be6f1dd..d33ea3d2cf 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c8d2..7aea8aad38 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -59,3 +60,29 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index 4e3d7b030e..53bb2665ff 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -46,14 +46,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +64,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +93,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index fa8deda08b..bdd6d0a8ac 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -227,6 +227,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -264,6 +266,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 0b474e81e6..7dbb2b7212 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1968,7 +1968,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* compression method is not supported by the target attribute then
* we need to decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 22bd0e87a3..54ff6fb937 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5254,6 +5254,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index d6a36f7e97..ed8169a01f 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5285,7 +5285,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5315,10 +5314,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f17222edaa..dedd126c13 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13036,6 +13036,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 8178a5124b..9bfa66ff09 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 85b4766016..b4185e946b 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -284,6 +284,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0596..913a633d83 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -26,18 +26,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -61,5 +68,6 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 86bad7e78c..e6b3ec90b5 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index e13e34cac3..81944912d2 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_compression */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 50b9463955..19fdc7dcca 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -236,13 +236,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
@@ -271,3 +309,4 @@ SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP ACCESS METHOD pglz2;
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 2ee1027ac6..2beb5dae95 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -96,6 +96,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
@@ -105,3 +115,4 @@ SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP ACCESS METHOD pglz2;
--
2.23.0
v17-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v17-0005-new-compression-method-extension-for-zlib.patchDownload
From e72a1a0bd814a70b65acf522563baac39258bc43 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v17 5/6] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..bf36854ef3
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ lz4
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..aa7c57030d
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.23.0
v17-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v17-0006-Support-compression-methods-options.patchDownload
From 9445f465eaa378c08b47a89fe697d4633fe7a568 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:32:45 +0530
Subject: [PATCH v17 6/6] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
---
contrib/cmzlib/cmzlib.c | 75 +++++++++++--
doc/src/sgml/ref/alter_table.sgml | 8 +-
doc/src/sgml/ref/create_table.sgml | 7 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 +++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 69 +++++++++++-
.../access/compression/compress_pglz.c | 98 +++++++++++++++--
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 ++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++++-
src/backend/commands/foreigncmds.c | 44 --------
src/backend/commands/tablecmds.c | 102 +++++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 20 +++-
src/bin/pg_dump/pg_dump.h | 2 +-
src/include/access/compressamapi.h | 19 +++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 54 +++++++++-
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 20 +++-
35 files changed, 660 insertions(+), 136 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..39fa415a3e 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 104bae26f6..48a471b87c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,14 +386,16 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. Compression method could be
created with <xref linkend="sql-create-access-method"/> or it can
be set to any available compression method. The available built-in
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ methods are <literal>pglz</literal> and <literal>lz4</literal>. If
+ the compression method has options they can be specified with the
+ <literal>WITH</literal> parameter.
The PRESERVE list contains list of compression methods used on the column
and determines which of them should be kept on the column. Without
PRESERVE or if all the previous compression methods are not preserved then
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 1bf7d4a674..d151f9d3f2 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -995,7 +995,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. Compression method
@@ -1004,7 +1004,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not sepcified for the compressible type then
it will have the default compression method. The default compression
- method is <literal>pglz</literal>.
+ method is <literal>pglz</literal>. If the compression method has options
+ they can be specified with the <literal>WITH</literal> parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 2b8a6de6ed..61fc6fbb30 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 3024670414..0cd7bdd730 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index 8ccc228a8c..9f0389a994 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index cd91aefef7..e9b5971928 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 34b0dc1002..df6cebb381 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,6 +16,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -23,6 +24,63 @@
#include "lz4.h"
#endif
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -30,7 +88,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -40,6 +98,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -47,9 +106,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d33ea3d2cf..0396729c70 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index b33c925a4b..7e399daaa1 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 6bb0c6ed1e..bbc849b541 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index e5554e2e33..89899ba382 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1151,6 +1159,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1409,7 +1418,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 7e6310b6f7..b878b4af7f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index e5452614f7..6cf32fef6a 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index fd5a6eec86..29a8145b1d 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -697,6 +697,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a4d11d2dc3..24be66bf45 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -27,6 +27,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -161,7 +162,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -187,6 +188,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -252,15 +267,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttributes compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index de31ddd1f3..95bdcb1874 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 340ddb913c..0dd470ef22 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -610,6 +610,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -814,6 +815,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -867,8 +870,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -922,8 +926,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2431,16 +2438,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2477,8 +2482,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2728,14 +2732,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6152,6 +6151,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6315,6 +6315,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6325,7 +6326,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7722,13 +7723,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7767,7 +7775,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7858,7 +7888,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15104,9 +15134,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15141,7 +15173,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15151,8 +15184,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15160,8 +15202,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 321da2accd..5955a67058 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2959,6 +2959,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index ade226efdc..982f1673e8 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2611,6 +2611,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 2e60bb750e..74faf0f997 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2878,6 +2878,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 54ff6fb937..65ea7a4d39 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -413,6 +413,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3452,11 +3453,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3464,14 +3471,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 4bbbf6370a..62ef275392 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index dedd126c13..516a91b360 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8703,10 +8703,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8759,6 +8766,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8788,6 +8796,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15844,7 +15853,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -15895,6 +15905,10 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index c08b1df75d..71a9e68017 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,7 +327,7 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ char **attcmoptions; /* per-attribute current compression options */
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 913a633d83..8123cc8cc7 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -37,8 +38,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -49,14 +53,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index 4e932bc067..1dc4aa9623 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 81944912d2..dc35064086 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index f94defff3c..bd192f457e 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3e6ed706ac..55ea9af6bd 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index d1880dfc5e..138ed6976f 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -138,6 +138,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,9 +152,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index c8135debbb..09e5e00d99 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 19fdc7dcca..f489a24441 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -273,6 +273,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
@@ -308,5 +360,5 @@ SELECT length(f1) FROM cmmove3;
(2 rows)
DROP MATERIALIZED VIEW mv;
-DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP TABLE cmdata, cmdata1, cmdata3, cmmove1, cmmove2, cmmove3, cmpart;
DROP ACCESS METHOD pglz2;
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..27aab82462 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 2beb5dae95..f87416a80f 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -106,6 +106,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
@@ -114,5 +132,5 @@ SELECT length(f1) FROM cmmove2;
SELECT length(f1) FROM cmmove3;
DROP MATERIALIZED VIEW mv;
-DROP TABLE cmdata, cmdata1, cmmove1, cmmove2, cmmove3, cmpart;
+DROP TABLE cmdata, cmdata1, cmdata3, cmmove1, cmmove2, cmmove3, cmpart;
DROP ACCESS METHOD pglz2;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.
I think this patch needs to be specifically concerned with pg_upgrade, so I
suggest to not drop your tables and MVs, to allow the pg_upgrade test to check
them. That exposes this issue:
pg_dump: error: Error message from server: ERROR: cache lookup failed for access method 36447
pg_dump: error: The command was: COPY public.cmdata (f1) TO stdout;
pg_dumpall: error: pg_dump failed on database "regression", exiting
waiting for server to shut down.... done
server stopped
pg_dumpall of post-upgrade database cluster failed
I found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | c
But in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:
16755 | pglz2 | pglzhandler | c
In my brief attempt to inspect it, I got this crash:
$ tmp_install/usr/local/pgsql/bin/postgres -D src/bin/pg_upgrade/tmp_check/data &
regression=# SELECT pg_column_compression(f1) FROM cmdata a;
server closed the connection unexpectedly
Thread 1 "postgres" received signal SIGSEGV, Segmentation fault.
__strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120
120 ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory.
(gdb) bt
#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120
#1 0x000055c6049fde62 in cstring_to_text (s=0x0) at varlena.c:193
#2 pg_column_compression () at varlena.c:5335
(gdb) up
#2 pg_column_compression () at varlena.c:5335
5335 PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
(gdb) l
5333 varvalue = (struct varlena *) DatumGetPointer(value);
5334
5335 PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
5336 toast_get_compression_oid(varvalue))));
I guess a missing AM here is a "shouldn't happen" case, but I'd prefer it to be
caught with an elog() (maybe in get_am_name()) or at least an Assert.
--
Justin
Re: [HACKERS] Custom compression methods
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.I think this patch needs to be specifically concerned with pg_upgrade, so I
suggest to not drop your tables and MVs, to allow the pg_upgrade test to check
them. That exposes this issue:
Thanks for the suggestion I will try this.
pg_dump: error: Error message from server: ERROR: cache lookup failed for access method 36447
pg_dump: error: The command was: COPY public.cmdata (f1) TO stdout;
pg_dumpall: error: pg_dump failed on database "regression", exiting
waiting for server to shut down.... done
server stopped
pg_dumpall of post-upgrade database cluster failedI found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | cBut in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:16755 | pglz2 | pglzhandler | c
Yeah, basically we are storing am oid in the compressed data so Oid
must be preserved. I will look into this and fix it.
In my brief attempt to inspect it, I got this crash:
$ tmp_install/usr/local/pgsql/bin/postgres -D src/bin/pg_upgrade/tmp_check/data &
regression=# SELECT pg_column_compression(f1) FROM cmdata a;
server closed the connection unexpectedlyThread 1 "postgres" received signal SIGSEGV, Segmentation fault.
__strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120
120 ../sysdeps/x86_64/multiarch/../strlen.S: No such file or directory.
(gdb) bt
#0 __strlen_sse2 () at ../sysdeps/x86_64/multiarch/../strlen.S:120
#1 0x000055c6049fde62 in cstring_to_text (s=0x0) at varlena.c:193
#2 pg_column_compression () at varlena.c:5335(gdb) up
#2 pg_column_compression () at varlena.c:5335
5335 PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
(gdb) l
5333 varvalue = (struct varlena *) DatumGetPointer(value);
5334
5335 PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
5336 toast_get_compression_oid(varvalue))));I guess a missing AM here is a "shouldn't happen" case, but I'd prefer it to be
caught with an elog() (maybe in get_am_name()) or at least an Assert.
Yeah, this makes sense.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.
I tested specifically pg_upgrade by removing all the DROP table and MV
and it is passing. I don't see the reason why should it fail. I mean
after the upgrade why COMPRESSION lz4 is missing?
I think this patch needs to be specifically concerned with pg_upgrade, so I
suggest to not drop your tables and MVs, to allow the pg_upgrade test to check
them. That exposes this issue:Thanks for the suggestion I will try this.
pg_dump: error: Error message from server: ERROR: cache lookup failed for access method 36447
pg_dump: error: The command was: COPY public.cmdata (f1) TO stdout;
pg_dumpall: error: pg_dump failed on database "regression", exiting
waiting for server to shut down.... done
server stopped
pg_dumpall of post-upgrade database cluster failedI found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | cBut in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:16755 | pglz2 | pglzhandler | c
Yeah, basically we are storing am oid in the compressed data so Oid
must be preserved. I will look into this and fix it.
On further analysis, if we are dumping and restoring then we will
compress the data back while inserting it so why would we need to old
OID. I mean in the new cluster we are inserting data again so it will
be compressed again and now it will store the new OID. Am I missing
something here?
In my brief attempt to inspect it, I got this crash:
$ tmp_install/usr/local/pgsql/bin/postgres -D src/bin/pg_upgrade/tmp_check/data &
regression=# SELECT pg_column_compression(f1) FROM cmdata a;
server closed the connection unexpectedly
I tried to test this after the upgrade but I can get the proper value.
Laptop309pnin:bin dilipkumar$ ./pg_ctl -D
/Users/dilipkumar/Documents/PG/custom_compression/src/bin/pg_upgrade/tmp_check/data.old/
start
waiting for server to start....2021-01-11 11:53:28.153 IST [43412]
LOG: starting PostgreSQL 14devel on x86_64-apple-darwin19.6.0,
compiled by Apple clang version 11.0.3 (clang-1103.0.32.62), 64-bit
2021-01-11 11:53:28.170 IST [43412] LOG: database system is ready to
accept connections
done
server started
Laptop309pnin:bin dilipkumar$ ./psql -d regression
regression[43421]=# SELECT pg_column_compression(f1) FROM cmdata a;
pg_column_compression
-----------------------
lz4
lz4
pglz2
(3 rows)
Manual test: (dump and load on the new cluster)
---------------
postgres[43903]=# CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER
pglzhandler;
CREATE ACCESS METHOD
postgres[43903]=# select oid from pg_am where amname='pglz2';
oid
-------
16384
(1 row)
postgres[43903]=# CREATE TABLE cmdata_test(f1 text COMPRESSION pglz2);
CREATE TABLE
postgres[43903]=# INSERT INTO cmdata_test
VALUES(repeat('1234567890',1000));
INSERT 0 1
postgres[43903]=# SELECT pg_column_compression(f1) FROM cmdata_test;
pg_column_compression
-----------------------
pglz2
(1 row)
Laptop309pnin:bin dilipkumar$ ./pg_dump -d postgres > 1.sql
—restore on new cluster—
postgres[44030]=# select oid from pg_am where amname='pglz2';
oid
-------
16385
(1 row)
postgres[44030]=# SELECT pg_column_compression(f1) FROM cmdata_test;
pg_column_compression
-----------------------
pglz2
(1 row)
You can see on the new cluster the OID of the pglz2 is changed but
there is no issue. Is it possible for you to give me a
self-contained test case to reproduce the issue or a theory that why
it should fail?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote:
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.I tested specifically pg_upgrade by removing all the DROP table and MV
and it is passing. I don't see the reason why should it fail. I mean
after the upgrade why COMPRESSION lz4 is missing?
How did you test it ?
I'm not completely clear how this is intended to work... has it been tested
before ? According to the comments, in binary upgrade mode, there's an ALTER
which is supposed to SET COMPRESSION, but that's evidently not happening.
I found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | cBut in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:16755 | pglz2 | pglzhandler | c
Yeah, basically we are storing am oid in the compressed data so Oid
must be preserved. I will look into this and fix it.On further analysis, if we are dumping and restoring then we will
compress the data back while inserting it so why would we need to old
OID. I mean in the new cluster we are inserting data again so it will
be compressed again and now it will store the new OID. Am I missing
something here?
I'm referring to pg_upgrade which uses pg_dump, but does *not* re-insert data,
but rather recreates catalogs only and then links to the old tables (either
with copy, link, or clone). Test with make -C src/bin/pg_upgrade (which is
included in make check-world).
--
Justin
Re: [HACKERS] Custom compression methods
On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote:
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.I tested specifically pg_upgrade by removing all the DROP table and MV
and it is passing. I don't see the reason why should it fail. I mean
after the upgrade why COMPRESSION lz4 is missing?How did you test it ?
I'm not completely clear how this is intended to work... has it been tested
before ? According to the comments, in binary upgrade mode, there's an ALTER
which is supposed to SET COMPRESSION, but that's evidently not happening.
I am able to reproduce this issue, If I run pg_dump with
binary_upgrade mode then I can see the issue (./pg_dump
--binary-upgrade -d Postgres). Yes you are right that for fixing
this there should be an ALTER..SET COMPRESSION method.
I found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | cBut in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:16755 | pglz2 | pglzhandler | c
Yeah, basically we are storing am oid in the compressed data so Oid
must be preserved. I will look into this and fix it.On further analysis, if we are dumping and restoring then we will
compress the data back while inserting it so why would we need to old
OID. I mean in the new cluster we are inserting data again so it will
be compressed again and now it will store the new OID. Am I missing
something here?I'm referring to pg_upgrade which uses pg_dump, but does *not* re-insert data,
but rather recreates catalogs only and then links to the old tables (either
with copy, link, or clone). Test with make -C src/bin/pg_upgrade (which is
included in make check-world).
Got this as well.
I will fix these two issues and post the updated patch by tomorrow.
Thanks for your findings.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Jan 11, 2021 at 3:40 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote:
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.I tested specifically pg_upgrade by removing all the DROP table and MV
and it is passing. I don't see the reason why should it fail. I mean
after the upgrade why COMPRESSION lz4 is missing?How did you test it ?
I'm not completely clear how this is intended to work... has it been tested
before ? According to the comments, in binary upgrade mode, there's an ALTER
which is supposed to SET COMPRESSION, but that's evidently not happening.I am able to reproduce this issue, If I run pg_dump with
binary_upgrade mode then I can see the issue (./pg_dump
--binary-upgrade -d Postgres). Yes you are right that for fixing
this there should be an ALTER..SET COMPRESSION method.I found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | cBut in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:16755 | pglz2 | pglzhandler | c
Yeah, basically we are storing am oid in the compressed data so Oid
must be preserved. I will look into this and fix it.On further analysis, if we are dumping and restoring then we will
compress the data back while inserting it so why would we need to old
OID. I mean in the new cluster we are inserting data again so it will
be compressed again and now it will store the new OID. Am I missing
something here?I'm referring to pg_upgrade which uses pg_dump, but does *not* re-insert data,
but rather recreates catalogs only and then links to the old tables (either
with copy, link, or clone). Test with make -C src/bin/pg_upgrade (which is
included in make check-world).Got this as well.
I will fix these two issues and post the updated patch by tomorrow.
Thanks for your findings.
I have fixed this issue in the v18 version, please test and let me
know your thoughts. There is one more issue pending from an upgrade
perspective in v18-0003, basically, for the preserved method we need
to restore the dependency as well. I will work on this part and
shared the next version soon.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v18-0002-alter-table-set-compression.patchapplication/octet-stream; name=v18-0002-alter-table-set-compression.patchDownload
From 059492582f9890df7955637c04c693e29a75b071 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v18 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 +++++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/sql/compression.sql | 21 +++
8 files changed, 272 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..c4b53ec1c5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. The Compression method can be
+ set to any available compression method. The built-in methods are
+ <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 53d6e627ae..bc07d93459 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3860,6 +3862,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4387,7 +4390,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4795,6 +4799,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5419,6 +5427,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7659,6 +7670,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7674,7 +7750,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7738,47 +7813,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -14999,6 +15035,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 34db078d48..b1674764ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2306,6 +2306,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9dcab0d2fa..3127665660 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,7 +2093,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 232c9f1248..ffb739f565 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,7 +1875,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4dea269fc3..84d31c781b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -142,6 +142,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0750a6c80e..3b0da88c19 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -65,6 +65,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v18-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v18-0003-Add-support-for-PRESERVE.patchDownload
From 4d52f76bf45f0436a163ac2fba0c992deee3163c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:15:20 +0530
Subject: [PATCH v18 3/6] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. For
supporting this the column will maintain the dependency with all
the supported compression methods. So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/objectaddress.c | 1 +
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 266 +++++++++++++++++++++
src/backend/commands/tablecmds.c | 117 +++++----
src/backend/executor/nodeModifyTable.c | 13 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/create_index.out | 56 +++--
src/test/regress/sql/compression.sql | 9 +
19 files changed, 553 insertions(+), 96 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c4b53ec1c5..081fe078a4 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. The Compression method can be
set to any available compression method. The built-in methods are
<literal>pglz</literal> and <literal>lz4</literal>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b690d8..ea9b90cc80 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da24322d..dd376484b7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..a4d11d2dc3
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,266 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency upon.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ if (compression->preserve != NIL)
+ {
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index bc07d93459..fc80cb5624 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -390,6 +390,7 @@ static bool check_for_column_name_collision(Relation rel, const char *colname,
bool if_not_exists);
static void add_column_datatype_dependency(Oid relid, int32 attnum, Oid typid);
static void add_column_collation_dependency(Oid relid, int32 attnum, Oid collid);
+static void add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid);
static void ATPrepDropNotNull(Relation rel, bool recurse, bool recursing);
static ObjectAddress ATExecDropNotNull(Relation rel, const char *colName, LOCKMODE lockmode);
static void ATPrepSetNotNull(List **wqueue, Relation rel,
@@ -530,7 +531,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +565,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +589,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +868,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2413,16 +2430,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2459,7 +2477,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2710,12 +2729,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4800,7 +4819,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6294,7 +6314,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6469,6 +6490,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6616,6 +6638,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+static void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11751,7 +11795,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11866,6 +11911,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15044,24 +15094,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15094,11 +15141,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17823,32 +17875,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 3fac8b7867..787860f481 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1933,6 +1934,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1941,7 +1943,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop for all the attributes in the tuple and check if any of the
* attribute is compressed in the source tuple and its compression method
- * is not same as the target compression method then we need to decompress
+ * is not supported by the target attribute then we need to decompress
* it.
*/
for (i = 0; i < natts; i++)
@@ -1962,11 +1964,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3ea32323cd..6588c84c4c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,7 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2952,6 +2952,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5640,6 +5652,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 227bb07bbd..3aa6176a30 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,7 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2618,6 +2618,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3693,6 +3703,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 187d3570e5..4f7053d63b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,7 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2883,6 +2883,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4230,6 +4240,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b1674764ee..c9f80c4de4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -594,7 +594,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2307,12 +2309,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3435,7 +3437,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3490,13 +3492,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e2ac159aa2..a2a6b218f3 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 3127665660..5b8a6994cc 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,6 +2098,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 3806a2e562..b107b82916 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ded2ad9692..d630c54efa 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ffb739f565..fd9c6abddf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 84d31c781b..151478bcdd 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -201,12 +201,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index fc6afab58a..d8300bdcdd 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 3b0da88c19..e447316573 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -86,6 +86,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v18-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v18-0005-new-compression-method-extension-for-zlib.patchDownload
From 3b0222e0771b0cd2849e9a8a8a1d04b38c4f58e3 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v18 5/6] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..bf36854ef3
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ lz4
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..aa7c57030d
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.23.0
v18-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v18-0004-Create-custom-compression-methods.patchDownload
From 954d2de5ff327d08caae6f479e944df1f30099c2 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 13 Jan 2021 12:14:40 +0530
Subject: [PATCH v18 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 7 ++-
doc/src/sgml/ref/create_access_method.sgml | 13 ++--
doc/src/sgml/ref/create_table.sgml | 3 +-
src/backend/access/common/detoast.c | 59 +++++++++++++++++--
src/backend/access/common/toast_internals.c | 19 +++++-
src/backend/access/compression/compress_lz4.c | 21 +++----
.../access/compression/compress_pglz.c | 20 +++----
.../access/compression/compressamapi.c | 27 +++++++++
src/backend/access/index/amapi.c | 50 ++++++++++++----
src/backend/commands/amcmds.c | 5 ++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 +
src/bin/psql/tab-complete.c | 6 ++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 16 +++--
src/include/access/detoast.h | 8 +++
src/include/access/toast_internals.h | 15 +++++
src/test/regress/expected/compression.out | 40 ++++++++++++-
src/test/regress/sql/compression.sql | 10 ++++
21 files changed, 272 insertions(+), 61 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 081fe078a4..46f34bffe5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,9 +390,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. The Compression method can be
- set to any available compression method. The built-in methods are
- <literal>pglz</literal> and <literal>lz4</literal>.
+ This clause adds compression to a column. The Compression method could be
+ created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The built-in methods
+ are <literal>pglz</literal> and <literal>lz4</literal>.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 7a18881e82..59fa3a1fe3 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -998,7 +998,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. The Compression
- method can be set from available compression methods. The built-in
+ method could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set from the available compression methods. The built-in
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not specified for the compressible type then
it will have the default compression method. The default compression
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index c90464053c..d7c1e19b7d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -457,13 +457,44 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -477,10 +508,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -496,9 +538,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -512,16 +556,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 00d5396f37..d71d7c552f 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index e8b035d138..34b0dc1002 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -30,7 +30,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -45,10 +45,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -56,7 +56,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -68,7 +68,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -81,9 +81,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -100,7 +100,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -112,9 +113,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 578be6f1dd..d33ea3d2cf 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c8d2..7aea8aad38 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -59,3 +60,29 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43514..89a4b01d78 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -46,14 +46,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +64,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +93,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 668e43faed..31d4a3e958 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -241,6 +241,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -278,6 +280,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 787860f481..1afa60acbc 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1968,7 +1968,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* compression method is not supported by the target attribute then
* we need to decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c9f80c4de4..7ab344b7ed 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5303,6 +5303,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index f54ba5ae0c..613b27e5d7 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5303,7 +5303,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5333,10 +5332,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 27bd5cb961..2dbb6fc9b0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13045,6 +13045,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5b8a6994cc..4ce630103e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index de758cab0b..367c53e6ae 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -284,6 +284,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0596..913a633d83 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -26,18 +26,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -61,5 +68,6 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..6cdc37542d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 0cb945952c..426c82b961 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 151478bcdd..636ab4021b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -235,13 +235,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index e447316573..0552eeca16 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -95,6 +95,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v18-0001-Built-in-compression-method.patchapplication/octet-stream; name=v18-0001-Built-in-compression-method.patchDownload
From 40723627bc687bcfc7643ab4ac694ba90cb4d30a Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v18 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 26 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++
.../access/compression/compress_pglz.c | 131 +++++++++
.../access/compression/compressamapi.c | 61 +++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 26 +-
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 ++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 46 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 ++-
src/include/access/compressamapi.h | 65 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 24 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 176 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 74 +++++
src/tools/pgindent/typedefs.list | 1 +
79 files changed, 2022 insertions(+), 668 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index b917a2a1c9..b6451bc63c 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -866,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8601,6 +8604,35 @@ fi
+#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12092,6 +12124,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13295,6 +13380,20 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 838d47dc22..74637d4221 100644
--- a/configure.ac
+++ b/configure.ac
@@ -999,6 +999,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1186,6 +1193,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1401,6 +1416,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index c12a32c8c7..56de239ab9 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from its parent table, however a different compression method can be set
+ for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..7a18881e82 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. The Compression
+ method can be set from available compression methods. The built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If the compression method is not specified for the compressible type then
+ it will have the default compression method. The default compression
+ method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..c90464053c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +512,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 70a9f77188..00d5396f37 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..c4814ef911
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..e8b035d138
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..578be6f1dd
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000000..663102c8d2
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 009e215da1..44fa2ca7b4 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 21f2240ade..66b8b1bb93 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1706,6 +1708,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cffbc0ac38..b4d19e33fa 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..668e43faed 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -34,6 +34,8 @@
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -83,7 +85,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -175,6 +189,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..59690ce854 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -581,6 +581,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..41c66c9e61 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -486,6 +486,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 993da56d43..53d6e627ae 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2408,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2457,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2703,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6233,6 +6275,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11752,6 +11806,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17631,3 +17701,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d7b8f65591..3fac8b7867 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1915,6 +1918,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2120,6 +2195,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba3ccc712c..3ea32323cd 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,6 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a2ef853dc2..227bb07bbd 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,6 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6be19916fc..c6ba9e1b05 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3858,6 +3858,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8392be6d44..187d3570e5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,6 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 31c95443a5..34db078d48 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -594,6 +594,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -629,9 +631,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3419,11 +3421,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3432,8 +3435,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3478,6 +3481,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3708,6 +3719,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15225,6 +15237,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15743,6 +15756,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..e2ac159aa2 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 5a62ab8bbc..43eee8a4b7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95079..f026104c01 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -128,6 +128,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 4838bfb4ff..f54ba5ae0c 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "common/hex_decode.h"
@@ -5293,6 +5295,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3bcbd4bdc3..27bd5cb961 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8612,6 +8614,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8697,6 +8700,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8715,7 +8727,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8742,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8770,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -13011,6 +13030,11 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -15800,6 +15824,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15824,6 +15849,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15859,6 +15887,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 731d0fe7ba..9e2390e1cd 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index caf97563f4..fc82412728 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..8226ae0596
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..0cb945952c 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7ac5..9d65bae238 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..00362d9db3 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 358f5aac8f..4a5c249ee9 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 059dec3647..e4df6bc5c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For the compressible atttypid this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible atttypid this must always be an invalid Oid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d7b55f57ea..eb870d4ba8 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7047,6 +7056,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7217,6 +7230,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
@@ -10688,6 +10708,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '1137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 49dc3e18e0..57e56e0177 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -619,6 +619,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e2d2a77ca4..3806a2e562 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 53f431e1ed..e612bda1f6 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index caed683ba9..ded2ad9692 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -510,6 +510,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dc2bb40926..232c9f1248 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 8c554e1f69..89bd2fd7a8 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f4d9f3b408..47f114b8a0 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..90c7454d2a 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..4dea269fc3
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,176 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..e078818496 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a687e99d1e..7560df4621 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3028,11 +3028,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3048,11 +3048,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3079,11 +3079,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 7bfeaf85f0..197c312ed6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e0e1ef71dd..2407ca945b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,6 +116,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 081fce32e7..152c8cb5b7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -203,3 +203,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..0750a6c80e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,74 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fb57b8393f..a7647db099 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -394,6 +394,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
v18-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v18-0006-Support-compression-methods-options.patchDownload
From bf40310f7c9a62a92d221fad9376d445efaf2dab Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 13 Jan 2021 12:17:02 +0530
Subject: [PATCH v18 6/6] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 +++++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 7 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 +++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 69 +++++++++++-
.../access/compression/compress_pglz.c | 98 +++++++++++++++--
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 ++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++++-
src/backend/commands/foreigncmds.c | 44 --------
src/backend/commands/tablecmds.c | 102 +++++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 20 +++-
src/bin/pg_dump/pg_dump.h | 2 +-
src/include/access/compressamapi.h | 19 +++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 +++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 ++++
35 files changed, 657 insertions(+), 133 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..39fa415a3e 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 46f34bffe5..e496f02cdc 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -393,7 +393,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
This clause adds compression to a column. The Compression method could be
created with <xref linkend="sql-create-access-method"/> or
it can be set to any available compression method. The built-in methods
- are <literal>pglz</literal> and <literal>lz4</literal>.
+ are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method has options they can be specified with the
+ <literal>WITH</literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 59fa3a1fe3..5f15ec286d 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. The Compression
@@ -1003,7 +1003,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not specified for the compressible type then
it will have the default compression method. The default compression
- method is <literal>pglz</literal>.
+ method is <literal>pglz</literal>. If the compression method has options
+ they can be specified with the <literal>WITH</literal> parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712c71..76f824bc6f 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d2ff..0d3307e94f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3ee9e..2dda6c038b 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d71d7c552f..6148f4af37 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 34b0dc1002..df6cebb381 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,6 +16,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -23,6 +24,63 @@
#include "lz4.h"
#endif
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -30,7 +88,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -40,6 +98,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -47,9 +106,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d33ea3d2cf..0396729c70 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9c3e..c9d44c62fa 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004e1b..a43e0e197c 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 66b8b1bb93..976f2008da 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1151,6 +1159,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1409,7 +1418,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4d19e33fa..0078de07d1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481557..348b7d3a37 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d5eeb493ac..931131480c 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -697,6 +697,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index a4d11d2dc3..24be66bf45 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -27,6 +27,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -161,7 +162,7 @@ IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -187,6 +188,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -252,15 +267,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttributes compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103fd3b..ae9ae2c51b 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fc80cb5624..7382badc6c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -610,6 +610,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -814,6 +815,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -867,8 +870,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -922,8 +926,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2431,16 +2438,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2477,8 +2482,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2728,14 +2732,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6152,6 +6151,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6315,6 +6315,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6325,7 +6326,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7722,13 +7723,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7767,7 +7775,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7858,7 +7888,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15104,9 +15134,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15141,7 +15173,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15151,8 +15184,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15160,8 +15202,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6588c84c4c..69e370de64 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2959,6 +2959,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3aa6176a30..1e3a01fbc3 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2623,6 +2623,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4f7053d63b..283452e454 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2890,6 +2890,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7ab344b7ed..e2f6f0bf4a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -414,6 +414,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3501,11 +3502,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3513,14 +3520,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a2a6b218f3..47fddc2ad1 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 2dbb6fc9b0..fb8b929fc7 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8704,10 +8704,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8760,6 +8767,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8789,6 +8797,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15853,7 +15862,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -15904,6 +15914,10 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9e2390e1cd..82de93c6a2 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,7 +328,7 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ char **attcmoptions; /* per-attribute current compression options */
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 913a633d83..8123cc8cc7 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -37,8 +38,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -49,14 +53,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc37f3..fe8de405ac 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 426c82b961..a527eb9506 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b49c..5ecb6f3653 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index e4df6bc5c1..26ecfcbd56 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index b107b82916..51be8967ca 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -138,6 +138,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,9 +152,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fd9c6abddf..a1b1158316 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 636ab4021b..9d2c00fb73 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -272,6 +272,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..27aab82462 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0552eeca16..59a32dd0a3 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
Re: [HACKERS] Custom compression methods
On Wed, Jan 13, 2021 at 2:14 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jan 11, 2021 at 3:40 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jan 11, 2021 at 12:21 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 11, 2021 at 12:11:54PM +0530, Dilip Kumar wrote:
On Mon, Jan 11, 2021 at 11:00 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Jan 10, 2021 at 10:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Mon, Jan 04, 2021 at 04:57:16PM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
And fails pg_upgrade check, apparently losing track of the compression (?)
CREATE TABLE public.cmdata2 ( - f1 text COMPRESSION lz4 + f1 text );I did not get this? pg_upgrade check is passing for me.
I realized that this was failing in your v16 patch sent Dec 25.
It's passing on current patches because they do "DROP TABLE cmdata2", but
that's only masking the error.I tested specifically pg_upgrade by removing all the DROP table and MV
and it is passing. I don't see the reason why should it fail. I mean
after the upgrade why COMPRESSION lz4 is missing?How did you test it ?
I'm not completely clear how this is intended to work... has it been tested
before ? According to the comments, in binary upgrade mode, there's an ALTER
which is supposed to SET COMPRESSION, but that's evidently not happening.I am able to reproduce this issue, If I run pg_dump with
binary_upgrade mode then I can see the issue (./pg_dump
--binary-upgrade -d Postgres). Yes you are right that for fixing
this there should be an ALTER..SET COMPRESSION method.I found that's the AM's OID in the old clsuter:
regression=# SELECT * FROM pg_am WHERE oid=36447;
oid | amname | amhandler | amtype
-------+--------+-------------+--------
36447 | pglz2 | pglzhandler | cBut in the new cluster, the OID has changed. Since that's written into table
data, I think you have to ensure that the compression OIDs are preserved on
upgrade:16755 | pglz2 | pglzhandler | c
Yeah, basically we are storing am oid in the compressed data so Oid
must be preserved. I will look into this and fix it.On further analysis, if we are dumping and restoring then we will
compress the data back while inserting it so why would we need to old
OID. I mean in the new cluster we are inserting data again so it will
be compressed again and now it will store the new OID. Am I missing
something here?I'm referring to pg_upgrade which uses pg_dump, but does *not* re-insert data,
but rather recreates catalogs only and then links to the old tables (either
with copy, link, or clone). Test with make -C src/bin/pg_upgrade (which is
included in make check-world).Got this as well.
I will fix these two issues and post the updated patch by tomorrow.
Thanks for your findings.
I have fixed this issue in the v18 version, please test and let me
know your thoughts. There is one more issue pending from an upgrade
perspective in v18-0003, basically, for the preserved method we need
to restore the dependency as well. I will work on this part and
shared the next version soon.
Now I have added support for handling the preserved method in the
binary upgrade, please find the updated patch set.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v19-0002-alter-table-set-compression.patchapplication/octet-stream; name=v19-0002-alter-table-set-compression.patchDownload
From 059492582f9890df7955637c04c693e29a75b071 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v19 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 +++++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/sql/compression.sql | 21 +++
8 files changed, 272 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..c4b53ec1c5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. The Compression method can be
+ set to any available compression method. The built-in methods are
+ <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 53d6e627ae..bc07d93459 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3860,6 +3862,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4387,7 +4390,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4795,6 +4799,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5419,6 +5427,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7659,6 +7670,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7674,7 +7750,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7738,47 +7813,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -14999,6 +15035,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 34db078d48..b1674764ee 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2306,6 +2306,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9dcab0d2fa..3127665660 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2093,7 +2093,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 232c9f1248..ffb739f565 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,7 +1875,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4dea269fc3..84d31c781b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -142,6 +142,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0750a6c80e..3b0da88c19 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -65,6 +65,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v19-0004-Create-custom-compression-methods.patchapplication/octet-stream; name=v19-0004-Create-custom-compression-methods.patchDownload
From 1ef192327e4f14ed2da74cef8f8585332e368d00 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Wed, 13 Jan 2021 12:14:40 +0530
Subject: [PATCH v19 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 7 ++-
doc/src/sgml/ref/create_access_method.sgml | 13 ++--
doc/src/sgml/ref/create_table.sgml | 3 +-
src/backend/access/common/detoast.c | 59 +++++++++++++++++--
src/backend/access/common/toast_internals.c | 19 +++++-
src/backend/access/compression/compress_lz4.c | 21 +++----
.../access/compression/compress_pglz.c | 20 +++----
.../access/compression/compressamapi.c | 27 +++++++++
src/backend/access/index/amapi.c | 50 ++++++++++++----
src/backend/commands/amcmds.c | 5 ++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 +
src/bin/psql/tab-complete.c | 6 ++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 16 +++--
src/include/access/detoast.h | 8 +++
src/include/access/toast_internals.h | 15 +++++
src/test/regress/expected/compression.out | 40 ++++++++++++-
src/test/regress/sql/compression.sql | 10 ++++
21 files changed, 272 insertions(+), 61 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 081fe078a4..46f34bffe5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,9 +390,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. The Compression method can be
- set to any available compression method. The built-in methods are
- <literal>pglz</literal> and <literal>lz4</literal>.
+ This clause adds compression to a column. The Compression method could be
+ created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The built-in methods
+ are <literal>pglz</literal> and <literal>lz4</literal>.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 7a18881e82..59fa3a1fe3 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -998,7 +998,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. The Compression
- method can be set from available compression methods. The built-in
+ method could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set from the available compression methods. The built-in
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not specified for the compressible type then
it will have the default compression method. The default compression
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index c90464053c..d7c1e19b7d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -457,13 +457,44 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the buil-in compression
+ * id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -477,10 +508,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -496,9 +538,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -512,16 +556,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 00d5396f37..d71d7c552f 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index e8b035d138..34b0dc1002 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -30,7 +30,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -45,10 +45,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -56,7 +56,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -68,7 +68,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -81,9 +81,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -100,7 +100,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -112,9 +113,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 578be6f1dd..d33ea3d2cf 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c8d2..7aea8aad38 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -59,3 +60,29 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43514..89a4b01d78 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -46,14 +46,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +64,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +93,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 668e43faed..31d4a3e958 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -241,6 +241,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -278,6 +280,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 787860f481..1afa60acbc 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -1968,7 +1968,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* compression method is not supported by the target attribute then
* we need to decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index c9f80c4de4..7ab344b7ed 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5303,6 +5303,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index f54ba5ae0c..613b27e5d7 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5303,7 +5303,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5333,10 +5332,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 6b3e8c7acc..bbe433ccc4 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13119,6 +13119,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5b8a6994cc..4ce630103e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index de758cab0b..367c53e6ae 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -284,6 +284,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0596..913a633d83 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -26,18 +26,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -61,5 +68,6 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..6cdc37542d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 0cb945952c..426c82b961 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 151478bcdd..636ab4021b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -235,13 +235,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index e447316573..0552eeca16 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -95,6 +95,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v19-0003-Add-support-for-PRESERVE.patchapplication/octet-stream; name=v19-0003-Add-support-for-PRESERVE.patchDownload
From 8c72251fb85de2a8fb79ff834c85098640cb8152 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:15:20 +0530
Subject: [PATCH v19 3/6] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. For
supporting this the column will maintain the dependency with all
the supported compression methods. So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/objectaddress.c | 1 +
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 293 +++++++++++++++++++++
src/backend/commands/tablecmds.c | 116 ++++----
src/backend/executor/nodeModifyTable.c | 13 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 103 ++++++++
src/bin/pg_dump/pg_dump.h | 14 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/create_index.out | 56 ++--
src/test/regress/sql/compression.sql | 9 +
22 files changed, 697 insertions(+), 97 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c4b53ec1c5..081fe078a4 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. The Compression method can be
set to any available compression method. The built-in methods are
<literal>pglz</literal> and <literal>lz4</literal>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b690d8..ea9b90cc80 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/relation.h"
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da24322d..dd376484b7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..eda0e672d2
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,293 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency upon.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribue.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode just create the dependency for all preserve
+ * list compression method as a dependecy.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index bc07d93459..5fe862c7a8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +937,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2413,16 +2429,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2459,7 +2476,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2710,12 +2728,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4800,7 +4818,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6294,7 +6313,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6469,6 +6489,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6616,6 +6637,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11751,7 +11794,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11866,6 +11910,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15044,24 +15093,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15094,11 +15140,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17823,32 +17874,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 3fac8b7867..787860f481 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -1933,6 +1934,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -1941,7 +1943,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop for all the attributes in the tuple and check if any of the
* attribute is compressed in the source tuple and its compression method
- * is not same as the target compression method then we need to decompress
+ * is not supported by the target attribute then we need to decompress
* it.
*/
for (i = 0; i < natts; i++)
@@ -1962,11 +1964,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3ea32323cd..6588c84c4c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,7 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2952,6 +2952,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5640,6 +5652,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 227bb07bbd..3aa6176a30 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,7 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2618,6 +2618,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3693,6 +3703,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 187d3570e5..4f7053d63b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,7 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2883,6 +2883,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4230,6 +4240,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b1674764ee..c9f80c4de4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -594,7 +594,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2307,12 +2309,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3435,7 +3437,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3490,13 +3492,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e2ac159aa2..a2a6b218f3 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 27bd5cb961..6b3e8c7acc 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9005,6 +9005,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16321,6 +16395,35 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, "\nWITH (%s) ", tbinfo->attcmoptions[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 9e2390e1cd..42d8d793d0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,7 +328,7 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -357,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 3127665660..5b8a6994cc 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,6 +2098,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 3806a2e562..b107b82916 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -147,6 +147,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index 08c463d3c4..c2347aeddc 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -93,5 +93,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ded2ad9692..d630c54efa 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ffb739f565..fd9c6abddf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 84d31c781b..151478bcdd 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -201,12 +201,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index fc6afab58a..d8300bdcdd 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 3b0da88c19..e447316573 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -86,6 +86,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
v19-0005-new-compression-method-extension-for-zlib.patchapplication/octet-stream; name=v19-0005-new-compression-method-extension-for-zlib.patchDownload
From a7acf849c27d9556bb222ffa88a77992b295a33b Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v19 5/6] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index 7a4866e338..f3a2f582bf 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..bf36854ef3
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ lz4
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..aa7c57030d
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION lz4);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.23.0
v19-0001-Built-in-compression-method.patchapplication/octet-stream; name=v19-0001-Built-in-compression-method.patchDownload
From 40723627bc687bcfc7643ab4ac694ba90cb4d30a Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v19 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 26 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++
.../access/compression/compress_pglz.c | 131 +++++++++
.../access/compression/compressamapi.c | 61 +++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 26 +-
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 ++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 46 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 ++-
src/include/access/compressamapi.h | 65 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 24 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 176 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 74 +++++
src/tools/pgindent/typedefs.list | 1 +
79 files changed, 2022 insertions(+), 668 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index b917a2a1c9..b6451bc63c 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -866,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1570,6 +1572,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8601,6 +8604,35 @@ fi
+#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=yes
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12092,6 +12124,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13295,6 +13380,20 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 838d47dc22..74637d4221 100644
--- a/configure.ac
+++ b/configure.ac
@@ -999,6 +999,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1186,6 +1193,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1401,6 +1416,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index c12a32c8c7..56de239ab9 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from its parent table, however a different compression method can be set
+ for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..7a18881e82 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. The Compression
+ method can be set from available compression methods. The built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If the compression method is not specified for the compressible type then
+ it will have the default compression method. The default compression
+ method is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..c90464053c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +512,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 70a9f77188..00d5396f37 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..c4814ef911
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..e8b035d138
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..578be6f1dd
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000000..663102c8d2
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 009e215da1..44fa2ca7b4 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 21f2240ade..66b8b1bb93 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -780,6 +781,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1706,6 +1708,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index cffbc0ac38..b4d19e33fa 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..668e43faed 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -34,6 +34,8 @@
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -83,7 +85,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -175,6 +189,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..59690ce854 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -581,6 +581,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..41c66c9e61 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -486,6 +486,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 993da56d43..53d6e627ae 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2395,6 +2408,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2429,6 +2457,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2674,6 +2703,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6233,6 +6275,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11752,6 +11806,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17631,3 +17701,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d7b8f65591..3fac8b7867 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -1915,6 +1918,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2120,6 +2195,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba3ccc712c..3ea32323cd 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,6 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a2ef853dc2..227bb07bbd 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,6 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6be19916fc..c6ba9e1b05 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3858,6 +3858,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8392be6d44..187d3570e5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,6 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 31c95443a5..34db078d48 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -594,6 +594,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -629,9 +631,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3419,11 +3421,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3432,8 +3435,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3478,6 +3481,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3708,6 +3719,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15225,6 +15237,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15743,6 +15756,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..e2ac159aa2 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 5a62ab8bbc..43eee8a4b7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95079..f026104c01 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -128,6 +128,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 4838bfb4ff..f54ba5ae0c 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/int.h"
#include "common/hex_decode.h"
@@ -5293,6 +5295,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3bcbd4bdc3..27bd5cb961 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8612,6 +8614,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8697,6 +8700,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8715,7 +8727,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8742,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8770,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -13011,6 +13030,11 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -15800,6 +15824,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15824,6 +15849,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15859,6 +15887,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 731d0fe7ba..9e2390e1cd 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index caf97563f4..fc82412728 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..8226ae0596
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..0cb945952c 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7ac5..9d65bae238 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..00362d9db3 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 358f5aac8f..4a5c249ee9 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 059dec3647..e4df6bc5c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For the compressible atttypid this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible atttypid this must always be an invalid Oid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index d7b55f57ea..eb870d4ba8 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7047,6 +7056,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7217,6 +7230,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
@@ -10688,6 +10708,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '1137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 49dc3e18e0..57e56e0177 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -619,6 +619,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e2d2a77ca4..3806a2e562 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 53f431e1ed..e612bda1f6 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -615,5 +615,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index caed683ba9..ded2ad9692 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -510,6 +510,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dc2bb40926..232c9f1248 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 8c554e1f69..89bd2fd7a8 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f4d9f3b408..47f114b8a0 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..90c7454d2a 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..4dea269fc3
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,176 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..e078818496 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index a687e99d1e..7560df4621 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3028,11 +3028,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3048,11 +3048,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3079,11 +3079,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 7bfeaf85f0..197c312ed6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -123,11 +123,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 162b591b31..a5b6c1dc24 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e0e1ef71dd..2407ca945b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,6 +116,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 081fce32e7..152c8cb5b7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -203,3 +203,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..0750a6c80e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,74 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index fb57b8393f..a7647db099 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -394,6 +394,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
v19-0006-Support-compression-methods-options.patchapplication/octet-stream; name=v19-0006-Support-compression-methods-options.patchDownload
From 9bbb6c1fb61b849c9967435a7641636f2334c80e Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 19 Jan 2021 15:10:14 +0530
Subject: [PATCH v19 6/6] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 +++++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 7 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 +++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 69 +++++++++++-
.../access/compression/compress_pglz.c | 98 +++++++++++++++--
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 ++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++++-
src/backend/commands/foreigncmds.c | 44 --------
src/backend/commands/tablecmds.c | 102 +++++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 20 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/include/access/compressamapi.h | 19 +++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 +++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 ++++
35 files changed, 658 insertions(+), 132 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..39fa415a3e 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 46f34bffe5..e496f02cdc 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -393,7 +393,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
This clause adds compression to a column. The Compression method could be
created with <xref linkend="sql-create-access-method"/> or
it can be set to any available compression method. The built-in methods
- are <literal>pglz</literal> and <literal>lz4</literal>.
+ are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method has options they can be specified with the
+ <literal>WITH</literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 59fa3a1fe3..5f15ec286d 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. The Compression
@@ -1003,7 +1003,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If the compression method is not specified for the compressible type then
it will have the default compression method. The default compression
- method is <literal>pglz</literal>.
+ method is <literal>pglz</literal>. If the compression method has options
+ they can be specified with the <literal>WITH</literal> parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712c71..76f824bc6f 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d2ff..0d3307e94f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3ee9e..2dda6c038b 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index d71d7c552f..6148f4af37 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 34b0dc1002..df6cebb381 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,6 +16,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -23,6 +24,63 @@
#include "lz4.h"
#endif
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -30,7 +88,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -40,6 +98,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -47,9 +106,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d33ea3d2cf..0396729c70 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9c3e..c9d44c62fa 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004e1b..a43e0e197c 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 66b8b1bb93..976f2008da 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -732,6 +732,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -787,6 +788,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -834,6 +840,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -852,7 +859,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -884,7 +892,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1151,6 +1159,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1409,7 +1418,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4d19e33fa..0078de07d1 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481557..348b7d3a37 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index d5eeb493ac..931131480c 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -697,6 +697,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index eda0e672d2..3b7ff7131b 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -29,6 +29,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -181,7 +182,7 @@ BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -207,6 +208,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -279,15 +294,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttributes compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103fd3b..ae9ae2c51b 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5fe862c7a8..5617917b86 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -609,6 +609,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -813,6 +814,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -866,8 +869,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -921,8 +925,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2430,16 +2437,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2476,8 +2481,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2727,14 +2731,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6151,6 +6150,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6314,6 +6314,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6324,7 +6325,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7721,13 +7722,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7766,7 +7774,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7857,7 +7887,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15103,9 +15133,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15140,7 +15172,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15150,8 +15183,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15159,8 +15201,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6588c84c4c..69e370de64 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2959,6 +2959,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3aa6176a30..1e3a01fbc3 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2623,6 +2623,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4f7053d63b..283452e454 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2890,6 +2890,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7ab344b7ed..e2f6f0bf4a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -414,6 +414,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3501,11 +3502,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3513,14 +3520,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a2a6b218f3..47fddc2ad1 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index bbe433ccc4..19aea38872 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8704,10 +8704,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8760,6 +8767,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8789,6 +8797,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15927,7 +15936,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -15978,6 +15988,10 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 42d8d793d0..6c4f854fe1 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,6 +328,8 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
+ char **attcmoptions; /* per-attribute current compression
+ options */
struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 913a633d83..8123cc8cc7 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -37,8 +38,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -49,14 +53,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc37f3..fe8de405ac 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 426c82b961..a527eb9506 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b49c..5ecb6f3653 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index e4df6bc5c1..26ecfcbd56 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index b107b82916..51be8967ca 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -138,6 +138,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -150,9 +152,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fd9c6abddf..a1b1158316 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 636ab4021b..9d2c00fb73 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -272,6 +272,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..27aab82462 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0552eeca16..59a32dd0a3 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.23.0
Re: [HACKERS] Custom compression methods
Thanks for updating the patch.
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The most recent patch doesn't compile --without-lz4:
On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think I first saw it on cfbot and I reproduced it locally, too.
http://cfbot.cputube.org/dilip-kumar.htmlI think you'll have to make --without-lz4 the default until the build
environments include it, otherwise the patch checker will show red :(Oh ok, but if we make by default --without-lz4 then the test cases
will start failing which is using lz4 compression. Am I missing
something?
The CIs are failing like this:
http://cfbot.cputube.org/dilip-kumar.html
|checking for LZ4_compress in -llz4... no
|configure: error: lz4 library not found
|If you have lz4 already installed, see config.log for details on the
|failure. It is possible the compiler isn't looking in the proper directory.
|Use --without-lz4 to disable lz4 support.
I thought that used to work (except for windows). I don't see that anything
changed in the configure tests... Is it because the CI moved off travis 2
weeks ago ? I don't' know whether the travis environment had liblz4, and I
don't remember if the build was passing or if it was failing for some other
reason. I'm guessing historic logs from travis are not available, if they ever
were.
I'm not sure how to deal with that, but maybe you'd need:
1) A separate 0001 patch *allowing* LZ4 to be enabled/disabled;
2) Current patchset needs to compile with/without LZ4, and pass tests in both
cases - maybe you can use "alternate test" output [0]cp -iv src/test/regress/results/compression.out src/test/regress/expected/compression_1.out to handle the "without"
case.
3) Eventually, the CI and build environments may have LZ4 installed, and then
we can have a separate debate about whether to enable it by default.
[0]: cp -iv src/test/regress/results/compression.out src/test/regress/expected/compression_1.out
On Tue, Jan 05, 2021 at 02:20:26PM +0530, Dilip Kumar wrote:
On Tue, Jan 5, 2021 at 11:19 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I see the windows build is failing:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.123730
|undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at src/tools/msvc/Mkvcbuild.pm line 852.
This needs to be patched: src/tools/msvc/Solution.pm
You can see my zstd/pg_dump patch for an example, if needed (actually I'm not
100% sure it's working yet, since the windows build failed for another reason).Okay, I will check that.
This still needs help.
perl ./src/tools/msvc/mkvcbuild.pl
...
undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at /home/pryzbyj/src/postgres/src/tools/msvc/Mkvcbuild.pm line 852.
Fix like:
+ HAVE_LIBLZ4 => $self->{options}->{zlib} ? 1 : undef,
Some more language fixes:
commit 3efafee52414503a87332fa6070541a3311a408c
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Sep 8 15:24:33 2020 +0530
Built-in compression method
+ If the compression method is not specified for the compressible type then
+ it will have the default compression method. The default compressionI think this should say:
If no compression method is specified, then compressible types will have the
default compression method (pglz).
+ *
+ * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressedShould say v14 ??
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 059dec3647..e4df6bc5c1 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For the compressible atttypid this must always be asay "For compressible types, ..."
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible atttypid this must always be an invalid Oid.say "must be InvalidOid"
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,This is interesting...
I have a patch to implement LIKE .. (INCLUDING ACCESS METHOD).
I guess I should change it to say LIKE .. (TABLE ACCESS METHOD), right ?
https://commitfest.postgresql.org/31/2865/
Your first patch is large due to updating a large number of test cases to
include the "compression" column in \d+ output. Maybe that column should be
hidden when HIDE_TABLEAM is set by pg_regress ? I think that would allow
testing with alternate, default compression.
commit ddcae4095e36e94e3e7080e2ab5a8d42cc2ca843
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Jan 19 15:10:14 2021 +0530
Support compression methods options
+ * we don't need do it again in cminitstate function.
need *to* do it again
+ * Fetch atttributes compression options
attribute's :)
commit b7946eda581230424f73f23d90843f4c2db946c2
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Wed Jan 13 12:14:40 2021 +0530
Create custom compression methods
+ * compression header otherwise, directly translate the buil-in compression
built-in
commit 0746a4d7a14209ebf62fe0dc1d12999ded879cfd
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Mon Jan 4 15:15:20 2021 +0530
Add support for PRESERVE
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -15,6 +15,7 @@#include "postgres.h"
+#include "access/compressamapi.h"
Unnecessary change to this file ?
+ * ... Collect the list of access method
+ * oids on which this attribute has a dependency upon."upon" is is redundant. Say "on which this attribute has a dependency".
+ * Check whether the given compression method oid is supported by
+ * the target attribue.attribute
+ * In binary upgrade mode just create the dependency for all preserve
+ * list compression method as a dependecy.dependency
I think you could say: "In binary upgrade mode, just create a dependency on all
preserved methods".
Import Notes
Reply to msg id not found: CAFiTN-updTo3aB+7vsECHOHpjB23Av5OfgZQrt90Q33gFK-oZQ@mail.gmail.comCAFiTN-utJPbTLQ9i10wT_zmHXun+RQMB1B1xbkTgrh971vqjw@mail.gmail.com | Resolved by subject fallback
Re: [HACKERS] Custom compression methods
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks for updating the patch.
Thanks for the review
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The most recent patch doesn't compile --without-lz4:
On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think I first saw it on cfbot and I reproduced it locally, too.
http://cfbot.cputube.org/dilip-kumar.htmlI think you'll have to make --without-lz4 the default until the build
environments include it, otherwise the patch checker will show red :(Oh ok, but if we make by default --without-lz4 then the test cases
will start failing which is using lz4 compression. Am I missing
something?The CIs are failing like this:
http://cfbot.cputube.org/dilip-kumar.html
|checking for LZ4_compress in -llz4... no
|configure: error: lz4 library not found
|If you have lz4 already installed, see config.log for details on the
|failure. It is possible the compiler isn't looking in the proper directory.
|Use --without-lz4 to disable lz4 support.I thought that used to work (except for windows). I don't see that anything
changed in the configure tests... Is it because the CI moved off travis 2
weeks ago ? I don't' know whether the travis environment had liblz4, and I
don't remember if the build was passing or if it was failing for some other
reason. I'm guessing historic logs from travis are not available, if they ever
were.I'm not sure how to deal with that, but maybe you'd need:
1) A separate 0001 patch *allowing* LZ4 to be enabled/disabled;
2) Current patchset needs to compile with/without LZ4, and pass tests in both
cases - maybe you can use "alternate test" output [0] to handle the "without"
case.
Okay, let me think about how to deal with this.
3) Eventually, the CI and build environments may have LZ4 installed, and then
we can have a separate debate about whether to enable it by default.[0] cp -iv src/test/regress/results/compression.out src/test/regress/expected/compression_1.out
On Tue, Jan 05, 2021 at 02:20:26PM +0530, Dilip Kumar wrote:
On Tue, Jan 5, 2021 at 11:19 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I see the windows build is failing:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.123730
|undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at src/tools/msvc/Mkvcbuild.pm line 852.
This needs to be patched: src/tools/msvc/Solution.pm
You can see my zstd/pg_dump patch for an example, if needed (actually I'm not
100% sure it's working yet, since the windows build failed for another reason).Okay, I will check that.
This still needs help.
perl ./src/tools/msvc/mkvcbuild.pl
...
undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at /home/pryzbyj/src/postgres/src/tools/msvc/Mkvcbuild.pm line 852.Fix like:
+ HAVE_LIBLZ4 => $self->{options}->{zlib} ? 1 : undef,
I will do that.
Some more language fixes:
commit 3efafee52414503a87332fa6070541a3311a408c
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Sep 8 15:24:33 2020 +0530Built-in compression method
+ If the compression method is not specified for the compressible type then + it will have the default compression method. The default compressionI think this should say:
If no compression method is specified, then compressible types will have the
default compression method (pglz).+ * + * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressedShould say v14 ??
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h index 059dec3647..e4df6bc5c1 100644 --- a/src/include/catalog/pg_attribute.h +++ b/src/include/catalog/pg_attribute.h @@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75, /* attribute's collation */ Oid attcollation;+ /* + * Oid of the compression method that will be used for compressing the value + * for this attribute. For the compressible atttypid this must always be asay "For compressible types, ..."
+ * valid Oid irrespective of what is the current value of the attstorage. + * And for the incompressible atttypid this must always be an invalid Oid.say "must be InvalidOid"
@@ -685,6 +686,7 @@ typedef enum TableLikeOption CREATE_TABLE_LIKE_INDEXES = 1 << 5, CREATE_TABLE_LIKE_STATISTICS = 1 << 6, CREATE_TABLE_LIKE_STORAGE = 1 << 7, + CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,This is interesting...
I have a patch to implement LIKE .. (INCLUDING ACCESS METHOD).
I guess I should change it to say LIKE .. (TABLE ACCESS METHOD), right ?
https://commitfest.postgresql.org/31/2865/Your first patch is large due to updating a large number of test cases to
include the "compression" column in \d+ output. Maybe that column should be
hidden when HIDE_TABLEAM is set by pg_regress ? I think that would allow
testing with alternate, default compression.commit ddcae4095e36e94e3e7080e2ab5a8d42cc2ca843
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Jan 19 15:10:14 2021 +0530Support compression methods options
+ * we don't need do it again in cminitstate function.
need *to* do it again
+ * Fetch atttributes compression options
attribute's :)
commit b7946eda581230424f73f23d90843f4c2db946c2
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Wed Jan 13 12:14:40 2021 +0530Create custom compression methods
+ * compression header otherwise, directly translate the buil-in compression
built-in
commit 0746a4d7a14209ebf62fe0dc1d12999ded879cfd
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Mon Jan 4 15:15:20 2021 +0530Add support for PRESERVE
--- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -15,6 +15,7 @@#include "postgres.h"
+#include "access/compressamapi.h"
Unnecessary change to this file ?
+ * ... Collect the list of access method + * oids on which this attribute has a dependency upon."upon" is is redundant. Say "on which this attribute has a dependency".
+ * Check whether the given compression method oid is supported by + * the target attribue.attribute
+ * In binary upgrade mode just create the dependency for all preserve + * list compression method as a dependecy.dependency
I think you could say: "In binary upgrade mode, just create a dependency on all
preserved methods".
I will work on other comments and send the updated patch in a day or two.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
I have been testing the patches for a while , below is the code coverage
observed on the v19 patches.
Sr No File name Code Coverage
Before After
Line % Function % Line % Function %
1 src/backend/access/brin/brin_tuple.c 96.7 100 96.7 100
2 src/backend/access/common/detoast.c 88 100 88.6 100
3 src/backend/access/common/indextuple.c 97.1 100 97.1 100
4 src/backend/access/common/toast_internals.c 88.8 88.9 88.6 88.9
5 src/backend/access/common/tupdesc.c 97.2 100 97.2 100
6 src/backend/access/compression/compress_lz4.c NA NA 93.5 100
7 src/backend/access/compression/compress_pglz.c NA NA 82.2 100
8 src/backend/access/compression/compressamapi.c NA NA 78.3 100
9 src/backend/access/index/amapi.c 73.5 100 74.5 100
10 src/backend/access/table/toast_helper.c 97.5 100 97.5 100
11 src/backend/access/common/reloptions.c 90.6 83.3 89.7 81.6
12 src/backend/bootstrap/bootparse.y 84.2 100 84.2 100
13 src/backend/bootstrap/bootstrap.c 66.4 100 66.4 100
14 src/backend/commands/cluster.c 90.4 100 90.4 100
15 src/backend/catalog/heap.c 97.3 100 97.3 100
16 src/backend/catalog/index.c 93.8 94.6 93.8 94.6
17 src/backend/catalog/toasting.c 96.7 100 96.8 100
18 src/backend/catalog/objectaddress.c 89.7 95.9 89.7 95.9
19 src/backend/catalog/pg_depend.c 98.6 100 98.6 100
20 src/backend/commands/foreigncmds.c 95.7 95.5 95.6 95.2
21 src/backend/commands/compressioncmds.c NA NA 97.2 100
22 src/backend/commands/amcmds.c 92.1 100 90.1 100
23 src/backend/commands/createas.c 96.8 90 96.8 90
24 src/backend/commands/matview.c 92.5 85.7 92.6 85.7
25 src/backend/commands/tablecmds.c 93.6 98.5 93.7 98.5
26 src/backend/executor/nodeModifyTable.c 93.8 92.9 93.7 92.9
27 src/backend/nodes/copyfuncs.c 79.1 78.7 79.2 78.8
28 src/backend/nodes/equalfuncs.c 28.8 23.9 28.7 23.8
29 src/backend/nodes/nodeFuncs.c 80.4 100 80.3 100
30 src/backend/nodes/outfuncs.c 38.2 38.1 38.1 38
31 src/backend/parser/gram.y 87.6 100 87.7 100
32 src/backend/parser/parse_utilcmd.c 91.6 100 91.6 100
33 src/backend/replication/logical/reorderbuffer.c 94.1 97 94.1 97
34 src/backend/utils/adt/pg_upgrade_support.c 56.2 83.3 58.4 84.6
35 src/backend/utils/adt/pseudotypes.c 18.5 11.3 18.3 10.9
36 src/backend/utils/adt/varlena.c 86.5 89 86.6 89.1
37 src/bin/pg_dump/pg_dump.c 89.4 97.4 89.5 97.4
38 src/bin/psql/tab-complete.c 50.8 57.7 50.8 57.7
39 src/bin/psql/describe.c 60.7 55.1 60.6 54.2
40 contrib/cmzlib/cmzlib.c NA NA 74.7 87.5
Thanks.
--
Regards,
Neha Sharma
On Wed, Jan 20, 2021 at 10:18 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Show quoted text
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby <pryzby@telsasoft.com>
wrote:Thanks for updating the patch.
Thanks for the review
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com>
wrote:
The most recent patch doesn't compile --without-lz4:
On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com>
wrote:
I think I first saw it on cfbot and I reproduced it locally, too.
http://cfbot.cputube.org/dilip-kumar.htmlI think you'll have to make --without-lz4 the default until the build
environments include it, otherwise the patch checker will show red :(Oh ok, but if we make by default --without-lz4 then the test cases
will start failing which is using lz4 compression. Am I missing
something?The CIs are failing like this:
http://cfbot.cputube.org/dilip-kumar.html
|checking for LZ4_compress in -llz4... no
|configure: error: lz4 library not found
|If you have lz4 already installed, see config.log for details on the
|failure. It is possible the compiler isn't looking in the properdirectory.
|Use --without-lz4 to disable lz4 support.
I thought that used to work (except for windows). I don't see that
anything
changed in the configure tests... Is it because the CI moved off travis
2
weeks ago ? I don't' know whether the travis environment had liblz4,
and I
don't remember if the build was passing or if it was failing for some
other
reason. I'm guessing historic logs from travis are not available, if
they ever
were.
I'm not sure how to deal with that, but maybe you'd need:
1) A separate 0001 patch *allowing* LZ4 to be enabled/disabled;
2) Current patchset needs to compile with/without LZ4, and pass tests inboth
cases - maybe you can use "alternate test" output [0] to handle the
"without"
case.
Okay, let me think about how to deal with this.
3) Eventually, the CI and build environments may have LZ4 installed, and
then
we can have a separate debate about whether to enable it by default.
[0] cp -iv src/test/regress/results/compression.out
src/test/regress/expected/compression_1.out
On Tue, Jan 05, 2021 at 02:20:26PM +0530, Dilip Kumar wrote:
On Tue, Jan 5, 2021 at 11:19 AM Dilip Kumar <dilipbalaut@gmail.com>
wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com>
wrote:
I see the windows build is failing:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.123730
|undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350
at src/tools/msvc/Mkvcbuild.pm line 852.
This needs to be patched: src/tools/msvc/Solution.pm
You can see my zstd/pg_dump patch for an example, if needed(actually I'm not
100% sure it's working yet, since the windows build failed for
another reason).
Okay, I will check that.
This still needs help.
perl ./src/tools/msvc/mkvcbuild.pl
...
undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at/home/pryzbyj/src/postgres/src/tools/msvc/Mkvcbuild.pm line 852.
Fix like:
+ HAVE_LIBLZ4 => $self->{options}->{zlib}
? 1 : undef,
I will do that.
Some more language fixes:
commit 3efafee52414503a87332fa6070541a3311a408c
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Sep 8 15:24:33 2020 +0530Built-in compression method
+ If the compression method is not specified for the compressible
type then
+ it will have the default compression method. The default
compression
I think this should say:
If no compression method is specified, then compressible types will havethe
default compression method (pglz).
+ * + * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressedShould say v14 ??
diff --git a/src/include/catalog/pg_attribute.hb/src/include/catalog/pg_attribute.h
index 059dec3647..e4df6bc5c1 100644 --- a/src/include/catalog/pg_attribute.h +++ b/src/include/catalog/pg_attribute.h @@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId)BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;+ /* + * Oid of the compression method that will be used forcompressing the value
+ * for this attribute. For the compressible atttypid this must
always be a
say "For compressible types, ..."
+ * valid Oid irrespective of what is the current value of the
attstorage.
+ * And for the incompressible atttypid this must always be an
invalid Oid.
say "must be InvalidOid"
@@ -685,6 +686,7 @@ typedef enum TableLikeOption CREATE_TABLE_LIKE_INDEXES = 1 << 5, CREATE_TABLE_LIKE_STATISTICS = 1 << 6, CREATE_TABLE_LIKE_STORAGE = 1 << 7, + CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,This is interesting...
I have a patch to implement LIKE .. (INCLUDING ACCESS METHOD).
I guess I should change it to say LIKE .. (TABLE ACCESS METHOD), right ?
https://commitfest.postgresql.org/31/2865/Your first patch is large due to updating a large number of test cases to
include the "compression" column in \d+ output. Maybe that columnshould be
hidden when HIDE_TABLEAM is set by pg_regress ? I think that would allow
testing with alternate, default compression.commit ddcae4095e36e94e3e7080e2ab5a8d42cc2ca843
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Jan 19 15:10:14 2021 +0530Support compression methods options
+ * we don't need do it again in cminitstate function.
need *to* do it again
+ * Fetch atttributes compression options
attribute's :)
commit b7946eda581230424f73f23d90843f4c2db946c2
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Wed Jan 13 12:14:40 2021 +0530Create custom compression methods
+ * compression header otherwise, directly translate the buil-in
compression
built-in
commit 0746a4d7a14209ebf62fe0dc1d12999ded879cfd
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Mon Jan 4 15:15:20 2021 +0530Add support for PRESERVE
--- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -15,6 +15,7 @@#include "postgres.h"
+#include "access/compressamapi.h"
Unnecessary change to this file ?
+ * ... Collect the list of access method + * oids on which this attribute has a dependency upon."upon" is is redundant. Say "on which this attribute has a dependency".
+ * Check whether the given compression method oid is supported by + * the target attribue.attribute
+ * In binary upgrade mode just create the dependency for
all preserve
+ * list compression method as a dependecy.
dependency
I think you could say: "In binary upgrade mode, just create a dependencyon all
preserved methods".
I will work on other comments and send the updated patch in a day or two.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Jan 29, 2021 at 9:47 AM Neha Sharma
<neha.sharma@enterprisedb.com> wrote:
Hi,
I have been testing the patches for a while , below is the code coverage observed on the v19 patches.
Sr NoFile nameCode Coverage
BeforeAfter
Line %Function %Line %Function %
1src/backend/access/brin/brin_tuple.c96.710096.7100
2src/backend/access/common/detoast.c8810088.6100
3src/backend/access/common/indextuple.c97.110097.1100
4src/backend/access/common/toast_internals.c88.888.988.688.9
5src/backend/access/common/tupdesc.c97.210097.2100
6src/backend/access/compression/compress_lz4.cNANA93.5100
7src/backend/access/compression/compress_pglz.cNANA82.2100
8src/backend/access/compression/compressamapi.cNANA78.3100
9src/backend/access/index/amapi.c73.510074.5100
10src/backend/access/table/toast_helper.c97.510097.5100
11src/backend/access/common/reloptions.c90.683.389.781.6
12src/backend/bootstrap/bootparse.y84.210084.2100
13src/backend/bootstrap/bootstrap.c66.410066.4100
14src/backend/commands/cluster.c90.410090.4100
15src/backend/catalog/heap.c97.310097.3100
16src/backend/catalog/index.c93.894.693.894.6
17src/backend/catalog/toasting.c96.710096.8100
18src/backend/catalog/objectaddress.c89.795.989.795.9
19src/backend/catalog/pg_depend.c98.610098.6100
20src/backend/commands/foreigncmds.c95.795.595.695.2
21src/backend/commands/compressioncmds.cNANA97.2100
22src/backend/commands/amcmds.c92.110090.1100
23src/backend/commands/createas.c96.89096.890
24src/backend/commands/matview.c92.585.792.685.7
25src/backend/commands/tablecmds.c93.698.593.798.5
26src/backend/executor/nodeModifyTable.c93.892.993.792.9
27src/backend/nodes/copyfuncs.c79.178.779.278.8
28src/backend/nodes/equalfuncs.c28.823.928.723.8
29src/backend/nodes/nodeFuncs.c80.410080.3100
30src/backend/nodes/outfuncs.c38.238.138.138
31src/backend/parser/gram.y87.610087.7100
32src/backend/parser/parse_utilcmd.c91.610091.6100
33src/backend/replication/logical/reorderbuffer.c94.19794.197
34src/backend/utils/adt/pg_upgrade_support.c56.283.358.484.6
35src/backend/utils/adt/pseudotypes.c18.511.318.310.9
36src/backend/utils/adt/varlena.c86.58986.689.1
37src/bin/pg_dump/pg_dump.c89.497.489.597.4
38src/bin/psql/tab-complete.c50.857.750.857.7
39src/bin/psql/describe.c60.755.160.654.2
40contrib/cmzlib/cmzlib.cNANA74.787.5
Thanks, Neha for testing this, overall coverage looks good to me
except compress_pglz.c, compressamapi.c and cmzlib.c. I will analyze
this and see if we can improve coverage for these files or not.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Jan 20, 2021 at 12:37 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks for updating the patch.
On Mon, Jan 4, 2021 at 6:52 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The most recent patch doesn't compile --without-lz4:
On Tue, Jan 05, 2021 at 11:19:33AM +0530, Dilip Kumar wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think I first saw it on cfbot and I reproduced it locally, too.
http://cfbot.cputube.org/dilip-kumar.htmlI think you'll have to make --without-lz4 the default until the build
environments include it, otherwise the patch checker will show red :(Oh ok, but if we make by default --without-lz4 then the test cases
will start failing which is using lz4 compression. Am I missing
something?The CIs are failing like this:
http://cfbot.cputube.org/dilip-kumar.html
|checking for LZ4_compress in -llz4... no
|configure: error: lz4 library not found
|If you have lz4 already installed, see config.log for details on the
|failure. It is possible the compiler isn't looking in the proper directory.
|Use --without-lz4 to disable lz4 support.I thought that used to work (except for windows). I don't see that anything
changed in the configure tests... Is it because the CI moved off travis 2
weeks ago ? I don't' know whether the travis environment had liblz4, and I
don't remember if the build was passing or if it was failing for some other
reason. I'm guessing historic logs from travis are not available, if they ever
were.I'm not sure how to deal with that, but maybe you'd need:
1) A separate 0001 patch *allowing* LZ4 to be enabled/disabled;
2) Current patchset needs to compile with/without LZ4, and pass tests in both
cases - maybe you can use "alternate test" output [0] to handle the "without"
case.
3) Eventually, the CI and build environments may have LZ4 installed, and then
we can have a separate debate about whether to enable it by default.[0] cp -iv src/test/regress/results/compression.out src/test/regress/expected/compression_1.out
I have done that so now default will be --without-lz4
On Tue, Jan 05, 2021 at 02:20:26PM +0530, Dilip Kumar wrote:
On Tue, Jan 5, 2021 at 11:19 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Jan 4, 2021 at 10:08 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I see the windows build is failing:
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.123730
|undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at src/tools/msvc/Mkvcbuild.pm line 852.
This needs to be patched: src/tools/msvc/Solution.pm
You can see my zstd/pg_dump patch for an example, if needed (actually I'm not
100% sure it's working yet, since the windows build failed for another reason).Okay, I will check that.
This still needs help.
perl ./src/tools/msvc/mkvcbuild.pl
...
undefined symbol: HAVE_LIBLZ4 at src/include/pg_config.h line 350 at /home/pryzbyj/src/postgres/src/tools/msvc/Mkvcbuild.pm line 852.Fix like:
+ HAVE_LIBLZ4 => $self->{options}->{zlib} ? 1 : undef,
I added HAVE_LIBLZ4 undef, but I haven't yet tested on windows as I
don't have a windows system. Later I will check this and fix if it
doesn't work.
Some more language fixes:
commit 3efafee52414503a87332fa6070541a3311a408c
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Sep 8 15:24:33 2020 +0530Built-in compression method
+ If the compression method is not specified for the compressible type then + it will have the default compression method. The default compressionI think this should say:
If no compression method is specified, then compressible types will have the
default compression method (pglz).+ * + * Since version 11 TOAST_COMPRESS_SET_RAWSIZE also marks compressedShould say v14 ??
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h index 059dec3647..e4df6bc5c1 100644 --- a/src/include/catalog/pg_attribute.h +++ b/src/include/catalog/pg_attribute.h @@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75, /* attribute's collation */ Oid attcollation;+ /* + * Oid of the compression method that will be used for compressing the value + * for this attribute. For the compressible atttypid this must always be asay "For compressible types, ..."
+ * valid Oid irrespective of what is the current value of the attstorage. + * And for the incompressible atttypid this must always be an invalid Oid.say "must be InvalidOid"
@@ -685,6 +686,7 @@ typedef enum TableLikeOption CREATE_TABLE_LIKE_INDEXES = 1 << 5, CREATE_TABLE_LIKE_STATISTICS = 1 << 6, CREATE_TABLE_LIKE_STORAGE = 1 << 7, + CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,This is interesting...
I have a patch to implement LIKE .. (INCLUDING ACCESS METHOD).
I guess I should change it to say LIKE .. (TABLE ACCESS METHOD), right ?
https://commitfest.postgresql.org/31/2865/Your first patch is large due to updating a large number of test cases to
include the "compression" column in \d+ output. Maybe that column should be
hidden when HIDE_TABLEAM is set by pg_regress ? I think that would allow
testing with alternate, default compression.
I am not sure whether we should hide the compression method when
HIDE_TABLEAM is set. I agree that it is actually an access method but
it is not the same right? Because we are using it for compression not
for storing data.
commit ddcae4095e36e94e3e7080e2ab5a8d42cc2ca843
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Tue Jan 19 15:10:14 2021 +0530Support compression methods options
+ * we don't need do it again in cminitstate function.
need *to* do it again
Fixed
+ * Fetch atttributes compression options
attribute's :)
Fixed
commit b7946eda581230424f73f23d90843f4c2db946c2
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Wed Jan 13 12:14:40 2021 +0530Create custom compression methods
+ * compression header otherwise, directly translate the buil-in compression
built-in
Fixed
commit 0746a4d7a14209ebf62fe0dc1d12999ded879cfd
Author: dilipkumar <dilipbalaut@gmail.com>
Date: Mon Jan 4 15:15:20 2021 +0530Add support for PRESERVE
--- a/src/backend/catalog/objectaddress.c +++ b/src/backend/catalog/objectaddress.c @@ -15,6 +15,7 @@#include "postgres.h"
+#include "access/compressamapi.h"
Unnecessary change to this file ?
Fixed
+ * ... Collect the list of access method + * oids on which this attribute has a dependency upon."upon" is is redundant. Say "on which this attribute has a dependency".
Changed
+ * Check whether the given compression method oid is supported by + * the target attribue.attribute
Fixed
+ * In binary upgrade mode just create the dependency for all preserve + * list compression method as a dependecy.dependency
I think you could say: "In binary upgrade mode, just create a dependency on all
preserved methods".
Fixed
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v20-0002-alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v20-0002-alter-table-set-compression.patchDownload
From 38d9a1c08ad89acc63f3684346b4b25d31d944a3 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v20 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++++
src/test/regress/expected/compression_1.out | 57 ++++++++
src/test/regress/sql/compression.sql | 21 +++
9 files changed, 329 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..c4b53ec 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +385,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. The Compression method can be
+ set to any available compression method. The built-in methods are
+ <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d7f4489..e29f16c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5527,6 +5535,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7768,6 +7779,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7782,7 +7858,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7921,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15034,6 +15070,92 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 355e273..f2e9fe4 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2307,6 +2307,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 17f7265..7aeabf9 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d..5a314f4 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 232c9f1..ffb739f 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,7 +1875,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4dea269..84d31c7 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -142,6 +142,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index a50c767..3250c12 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -139,6 +139,63 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0750a6c..3b0da88 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -65,6 +65,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v20-0003-Add-support-for-PRESERVE.patchtext/x-patch; charset=UTF-8; name=v20-0003-Add-support-for-PRESERVE.patchDownload
From 46be791237118717842d6a5abf2bd4dfc20f3ef4 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:15:20 +0530
Subject: [PATCH v20 3/6] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. For
supporting this the column will maintain the dependency with all
the supported compression methods. So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 293 ++++++++++++++++++++++++++++
src/backend/commands/tablecmds.c | 116 ++++++-----
src/backend/executor/nodeModifyTable.c | 13 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 101 ++++++++++
src/bin/pg_dump/pg_dump.h | 14 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 +++-
src/test/regress/expected/compression_1.out | 39 +++-
src/test/regress/expected/create_index.out | 56 +++---
src/test/regress/sql/compression.sql | 9 +
22 files changed, 732 insertions(+), 98 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c4b53ec..081fe07 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. The Compression method can be
set to any available compression method. The built-in methods are
<literal>pglz</literal> and <literal>lz4</literal>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da243..dd37648 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0..a7395ad 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000..112998c
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,293 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribute.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode, just create a dependency on all preserved
+ * methods.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e29f16c..7e17aaa 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rel = relation_open(relationId, AccessExclusiveLock);
/*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
+ /*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
* parsetrees; we need to transform them to executable expression trees
@@ -2415,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4908,7 +4926,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6402,7 +6421,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6577,6 +6597,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6725,6 +6746,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
+/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11859,7 +11902,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11974,6 +12018,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15078,24 +15127,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15128,12 +15174,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
+
+ atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -17857,32 +17908,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 684e836..916f654 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2054,6 +2055,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -2062,7 +2064,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop for all the attributes in the tuple and check if any of the
* attribute is compressed in the source tuple and its compression method
- * is not same as the target compression method then we need to decompress
+ * is not supported by the target attribute then we need to decompress
* it.
*/
for (i = 0; i < natts; i++)
@@ -2083,11 +2085,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3ea3232..6588c84 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,7 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2952,6 +2952,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5640,6 +5652,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 227bb07..3aa6176 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,7 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2619,6 +2619,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
}
static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
+static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
COMPARE_SCALAR_FIELD(contype);
@@ -3693,6 +3703,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 187d357..4f7053d 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,7 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2884,6 +2884,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
}
static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
+static void
_outTypeName(StringInfo str, const TypeName *node)
{
WRITE_NODE_TYPE("TYPENAME");
@@ -4230,6 +4240,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f2e9fe4..332e60f 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -595,7 +595,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2308,12 +2310,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3436,7 +3438,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3491,13 +3493,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e2ac159..a2a6b21 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index b5c34e2..57df7b9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9010,6 +9010,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16326,6 +16400,33 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 53ece7a..2a75132 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,7 +328,7 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -357,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 7aeabf9..c40d498 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2103,6 +2103,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e5aea8a..bd53f9b 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index b3d30ac..e6c98e6 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -97,5 +97,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ded2ad9..d630c54 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ffb739f..fd9c6ab 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,20 @@ typedef struct RangeTableSample
} RangeTableSample;
/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
+/*
* ColumnDef - column definition (used in various creates)
*
* If the column has a default value, we may have the value expression
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 84d31c7..151478b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -201,12 +201,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3250c12..81182fe 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -196,12 +196,49 @@ SELECT pg_column_compression(f1) FROM cmpart;
pglz
(1 row)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+HINT: use "pg_column_compression" function for list of compression methods
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ce734f7..b61c627 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 3b0da88..e447316 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -86,6 +86,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v20-0005-new-compression-method-extension-for-zlib.patchtext/x-patch; charset=US-ASCII; name=v20-0005-new-compression-method-extension-for-zlib.patchDownload
From 1bbf5c3058b8d2780f98ec03b82f96f4bfbff826 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v20 5/6] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 ++++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 ++
contrib/cmzlib/expected/cmzlib.out | 53 +++++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index cdc041c..6c61872 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000..5dcb3ff
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000..956fbe7
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000..41f2f95
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000..686a7c7
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000..2eb10f3
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000..2b6fac7
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ pglz
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000..ea8d206
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
1.8.3.1
v20-0001-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v20-0001-Built-in-compression-method.patchDownload
From 84fcef685cf3c9ba011284ceaac8ad499bc02c7a Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v20 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++---
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 +++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 ++++---
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++---
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++++++
src/backend/access/compression/compress_pglz.c | 131 ++++++++++++
src/backend/access/compression/compressamapi.c | 61 ++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 26 ++-
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 +++++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 +++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 46 ++++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 +++-
src/include/access/compressamapi.h | 65 ++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 24 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 176 ++++++++++++++++
src/test/regress/expected/compression_1.out | 171 ++++++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 ++++++-------
src/test/regress/expected/create_table_like.out | 88 ++++----
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 ++++++++++++------------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 ++++++-------
src/test/regress/expected/insert.out | 118 +++++------
src/test/regress/expected/matview.out | 80 ++++----
src/test/regress/expected/psql.out | 108 +++++-----
src/test/regress/expected/publication.out | 40 ++--
src/test/regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +--
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 74 +++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
81 files changed, 2193 insertions(+), 668 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index e202697..a7b4317 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -865,6 +866,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8597,6 +8600,35 @@ fi
#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12087,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13292,6 +13377,20 @@ fi
fi
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index a5ad072..ce29d6e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -996,6 +996,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1182,6 +1189,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1397,6 +1412,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044..6ee0776 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a462..5bf614c 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from its parent table, however a different compression method can be set
+ for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9..d514c7f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -606,6 +607,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. The Compression
+ method can be set from available compression methods. The built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If no compression method is specified, then compressible types will have
+ the default compression method <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..c904640 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +512,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1d43d5d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..c4814ef
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..e8b035d
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..578be6f
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000..663102c
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 009e215..44fa2ca 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..b53b6b5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b8cd35e..e604092 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..668e43f 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -34,6 +34,8 @@
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -83,7 +85,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -176,6 +190,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..59690ce 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -582,6 +582,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..41c66c9 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -487,6 +487,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e..d7f4489 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5d90337..684e836 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2036,6 +2039,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2319,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba3ccc7..3ea3232 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,6 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a2ef853..227bb07 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,6 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6be1991..c6ba9e1 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3858,6 +3858,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8392be6d..187d357 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,6 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7574d54..355e273 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -595,6 +595,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -630,9 +632,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3420,11 +3422,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3433,8 +3436,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3479,6 +3482,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3709,6 +3720,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15237,6 +15249,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15755,6 +15768,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3af..e2ac159 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 5a62ab8..43eee8a 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95..f026104 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -129,6 +129,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
}
Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
+Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
text *extName;
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..aff5b7e 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,50 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a..5e168a3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 798d145..b5c34e2 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -13016,6 +13035,11 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -15805,6 +15829,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15829,6 +15854,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15864,6 +15892,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1290f96..53ece7a 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..4f82929 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..8226ae0
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..84ba303 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 14 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7..9d65bae 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..00362d9 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 358f5aa..4a5c249 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 059dec3..a6cb1b4 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For compressible types this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible types this must be InvalidOid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b5f52d4..7458ed7 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7087,6 +7096,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7257,6 +7270,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
@@ -10749,6 +10769,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '1137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 56da291..984002b 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -625,6 +625,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 758c3ca..36d32f9 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -616,5 +616,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index caed683..ded2ad9 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -510,6 +510,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dc2bb40..232c9f1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 8c554e1..89bd2fd 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f4d9f3b..47f114b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..90c7454 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4..138df4b 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..4dea269
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,176 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..a50c767
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,171 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: not built with lz4 support
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+--------
+(0 rows)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+(0 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f071..1778c15 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b..3105115 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be..4da878e 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c0..6268b26 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e2582..89466da 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca033..dc2af07 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef..7e70991 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3..3cd4ad9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0..e078818 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..d6912f2 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7..71388c1 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 7900219..20ac001 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aae..7ebdc30 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6173473..8546f02 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3035,11 +3035,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3086,11 +3086,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 431b3fa..a36b7e6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -125,11 +125,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d7..58e996f 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 94c5f02..458c297 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e0e1ef7..2407ca9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,6 +116,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 081fce3..152c8cb 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -203,3 +203,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..0750a6c
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,74 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2f28de0..9ebfe19 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 721b230..49a62d6 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v20-0004-Create-custom-compression-methods.patchtext/x-patch; charset=US-ASCII; name=v20-0004-Create-custom-compression-methods.patchDownload
From 2b12cef37af56f27c9c3be254c067137abb15583 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 29 Jan 2021 15:44:09 +0530
Subject: [PATCH v20 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 7 +--
doc/src/sgml/ref/create_access_method.sgml | 13 +++---
doc/src/sgml/ref/create_table.sgml | 3 +-
src/backend/access/common/detoast.c | 59 +++++++++++++++++++++++---
src/backend/access/common/toast_internals.c | 19 +++++++--
src/backend/access/compression/compress_lz4.c | 21 ++++-----
src/backend/access/compression/compress_pglz.c | 20 ++++-----
src/backend/access/compression/compressamapi.c | 27 ++++++++++++
src/backend/access/index/amapi.c | 50 ++++++++++++++++------
src/backend/commands/amcmds.c | 5 +++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 ++
src/bin/psql/tab-complete.c | 6 +++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 16 +++++--
src/include/access/detoast.h | 8 ++++
src/include/access/toast_internals.h | 15 +++++++
src/test/regress/expected/compression.out | 40 ++++++++++++++++-
src/test/regress/expected/compression_1.out | 41 +++++++++++++++++-
src/test/regress/sql/compression.sql | 10 +++++
22 files changed, 312 insertions(+), 62 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 081fe07..46f34bf 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,9 +390,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. The Compression method can be
- set to any available compression method. The built-in methods are
- <literal>pglz</literal> and <literal>lz4</literal>.
+ This clause adds compression to a column. The Compression method could be
+ created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The built-in methods
+ are <literal>pglz</literal> and <literal>lz4</literal>.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43db..79f1290 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index d514c7f..10b468f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -998,7 +998,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. The Compression
- method can be set from available compression methods. The built-in
+ method could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set from the available compression methods. The built-in
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If no compression method is specified, then compressible types will have
the default compression method <literal>pglz</literal>.
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index c904640..e032c30 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -458,12 +458,43 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the built-in
+ * compression id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
+/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -477,10 +508,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -496,9 +538,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -512,16 +556,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index b04c5a5..a353906 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index e8b035d..34b0dc1 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -30,7 +30,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -45,10 +45,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -56,7 +56,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -68,7 +68,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -81,9 +81,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -100,7 +100,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -112,9 +113,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 578be6f..d33ea3d 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c..7aea8aa 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -59,3 +60,29 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43..89a4b01 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -46,14 +46,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +64,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +93,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 668e43f..31d4a3e 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -241,6 +241,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -278,6 +280,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 916f654..a637372 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2089,7 +2089,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* compression method is not supported by the target attribute then
* we need to decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 332e60f..0a0ea05 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5304,6 +5304,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index aff5b7e..ebd7971 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5309,7 +5309,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5339,10 +5338,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 57df7b9..ca2063e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13124,6 +13124,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index c40d498..5de2a6a 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb..0f7b5b0 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -285,6 +285,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0..913a633 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -26,18 +26,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -61,5 +68,6 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 84ba303..c158354 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 151478b..636ab40 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -235,13 +235,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 81182fe..3177ff9 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -232,13 +232,52 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index e447316..0552eec 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -95,6 +95,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v20-0006-Support-compression-methods-options.patchtext/x-patch; charset=US-ASCII; name=v20-0006-Support-compression-methods-options.patchDownload
From ac3c29097093295766169b05084c42fd56539aa2 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 29 Jan 2021 15:46:33 +0530
Subject: [PATCH v20 6/6] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 +++++++++++++++---
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 ++++++++++++++++
src/backend/access/common/toast_internals.c | 14 +++-
src/backend/access/compression/compress_lz4.c | 69 ++++++++++++++++-
src/backend/access/compression/compress_pglz.c | 98 ++++++++++++++++++++++--
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 43 +++++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 ++++++++++++++++++-
src/backend/commands/foreigncmds.c | 44 -----------
src/backend/commands/tablecmds.c | 102 ++++++++++++++++++-------
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 23 +++++-
src/bin/pg_dump/pg_dump.h | 2 +
src/include/access/compressamapi.h | 19 ++++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 ++-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 +++++++++++++
src/test/regress/expected/compression_1.out | 55 +++++++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 +++++
36 files changed, 717 insertions(+), 132 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7..d8c2865 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -46,29 +47,83 @@ typedef struct
} zlib_state;
/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
+/*
* zlib_cmcompress - compression routine for zlib compression method
*
* Compresses source into dest using the default compression level.
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 46f34bf..e496f02 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -393,7 +393,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
This clause adds compression to a column. The Compression method could be
created with <xref linkend="sql-create-access-method"/> or
it can be set to any available compression method. The built-in methods
- are <literal>pglz</literal> and <literal>lz4</literal>.
+ are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method has options they can be specified with the
+ <literal>WITH</literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 10b468f..e5514bb 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. The Compression
@@ -1002,7 +1002,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
it can be set from the available compression methods. The built-in
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If no compression method is specified, then compressible types will have
- the default compression method <literal>pglz</literal>.
+ the default compression method <literal>pglz</literal>. If the compression
+ method has options they can be specified with the <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712..76f824b 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d..0d3307e 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3e..2dda6c0 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a353906..5ed3312 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 34b0dc1..375c4e8 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,6 +16,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -24,13 +25,70 @@
#endif
/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
+/*
* lz4_cmcompress - compression routine for lz4 compression method
*
* Compresses source into dest using the default strategy. Returns the
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -40,6 +98,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -47,9 +106,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d33ea3d..0396729 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9..c9d44c6 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004..a43e0e1 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b53b6b5..18b0481 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -741,6 +741,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -796,6 +797,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -843,6 +849,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -861,7 +868,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -893,7 +901,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1160,6 +1168,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1418,7 +1427,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e604092..2a39e0c 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481..348b7d3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 096a06f..0ad946d 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -699,6 +699,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 112998c..de57b55 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -29,6 +29,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -181,7 +182,7 @@ BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -207,6 +208,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -279,15 +294,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttribute's compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103f..ae9ae2c 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7e17aaa..cc3a0cb 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -609,6 +609,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -813,6 +814,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -866,8 +869,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -921,8 +925,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2432,16 +2439,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2478,8 +2483,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2729,14 +2733,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6259,6 +6258,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6422,6 +6422,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6432,7 +6433,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7829,13 +7830,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7874,7 +7882,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7965,7 +7995,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15137,9 +15167,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15174,7 +15206,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15184,8 +15217,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15193,8 +15235,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6588c84..69e370d 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2959,6 +2959,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3aa6176..1e3a01f 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2623,6 +2623,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4f7053d..283452e 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2890,6 +2890,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0a0ea05..2d7b307 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3502,11 +3503,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3514,14 +3521,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a2a6b21..47fddc2 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index ca2063e..a7e8938 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8709,10 +8709,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8765,6 +8772,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8794,6 +8802,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15932,7 +15941,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -15983,6 +15993,10 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
@@ -16414,6 +16428,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, "\nWITH (%s) ", tbinfo->attcmoptions[j]);
+
if (cminfo->nitems > 0)
{
appendPQExpBuffer(q, "\nPRESERVE (");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 2a75132..47ed111 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,6 +328,8 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
+ char **attcmoptions; /* per-attribute current compression
+ options */
struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 913a633..8123cc8 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -37,8 +38,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -49,14 +53,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc3..fe8de40 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index c158354..3b143b8 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b..5ecb6f3 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a6cb1b4..71cbc99 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bd53f9b..8271e97 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -134,6 +134,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -146,9 +148,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fd9c6ab..a1b1158 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 636ab40..9d2c00f 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -272,6 +272,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3177ff9..4c0952b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -270,6 +270,61 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ERROR: not built with lz4 support
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+HINT: use "pg_column_compression" function for list of compression methods
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afee..27aab82 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0552eec..59a32dd 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
Re: [HACKERS] Custom compression methods
+ * Helper function for ATExecSetStorage and ATExecSetCompression + * + * Set the attcompression or attstorage for the respective index attribute. If + * attcompression is a valid oid then it will update the attcompression for the + * index attribute otherwise it will update the attstorage. + */ +static void +ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum, + Oid newcompression, char newstorage, LOCKMODE lockmode)
The comment should refer to "newcompression" not attcompression.
Maybe you'd want to assert that exactly one of these is valid.
Or maybe you should allow setting *both* if they're both valid.
case AMTYPE_TABLE: return "TABLE"; + case AMTYPE_COMPRESSION: + return "TABLE";
Looks like this part got confused, maybe during rebase.
+ if (strcmp(def->defname, "acceleration") == 0) + { + int32 acceleration = + pg_atoi(defGetString(def), sizeof(acceleration), 0); + + if (acceleration < INT32_MIN || acceleration > INT32_MAX)
Is this even reachable given that it's int32 type ?
What do you think about this idea:
On Sun, Jan 03, 2021 at 07:22:20PM -0600, Justin Pryzby wrote:
I think there should also be an option for pg_restore, like --no-tablespaces.
And I think there should be a GUC for default_compression, like
default_table_access_method, so one can restore into an alternate compression
by setting PGOPTIONS=-cdefault_compression=lz4.I'd like to be able to make all compressible columns of a table use a
non-default compression (except those which cannot), without having to use
\gexec... We have tables with up to 1600 columns. So a GUC would allow that.Previously (on separate threads) I wondered whether pg_dump
--no-table-access-method was needed - maybe that be sufficient for this case,
too, but I think it should be possible to separately avoid restoring
compression AM and AM "proper". So maybe it'd be like --no-tableam=compress
--no-tableam=storage or --no-tableam-all.
I implemented the GUC. I'm sending a patchset rebased with this as 0002, and
an un-rebased version in case you prefer that.
I think the pg_dump half (--no-acccess-method=compress) is independently useful
for tableams.
Your first patch is large due to updating a large number of test cases to
include the "compression" column in \d+ output. Maybe that column should be
hidden when HIDE_TABLEAM is set by pg_regress ? I think that would allow
testing with alternate, default compression.I am not sure whether we should hide the compression method when
HIDE_TABLEAM is set. I agree that it is actually an access method but
it is not the same right? Because we are using it for compression not
for storing data.
You're right that TABLEAM should only be for table AM's. So I suggest to do
something similar for HIDE_COMPRESSAM. That allows testing with an alternate,
default compression (which is possible with the added GUC).
Language/style fixen to follow:
+ /* acoid should be found in some cases */
amoid
+-- test alter compression method for the partioned table
partitioned
+ /* + * Loop for all the attributes in the tuple and check if any of the + * attribute is compressed in the source tuple and its compression method + * is not same as the target compression method then we need to decompress + * it.
Loop *over*
any attribute is compressed (or any of the attributes *are* compressed)
+ * If we have decompressed any of the field in the source tuple then free + * the existing tuple.
any field (or any of the fields)
+ /* Likewise, copy compression if requested */ + if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION + && OidIsValid(attribute->attcompression))
Should say (.. & ..) != 0 && OidIsValid()
+ else if (strcmp(def->compression, compression))
I think postgres likes you to explicitly write != 0
+ n->def = (Node *) makeString($5);;
The 0002 patch has this double semicolon, which is removed by 0003.
+ By default, each column in a partition inherits the compression method + from its parent table, however a different compression method can be set + for each partition.
I guess it should say that it inherits from the parent table's *column*.
This clause specifies the type of access method to define. - Only <literal>TABLE</literal> and <literal>INDEX</literal> + <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal> are supported at present.
I think "ONLY" got lost (?)
Maybe it should say: "Currently, TABLE, INDEX, and COMPRESSION access methods
are supported.
--
Justin
Attachments:
v21-0001-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From 6aab69e3196e2e15b679050e2db2fc9907624715 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Tue, 8 Sep 2020 15:24:33 +0530
Subject: [PATCH v21 1/7] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 99 +++++++
configure.ac | 22 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 147 ++++++++++
.../access/compression/compress_pglz.c | 131 +++++++++
.../access/compression/compressamapi.c | 61 +++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 6 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 26 +-
src/backend/commands/createas.c | 7 +
src/backend/commands/matview.c | 7 +
src/backend/commands/tablecmds.c | 101 ++++++-
src/backend/executor/nodeModifyTable.c | 83 ++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
.../replication/logical/reorderbuffer.c | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 46 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/bin/psql/describe.c | 42 ++-
src/include/access/compressamapi.h | 65 +++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 22 +-
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_am.dat | 7 +-
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 10 +-
src/include/catalog/pg_proc.dat | 24 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 176 ++++++++++++
src/test/regress/expected/compression_1.out | 171 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 3 +
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 74 +++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
81 files changed, 2193 insertions(+), 668 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/backend/access/compression/compressamapi.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index e202697bbf..a7b4317b90 100755
--- a/configure
+++ b/configure
@@ -698,6 +698,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -865,6 +866,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
enable_largefile
'
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --without-lz4 do not use lz4
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
Some influential environment variables:
@@ -8596,6 +8599,35 @@ fi
+#
+# lz4
+#
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+ :
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12087,6 +12119,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13290,6 +13375,20 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+
+else
+ as_fn_error $? "lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index a5ad072ee4..ce29d6e97f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -995,6 +995,13 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# lz4
+#
+PGAC_ARG_BOOL(with, lz4, yes,
+ [do not use lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1182,6 +1189,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1397,6 +1412,13 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADER(lz4.h, [], [AC_MSG_ERROR([lz4 header not found
+If you have lz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a4625cc..5bf614cc8a 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from its parent table, however a different compression method can be set
+ for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..d514c7f8c1 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This clause adds the compression method to a column. The Compression
+ method can be set from available compression methods. The built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If no compression method is specified, then compressible types will have
+ the default compression method <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..c90464053c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +512,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..b04c5a5eb8 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..c4814ef911
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o compressamapi.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..e8b035d138
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,147 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2016-2020, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1990-1993, Regents of the University of California
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#ifdef HAVE_LIBLZ4
+#include "lz4.h"
+#endif
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ {
+ pfree(tmp);
+ elog(ERROR, "lz4: could not compress data");
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ elog(ERROR, "lz4: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..578be6f1dd
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,131 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+
+ if (len >= 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ return tmp;
+ }
+
+ pfree(tmp);
+
+ return NULL;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+
+ if (rawsize < 0)
+ elog(ERROR, "pglz: compressed data is corrupted");
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
new file mode 100644
index 0000000000..663102c8d2
--- /dev/null
+++ b/src/backend/access/compression/compressamapi.c
@@ -0,0 +1,61 @@
+/*-------------------------------------------------------------------------
+ *
+ * compression/compressamapi.c
+ * Functions for compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compressamapi.c
+ *-------------------------------------------------------------------------
+ */
+
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/htup.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "access/table.h"
+#include "utils/fmgroids.h"
+#include "utils/syscache.h"
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "Invalid compression method id %d", cmid);
+ }
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index 009e215da1..44fa2ca7b4 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -181,6 +181,9 @@ my $C_COLLATION_OID =
my $PG_CATALOG_NAMESPACE =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_namespace},
'PG_CATALOG_NAMESPACE');
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
@@ -808,6 +811,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1f55..b53b6b50e6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b8cd35e995..e6040923dc 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -347,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..668e43faed 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -34,6 +34,8 @@
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -83,7 +85,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* acoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -175,6 +189,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..59690ce854 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -581,6 +581,13 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..41c66c9e61 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -486,6 +486,13 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(slot, myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..d7f4489c57 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,32 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * Get compression method Oid for the attribute.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ return get_compression_am_oid(compression, false);
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5d90337498..684e836071 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,12 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2036,6 +2039,78 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value.
+ */
+void
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool isnull = false;
+ bool decompressed_any = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return;
+
+ /*
+ * Loop for all the attributes in the tuple and check if any of the
+ * attribute is compressed in the source tuple and its compression method
+ * is not same as the target compression method then we need to decompress
+ * it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value = (struct varlena *)
+ DatumGetPointer(slot_getattr(slot, attnum, &isnull));
+
+ /* nothing to be done, if the value is null */
+ if (isnull)
+ continue;
+
+ /* nothing to be done. if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the field in the source tuple then free
+ * the existing tuple.
+ */
+ if (decompressed_any)
+ {
+ /* deform complete tuple before we clear the existing tuple */
+ slot_getallattrs(slot);
+
+ ExecClearTuple(slot);
+ slot->tts_nvalid = natts;
+ slot->tts_flags &= ~TTS_FLAG_EMPTY;
+ }
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2319,14 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ CompareCompressionMethodAndDecompress(
+ slot, resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index ba3ccc712c..3ea32323cd 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,6 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index a2ef853dc2..227bb07bbd 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,6 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 6be19916fc..c6ba9e1b05 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3858,6 +3858,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8392be6d44..187d3570e5 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,6 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 7574d545e0..355e273192 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -595,6 +595,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -630,9 +632,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3420,11 +3422,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3433,8 +3436,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3479,6 +3482,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3709,6 +3720,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15237,6 +15249,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15755,6 +15768,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..e2ac159aa2 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 5a62ab8bbc..43eee8a4b7 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -87,6 +87,7 @@
#include "access/detoast.h"
#include "access/heapam.h"
#include "access/rewriteheap.h"
+#include "access/toast_internals.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95079..f026104c01 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -128,6 +128,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..aff5b7e95e 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_internals.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5299,6 +5301,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 39da742e32..becf565cdd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -13016,6 +13035,11 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -15805,6 +15829,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15829,6 +15854,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15864,6 +15892,22 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Compression
+ *
+ * In binary-upgrade mode, compression is assigned by
+ * ALTER. Even if we're skipping compression the attribute
+ * will get default compression. It's the task for ALTER
+ * command to restore compression info.
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ has_non_default_compression)
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1290f9659b..53ece7abd7 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
+
/*
* Stuff computed only for dumpable tables.
*/
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..4f82929fdb 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,27 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ bool mustfree = false;
+ const int trunclen = 100;
+ char *val = PQgetvalue(res, i, attcompression_col);
+
+ /* truncate the options if they're too long */
+ if (strlen(val) > trunclen + 3)
+ {
+ char *trunc = pg_malloc0(trunclen + 4);
+ strncpy(trunc, val, trunclen);
+ strncpy(trunc + trunclen, "...", 4);
+
+ val = trunc;
+ mustfree = true;
+ }
+
+ printTableAddCell(&cont, val, false, mustfree);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..8226ae0596
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,65 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2015-2017, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/* access/compression/compressamapi.c */
+extern CompressionId CompressionOidToId(Oid cmoid);
+extern Oid CompressionIdToOid(CompressionId cmid);
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..84ba303e7e 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,33 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
+#define RAWSIZEMASK (0x3FFFFFFFU)
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 14 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= RAWSIZEMASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7ac5..9d65bae238 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..00362d9db3 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,10 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
-
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index 358f5aac8f..4a5c249ee9 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX(pg_am_oid_index, 2652, on pg_am using btree(oid oid_ops));
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 059dec3647..a6cb1b4e5b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -156,6 +156,14 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation */
Oid attcollation;
+ /*
+ * Oid of the compression method that will be used for compressing the value
+ * for this attribute. For compressible types this must always be a
+ * valid Oid irrespective of what is the current value of the attstorage.
+ * And for the incompressible types this must be InvalidOid.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -183,7 +191,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index b5f52d4e4a..7458ed7d0d 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7087,6 +7096,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7257,6 +7270,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
@@ -10749,6 +10769,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '1137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 56da2913bd..984002ba21 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -625,6 +625,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540c94..e5aea8a240 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 758c3ca097..36d32f94aa 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -616,5 +616,6 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern void CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index caed683ba9..ded2ad9692 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -510,6 +510,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index dc2bb40926..232c9f1248 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 8c554e1f69..89bd2fd7a8 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -87,6 +87,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index f4d9f3b408..47f114b8a0 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..90c7454d2a 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,9 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
- * This struct must not contain any padding, because we sometimes compare
- * these pointers using memcmp.
+ * The data is compressed if and only if size in
+ * va_extinfo < va_rawsize - VARHDRSZ. This struct must not contain any
+ * padding, because we sometimes compare these pointers using memcmp.
*
* Note that this information is stored unaligned within actual tuples, so
* you need to memcpy from the tuple into a local struct variable before
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,11 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & 0x3FFFFFFF)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> 30)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..4dea269fc3
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,176 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..a50c7679bb
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,171 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: not built with lz4 support
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+--------
+(0 rows)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+(0 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..e078818496 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6173473de9..8546f02c1c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3035,11 +3035,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3086,11 +3086,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 431b3fa3de..a36b7e6069 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -125,11 +125,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 94c5f023c6..458c297670 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -151,10 +151,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -171,10 +171,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e0e1ef71dd..2407ca945b 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -116,6 +116,9 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+#test toast compression
+test: compression
+
# event triggers cannot run concurrently with any test that runs DDL
test: event_trigger
# this test also uses event triggers, so likewise run it by itself
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 081fce32e7..152c8cb5b7 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -203,3 +203,4 @@ test: explain
test: event_trigger
test: fast_default
test: stats
+test: compression
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..0750a6c80e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,74 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2f28de0355..9ebfe19a34 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 721b230bf2..49a62d6e33 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
v21-0002-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From b49680d96644c4a83f081f490a354363f81d7091 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH v21 2/7] Add default_toast_compression GUC
---
src/backend/access/common/tupdesc.c | 3 +-
.../access/compression/compressamapi.c | 54 +++++++++++++++++++
src/backend/bootstrap/bootstrap.c | 7 ++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/cache/syscache.c | 6 ++-
src/backend/utils/misc/guc.c | 12 +++++
src/include/access/compressamapi.h | 13 ++++-
7 files changed, 92 insertions(+), 7 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab487..b56b689493 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -25,6 +25,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
+#include "commands/defrem.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
@@ -668,7 +669,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = get_compression_am_oid(default_toast_compression, false);
else
att->attcompression = InvalidOid;
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 663102c8d2..e8bfb23ec3 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,9 +19,12 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "commands/defrem.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION; // maybe needs pg_dump support ?
+
/*
* CompressionOidToId - Convert compression Oid to built-in compression id.
*
@@ -59,3 +62,54 @@ CompressionIdToOid(CompressionId cmid)
elog(ERROR, "Invalid compression method id %d", cmid);
}
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451eaa71..1b885f873a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -29,6 +29,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
+#include "commands/defrem.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -732,8 +733,12 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ {
+ /* Cannot call get_compression_am_oid this early */
+ attrtypes[attnum]->attcompression = PGLZ_COMPRESSION_AM_OID;
+ }
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d7f4489c57..082db26bfa 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = get_compression_am_oid(default_toast_compression, false);
}
else
attTup->attcompression = InvalidOid;
@@ -17760,7 +17760,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return get_compression_am_oid(default_toast_compression, false);
return get_compression_am_oid(compression, false);
}
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index e4dc4ee34e..74e42a3394 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -1127,8 +1127,10 @@ HeapTuple
SearchSysCache1(int cacheId,
Datum key1)
{
- Assert(cacheId >= 0 && cacheId < SysCacheSize &&
- PointerIsValid(SysCache[cacheId]));
+ Assert(cacheId >= 0);
+ Assert(cacheId < SysCacheSize);
+ // fprintf(stderr, "cache %d\n", cacheId);
+ Assert(PointerIsValid(SysCache[cacheId]));
Assert(SysCache[cacheId]->cc_nkeys == 1);
return SearchCatCache1(SysCache[cacheId], key1);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index eafdb1118e..70b26ea72a 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8226ae0596..be5b4da7b8 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -15,8 +15,12 @@
#include "postgres.h"
+#include "access/xact.h"
#include "catalog/pg_am_d.h"
+#include "miscadmin.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
+#include "nodes/pg_list.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +33,11 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+extern char *default_toast_compression;
+
/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
@@ -62,4 +69,8 @@ extern const CompressionAmRoutine lz4_compress_methods;
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern void assign_default_toast_compression(const char *newval, void **extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+
#endif /* COMPRESSAMAPI_H */
--
2.17.0
v21-0003-alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From 8e5276e8fd282078ca3e741b230fda897b9a6c7e Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v21 3/7] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 14 ++
src/backend/commands/tablecmds.c | 208 ++++++++++++++++----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 59 ++++++
src/test/regress/expected/compression_1.out | 57 ++++++
src/test/regress/sql/compression.sql | 21 ++
9 files changed, 329 insertions(+), 45 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..c4b53ec1c5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +384,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This clause adds compression to a column. The Compression method can be
+ set to any available compression method. The built-in methods are
+ <literal>pglz</literal> and <literal>lz4</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 082db26bfa..89cca760c5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5527,6 +5535,9 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ CompareCompressionMethodAndDecompress(oldslot, newTupDesc);
+
/* copy attributes */
memcpy(newslot->tts_values, oldslot->tts_values,
sizeof(Datum) * oldslot->tts_nvalid);
@@ -7767,6 +7778,71 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression or attstorage for the respective index attribute. If
+ * attcompression is a valid oid then it will update the attcompression for the
+ * index attribute otherwise it will update the attstorage.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ /*
+ * If a valid compression method Oid is passed then update the
+ * attcompression, otherwise update the attstorage.
+ */
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+ else
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7858,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7921,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15069,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 355e273192..f2e9fe4ff8 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2307,6 +2307,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);;
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 17f7265038..7aeabf9543 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 232c9f1248..ffb739f565 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1875,7 +1875,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4dea269fc3..84d31c781b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -142,6 +142,65 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index a50c7679bb..3250c12859 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -139,6 +139,63 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0750a6c80e..3b0da88c19 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -65,6 +65,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v21-0004-Add-support-for-PRESERVE.patchtext/x-diff; charset=iso-8859-1Download
From d6026a83e26299a5f79a4151b78b150f7d4af13c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 4 Jan 2021 15:15:20 +0530
Subject: [PATCH v21 4/7] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. Â For
supporting this the column will maintain the dependency with all
the supported compression methods. Â So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. Â If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 293 ++++++++++++++++++++
src/backend/commands/tablecmds.c | 116 ++++----
src/backend/executor/nodeModifyTable.c | 13 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 101 +++++++
src/bin/pg_dump/pg_dump.h | 14 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/compression_1.out | 39 ++-
src/test/regress/expected/create_index.out | 56 ++--
src/test/regress/sql/compression.sql | 9 +
22 files changed, 732 insertions(+), 98 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c4b53ec1c5..081fe078a4 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -386,13 +386,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
This clause adds compression to a column. The Compression method can be
set to any available compression method. The built-in methods are
<literal>pglz</literal> and <literal>lz4</literal>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da24322d..dd376484b7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..50f0f7faaa
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,293 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribute.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression != NULL)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ return InvalidOid;
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return get_compression_am_oid(default_toast_compression, false);
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode, just create a dependency on all preserved
+ * methods.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p),
+ errhint("use \"pg_column_compression\" function for list of compression methods")));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 89cca760c5..9d732fc167 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +937,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2415,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression))
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4908,7 +4926,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6402,7 +6421,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6577,6 +6597,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6724,6 +6745,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11859,7 +11902,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11974,6 +12018,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15078,24 +15127,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15128,11 +15174,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17857,32 +17908,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * Get compression method Oid for the attribute.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression != NULL)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- return InvalidOid;
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return get_compression_am_oid(default_toast_compression, false);
-
- return get_compression_am_oid(compression, false);
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 684e836071..916f6541c1 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -45,6 +45,7 @@
#include "access/toast_internals.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2054,6 +2055,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
int natts = slot->tts_tupleDescriptor->natts;
bool isnull = false;
bool decompressed_any = false;
+ Oid cmoid = InvalidOid;
TupleDesc tupleDesc = slot->tts_tupleDescriptor;
if (natts == 0)
@@ -2062,7 +2064,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop for all the attributes in the tuple and check if any of the
* attribute is compressed in the source tuple and its compression method
- * is not same as the target compression method then we need to decompress
+ * is not supported by the target attribute then we need to decompress
* it.
*/
for (i = 0; i < natts; i++)
@@ -2083,11 +2085,12 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
continue;
/*
- * Get the compression method stored in the toast header and
- * compare with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
- if (targetTupDesc->attrs[i].attcompression !=
- CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value)))
+ cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 3ea32323cd..6588c84c4c 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2932,7 +2932,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2952,6 +2952,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5640,6 +5652,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 227bb07bbd..3aa6176a30 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2598,7 +2598,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2618,6 +2618,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3693,6 +3703,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 187d3570e5..4f7053d63b 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2865,7 +2865,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2883,6 +2883,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4230,6 +4240,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f2e9fe4ff8..332e60f170 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -595,7 +595,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2308,12 +2310,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);;
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3436,7 +3438,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3491,13 +3493,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e2ac159aa2..a2a6b218f3 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index becf565cdd..c09f3972c6 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9010,6 +9010,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16326,6 +16400,33 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 53ece7abd7..2a751321f5 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,7 +328,7 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -357,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 7aeabf9543..c40d498a30 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2103,6 +2103,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e5aea8a240..bd53f9bb0f 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index b3d30acc35..e6c98e65d4 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -97,5 +97,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index ded2ad9692..d630c54efa 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -479,6 +479,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ffb739f565..fd9c6abddf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 84d31c781b..151478bcdd 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -201,12 +201,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3250c12859..81182fe949 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -196,12 +196,49 @@ SELECT pg_column_compression(f1) FROM cmpart;
pglz
(1 row)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+HINT: use "pg_column_compression" function for list of compression methods
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ce734f7ef3..b61c6278a2 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 3b0da88c19..e447316573 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -86,6 +86,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v21-0005-Create-custom-compression-methods.patchtext/x-diff; charset=us-asciiDownload
From d00579f5a4bf5ea6c568f96998cd17a7313fd838 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 29 Jan 2021 15:44:09 +0530
Subject: [PATCH v21 5/7] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 7 ++-
doc/src/sgml/ref/create_access_method.sgml | 13 ++--
doc/src/sgml/ref/create_table.sgml | 3 +-
src/backend/access/common/detoast.c | 59 +++++++++++++++++--
src/backend/access/common/toast_internals.c | 19 +++++-
src/backend/access/compression/compress_lz4.c | 21 +++----
.../access/compression/compress_pglz.c | 20 +++----
.../access/compression/compressamapi.c | 27 +++++++++
src/backend/access/index/amapi.c | 50 ++++++++++++----
src/backend/commands/amcmds.c | 5 ++
src/backend/executor/nodeModifyTable.c | 2 +-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/varlena.c | 7 +--
src/bin/pg_dump/pg_dump.c | 3 +
src/bin/psql/tab-complete.c | 6 ++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 16 +++--
src/include/access/detoast.h | 8 +++
src/include/access/toast_internals.h | 15 +++++
src/test/regress/expected/compression.out | 40 ++++++++++++-
src/test/regress/expected/compression_1.out | 41 ++++++++++++-
src/test/regress/sql/compression.sql | 10 ++++
22 files changed, 312 insertions(+), 62 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 081fe078a4..46f34bffe5 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -390,9 +390,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This clause adds compression to a column. The Compression method can be
- set to any available compression method. The built-in methods are
- <literal>pglz</literal> and <literal>lz4</literal>.
+ This clause adds compression to a column. The Compression method could be
+ created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The built-in methods
+ are <literal>pglz</literal> and <literal>lz4</literal>.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..79f1290a58 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,7 +61,7 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
+ <literal>TABLE</literal>, <literal>INDEX</literal> and <literal>COMPRESSION</literal>
are supported at present.
</para>
</listitem>
@@ -77,12 +77,15 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
- API is described in <xref linkend="indexam"/>.
+ is described in <xref linkend="tableam"/>, the index access method
+ API is described in <xref linkend="indexam"/> an the compression access
+ method API is described in <xref linkend="compression-am"/>.
</para>
</listitem>
</varlistentry>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index d514c7f8c1..10b468fb2d 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -998,7 +998,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<listitem>
<para>
This clause adds the compression method to a column. The Compression
- method can be set from available compression methods. The built-in
+ method could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set from the available compression methods. The built-in
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If no compression method is specified, then compressible types will have
the default compression method <literal>pglz</literal>.
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index c90464053c..e032c30d25 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -457,13 +457,44 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the Oid of the compresion method stored in the compressed data.
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
+ */
+Oid
+toast_get_compression_oid(struct varlena *attr)
+{
+ CompressionId cmid;
+
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the built-in
+ * compression id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
+}
+
/* ----------
* toast_get_compression_handler - get the compression handler routines
*
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -477,10 +508,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "Invalid compression method id %d", cmid);
}
@@ -496,9 +538,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -512,16 +556,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index b04c5a5eb8..a3539065d3 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index e8b035d138..34b0dc1002 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -30,7 +30,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -45,10 +45,10 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
- tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
{
@@ -56,7 +56,7 @@ lz4_cmcompress(const struct varlena *value)
elog(ERROR, "lz4: could not compress data");
}
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -68,7 +68,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -81,9 +81,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
elog(ERROR, "lz4: compressed data is corrupted");
@@ -100,7 +100,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -112,9 +113,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARSIZE(value) - header_size,
slicelength,
TOAST_COMPRESS_RAWSIZE(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 578be6f1dd..d33ea3d2cf 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -44,16 +44,16 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ header_size);
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
+ (char *) tmp + header_size,
NULL);
if (len >= 0)
{
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -68,7 +68,7 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
@@ -76,8 +76,8 @@ pglz_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- TOAST_COMPRESS_SIZE(value),
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
TOAST_COMPRESS_RAWSIZE(value), true);
@@ -95,7 +95,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -103,8 +103,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
- VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index e8bfb23ec3..19780f57e7 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -19,6 +19,7 @@
#include "access/htup_details.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "commands/defrem.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
@@ -63,6 +64,32 @@ CompressionIdToOid(CompressionId cmid)
}
}
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
/* check_hook: validate new default_toast_compression */
bool
check_default_toast_compression(char **newval, void **extra, GucSource source)
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43514..89a4b01d78 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -46,14 +46,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +64,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +93,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 668e43faed..31d4a3e958 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -241,6 +241,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "TABLE";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -278,6 +280,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 916f6541c1..a63737213d 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2089,7 +2089,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
* compression method is not supported by the target attribute then
* we need to decompress it.
*/
- cmoid = CompressionIdToOid(TOAST_COMPRESS_METHOD(new_value));
+ cmoid = toast_get_compression_oid(new_value);
if (!IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 332e60f170..0a0ea0562a 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5304,6 +5304,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index aff5b7e95e..ebd797158a 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -5309,7 +5309,6 @@ Datum
pg_column_compression(PG_FUNCTION_ARGS)
{
Datum value = PG_GETARG_DATUM(0);
- char *compression;
int typlen;
struct varlena *varvalue;
@@ -5339,10 +5338,8 @@ pg_column_compression(PG_FUNCTION_ARGS)
varvalue = (struct varlena *) DatumGetPointer(value);
- compression =
- get_am_name(CompressionIdToOid(TOAST_COMPRESS_METHOD(varvalue)));
-
- PG_RETURN_TEXT_P(cstring_to_text(compression));
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(
+ toast_get_compression_oid(varvalue))));
}
/*
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index c09f3972c6..3a79754550 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13124,6 +13124,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index c40d498a30..5de2a6a18b 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb559..0f7b5b0794 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -285,6 +285,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index be5b4da7b8..36c9f57628 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -30,7 +30,9 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
#define DEFAULT_TOAST_COMPRESSION "pglz"
@@ -38,13 +40,18 @@ typedef enum CompressionId
extern char *default_toast_compression;
/* Use default compression method if it is not specified. */
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -68,6 +75,7 @@ extern const CompressionAmRoutine lz4_compress_methods;
/* access/compression/compressamapi.c */
extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
+extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
extern void assign_default_toast_compression(const char *newval, void **extra);
extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..6cdc37542d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 84ba303e7e..c158354272 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,6 +27,17 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
#define RAWSIZEMASK (0x3FFFFFFFU)
/*
@@ -38,6 +49,7 @@ typedef struct toast_compress_header
* two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & RAWSIZEMASK)
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> 30)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
@@ -49,6 +61,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << 30); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 151478bcdd..636ab4021b 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -235,13 +235,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 81182fe949..3177ff91e8 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -232,13 +232,52 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index e447316573..0552eeca16 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -95,6 +95,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v21-0006-new-compression-method-extension-for-zlib.patchtext/x-diff; charset=us-asciiDownload
From cca457e70e163c27b02c3d1374a57030771442c6 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v21 6/7] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index cdc041c7db..6c61872d94 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..2b6fac7e0b
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ pglz
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..ea8d206625
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.17.0
v21-0007-Support-compression-methods-options.patchtext/x-diff; charset=us-asciiDownload
From 172956bacd687fdeb6628d74b00efa5d17c642ef Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 29 Jan 2021 15:46:33 +0530
Subject: [PATCH v21 7/7] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 +++++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 8 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 +++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 69 +++++++++++-
.../access/compression/compress_pglz.c | 98 +++++++++++++++--
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 ++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++++-
src/backend/commands/foreigncmds.c | 44 --------
src/backend/commands/tablecmds.c | 102 +++++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 23 +++-
src/bin/pg_dump/pg_dump.h | 2 +
src/include/access/compressamapi.h | 20 +++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 +++++++++
src/test/regress/expected/compression_1.out | 55 ++++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 ++++
36 files changed, 717 insertions(+), 133 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..d8c2865f21 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 46f34bffe5..e496f02cdc 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -393,7 +393,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
This clause adds compression to a column. The Compression method could be
created with <xref linkend="sql-create-access-method"/> or
it can be set to any available compression method. The built-in methods
- are <literal>pglz</literal> and <literal>lz4</literal>.
+ are <literal>pglz</literal> and <literal>lz4</literal>. If the
+ compression method has options they can be specified with the
+ <literal>WITH</literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 10b468fb2d..e5514bbd1b 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This clause adds the compression method to a column. The Compression
@@ -1002,7 +1002,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
it can be set from the available compression methods. The built-in
methods are <literal>pglz</literal> and <literal>lz4</literal>.
If no compression method is specified, then compressible types will have
- the default compression method <literal>pglz</literal>.
+ the default compression method <literal>pglz</literal>. If the compression
+ method has options they can be specified with the <literal>WITH</literal>
+ parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712c71..76f824bc6f 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d2ff..0d3307e94f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3ee9e..2dda6c038b 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a3539065d3..5ed3312f39 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 34b0dc1002..375c4e852d 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -16,6 +16,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -23,6 +24,63 @@
#include "lz4.h"
#endif
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ {
+ int32 acceleration =
+ pg_atoi(defGetString(def), sizeof(acceleration), 0);
+
+ if (acceleration < INT32_MIN || acceleration > INT32_MAX)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for lz4 compression acceleration: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between INT32_MIN and INT32_MAX")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -30,7 +88,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -40,6 +98,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -47,9 +106,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(VARSIZE_ANY_EXHDR(value));
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
{
pfree(tmp);
@@ -129,6 +188,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index d33ea3d2cf..0396729c70 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -15,11 +15,92 @@
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -27,20 +108,22 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
@@ -49,7 +132,7 @@ pglz_cmcompress(const struct varlena *value, int32 header_size)
len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
+ strategy);
if (len >= 0)
{
@@ -118,10 +201,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9c3e..c9d44c62fa 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004e1b..a43e0e197c 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b53b6b50e6..18b04816f7 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -741,6 +741,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -796,6 +797,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -843,6 +849,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -861,7 +868,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -893,7 +901,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1160,6 +1168,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1418,7 +1427,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index e6040923dc..2a39e0ce64 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -107,10 +107,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -272,7 +274,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -333,6 +336,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -349,6 +355,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -489,7 +512,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -507,7 +530,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -720,6 +744,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -875,6 +900,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -883,7 +910,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -974,7 +1002,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481557..348b7d3a37 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 096a06f7b3..0ad946d802 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -699,6 +699,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 50f0f7faaa..d70408c7c2 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -29,6 +29,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -181,7 +182,7 @@ BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -207,6 +208,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
cmoid = get_compression_am_oid(compression->cmname, false);
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -279,15 +294,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttribute's compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103fd3b..ae9ae2c51b 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9d732fc167..93c1374bd4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -609,6 +609,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -813,6 +814,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -866,8 +869,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -921,8 +925,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2432,16 +2439,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2478,8 +2483,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2729,14 +2733,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6259,6 +6258,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6422,6 +6422,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6432,7 +6433,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7829,13 +7830,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* index attribute otherwise it will update the attstorage.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7874,7 +7882,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
else
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7965,7 +7995,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15137,9 +15167,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15174,7 +15206,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15184,8 +15217,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15193,8 +15235,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6588c84c4c..69e370de64 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2959,6 +2959,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 3aa6176a30..1e3a01fbc3 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2623,6 +2623,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 4f7053d63b..283452e454 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2890,6 +2890,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 0a0ea0562a..2d7b307661 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3502,11 +3503,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3514,14 +3521,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index a2a6b218f3..47fddc2ad1 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if (table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3a79754550..7ee41ca57d 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8709,10 +8709,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8765,6 +8772,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8794,6 +8802,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15932,7 +15941,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -15983,6 +15993,10 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
@@ -16414,6 +16428,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, "\nWITH (%s) ", tbinfo->attcmoptions[j]);
+
if (cminfo->nitems > 0)
{
appendPQExpBuffer(q, "\nPRESERVE (");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 2a751321f5..47ed1112e8 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -328,6 +328,8 @@ typedef struct _tableInfo
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
+ char **attcmoptions; /* per-attribute current compression
+ options */
struct _attrCompressionInfo **attcompression; /* per-attribute all compression data */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 36c9f57628..92aba67407 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -19,8 +19,8 @@
#include "catalog/pg_am_d.h"
#include "miscadmin.h"
#include "nodes/nodes.h"
-#include "utils/guc.h"
#include "nodes/pg_list.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -44,8 +44,11 @@ extern char *default_toast_compression;
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -56,14 +59,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc37f3..fe8de405ac 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index c158354272..3b143b8c0b 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -64,7 +64,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b49c..5ecb6f3653 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index a6cb1b4e5b..71cbc99116 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -176,6 +176,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bd53f9bb0f..8271e97dea 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -134,6 +134,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -146,9 +148,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index fd9c6abddf..a1b1158316 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 636ab4021b..9d2c00fb73 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -272,6 +272,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3177ff91e8..4c0952bc0a 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -270,6 +270,61 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ERROR: not built with lz4 support
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+HINT: use "pg_column_compression" function for list of compression methods
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index d40afeef78..27aab82462 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -97,6 +97,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -107,5 +108,5 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0552eeca16..59a32dd0a3 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0001-Add-default_toast_compression-GUC.beforerebasetext/x-diff; charset=us-asciiDownload
From 4b5a3855e1900a79d1b65d3606c7e9f165605d51 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH] Add default_toast_compression GUC
---
src/backend/access/common/tupdesc.c | 3 +-
.../access/compression/compressamapi.c | 54 +++++++++++++++++++
src/backend/bootstrap/bootstrap.c | 7 ++-
src/backend/commands/compressioncmds.c | 2 +-
src/backend/commands/tablecmds.c | 2 +-
src/backend/utils/cache/syscache.c | 6 ++-
src/backend/utils/misc/guc.c | 12 +++++
src/include/access/compressamapi.h | 12 ++++-
8 files changed, 91 insertions(+), 7 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab487..b56b689493 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -25,6 +25,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
+#include "commands/defrem.h"
#include "miscadmin.h"
#include "parser/parse_type.h"
#include "utils/acl.h"
@@ -668,7 +669,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = get_compression_am_oid(default_toast_compression, false);
else
att->attcompression = InvalidOid;
diff --git a/src/backend/access/compression/compressamapi.c b/src/backend/access/compression/compressamapi.c
index 7aea8aad38..19780f57e7 100644
--- a/src/backend/access/compression/compressamapi.c
+++ b/src/backend/access/compression/compressamapi.c
@@ -20,9 +20,12 @@
#include "access/reloptions.h"
#include "access/table.h"
#include "catalog/pg_am.h"
+#include "commands/defrem.h"
#include "utils/fmgroids.h"
#include "utils/syscache.h"
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION; // maybe needs pg_dump support ?
+
/*
* CompressionOidToId - Convert compression Oid to built-in compression id.
*
@@ -86,3 +89,54 @@ GetCompressionAmRoutineByAmId(Oid amoid)
return routine;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451eaa71..1b885f873a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -29,6 +29,7 @@
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/link-canary.h"
+#include "commands/defrem.h"
#include "libpq/pqsignal.h"
#include "miscadmin.h"
#include "nodes/makefuncs.h"
@@ -732,8 +733,12 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ {
+ /* Cannot call get_compression_am_oid this early */
+ attrtypes[attnum]->attcompression = PGLZ_COMPRESSION_AM_OID;
+ }
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index de57b55aab..d70408c7c2 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -204,7 +204,7 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return get_compression_am_oid(default_toast_compression, false);
cmoid = get_compression_am_oid(compression->cmname, false);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cc3a0cb753..93c1374bd4 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -12035,7 +12035,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = get_compression_am_oid(default_toast_compression, false);
}
else
attTup->attcompression = InvalidOid;
diff --git a/src/backend/utils/cache/syscache.c b/src/backend/utils/cache/syscache.c
index e4dc4ee34e..74e42a3394 100644
--- a/src/backend/utils/cache/syscache.c
+++ b/src/backend/utils/cache/syscache.c
@@ -1127,8 +1127,10 @@ HeapTuple
SearchSysCache1(int cacheId,
Datum key1)
{
- Assert(cacheId >= 0 && cacheId < SysCacheSize &&
- PointerIsValid(SysCache[cacheId]));
+ Assert(cacheId >= 0);
+ Assert(cacheId < SysCacheSize);
+ // fprintf(stderr, "cache %d\n", cacheId);
+ Assert(PointerIsValid(SysCache[cacheId]));
Assert(SysCache[cacheId]->cc_nkeys == 1);
return SearchCatCache1(SysCache[cacheId], key1);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index eafdb1118e..70b26ea72a 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 8123cc8cc7..92aba67407 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -15,9 +15,12 @@
#include "postgres.h"
+#include "access/xact.h"
#include "catalog/pg_am_d.h"
+#include "miscadmin.h"
#include "nodes/nodes.h"
#include "nodes/pg_list.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -32,8 +35,11 @@ typedef enum CompressionId
CUSTOM_COMPRESSION_ID = 3
} CompressionId;
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+extern char *default_toast_compression;
+
/* Use default compression method if it is not specified. */
-#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
@@ -85,4 +91,8 @@ extern CompressionId CompressionOidToId(Oid cmoid);
extern Oid CompressionIdToOid(CompressionId cmid);
extern CompressionAmRoutine *GetCompressionAmRoutineByAmId(Oid amoid);
+extern void assign_default_toast_compression(const char *newval, void **extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+
#endif /* COMPRESSAMAPI_H */
--
2.17.0
Re: [HACKERS] Custom compression methods
Some more review comments:
'git am' barfs on v0001 because it's got a whitespace error.
VARFLAGS_4B_C() doesn't seem to be used in any of the patches. I'm OK
with keeping it even if it's not used just because maybe someone will
need it later but, uh, don't we need to use it someplace?
To avoid moving the goalposts for a basic install, I suggest that
--with-lz4 should default to disabled. Maybe we'll want to rethink
that at some point, but since we're just getting started with this
whole thing, I don't think now is the time.
The change to ddl.sgml doesn't seem to make sense to me. There might
be someplace where we want to explain how properties are inherited in
partitioning hierarchies, but I don't think this is the right place,
and I don't think this explanation is particularly clear.
+ This clause adds the compression method to a column. The Compression
+ method can be set from available compression methods. The built-in
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ If no compression method is specified, then compressible types will have
+ the default compression method <literal>pglz</literal>.Suggest: This sets the compression method for a column. The supported
compression methods are <literal>pglz</literal> and
<literal>lz4</literal>. <literal>lz4</literal> is available only if
<literal>--with-lz4</literal> was used when building
<productname>PostgreSQL</productname>. The default is
<literal>pglz</literal>.
We should make sure, if you haven't already, that trying to create a
column with LZ4 compression fails at table creation time if the build
does not support LZ4. But, someone could also create a table using a
build that has LZ4 support and then switch to a different set of
binaries that do not have it, so we need the runtime checks also.
However, those runtime checks shouldn't fail simplify from trying to
access a table that is set to use LZ4 compression; they should only
fail if we actually need to decompress an LZ4'd value.
Since indexes don't have TOAST tables, it surprises me that
brin_form_tuple() thinks it can TOAST anything. But I guess that's not
this patch's problem, if it's a problem at all.
I like the fact that you changed the message "compressed data is
corrupt" to indicate the compression method, but I think the resulting
message doesn't follow style guidelines because I don't believe we
normally put something with a colon prefix at the beginning of a
primary error message. So instead of saying "pglz: compressed data is
corrupt" I think you should say something like "compressed pglz data
is corrupt". Also, I suggest that we take this opportunity to switch
to ereport() rather than elog() and set
errcode(ERRCODE_DATA_CORRUPTED).
What testing have you done for performance impacts? Does the patch
slow things down noticeably with pglz? (Hopefully not.) Can you
measure a performance improvement with pglz? (Hopefully so.) Is it
likely to hurt performance that there's no minimum size for lz4
compression as we have for pglz? Seems like that could result in a lot
of wasted cycles trying to compress short strings.
pglz_cmcompress() cancels compression if the resulting value would be
larger than the original one, but it looks like lz4_cmcompress() will
just store the enlarged value. That seems bad.
pglz_cmcompress() doesn't need to pfree(tmp) before elog(ERROR).
CompressionOidToId(), CompressionIdToOid() and maybe other places need
to remember the message style guidelines. Primary error messages are
not capitalized.
Why should we now have to include toast_internals.h in
reorderbuffer.c, which has no other changes? That definitely shouldn't
be necessary. If something in another header file now requires
something from toast_internals.h, then that header file would be
obliged to include toast_internals.h itself. But actually that
shouldn't happen, because the whole point of toast_internals.h is that
it should not be included in very many places at all. If we're adding
stuff there that is going to be broadly needed, we're adding it in the
wrong place.
varlena.c shouldn't need toast_internals.h either, and if it did, it
should be in alphabetical order.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Even more review comments, still looking mostly at 0001:
If there's a reason why parallel_schedule is arranging to run the
compression test in parallel with nothing else, the comment in that
file should explain the reason. If there isn't, it should be added to
a parallel group that doesn't have the maximum number of tests yet,
probably the last such group in the file.
serial_schedule should add the test in a position that roughly
corresponds to where it appears in parallel_schedule.
I believe it's relatively standard practice to put variable
declarations at the top of the file. compress_lz4.c and
compress_pglz.c instead put those declarations nearer to the point of
use.
compressamapi.c has an awful lot of #include directives for the code
it actually contains. I believe that we should cut that down to what
is required by 0001, and other patches can add more later as required.
In fact, it's tempting to just get rid of this .c file altogether and
make the two functions it contains static inline functions in the
header, but I'm not 100% sure that's a good idea.
The copyright dates in a number of the file headers are out of date.
binary_upgrade_next_pg_am_oid and the related changes to
CreateAccessMethod don't belong in 0001, because it doesn't support
non-built-in compression methods. These changes and the related
pg_dump change should be moved to the patch that adds support for
that.
The comments added to dumpTableSchema() say that "compression is
assigned by ALTER" but don't give a reason. I think they should. I
don't know how much they need to explain about what the code does, but
they definitely need to explain why it does it. Also, isn't this bad?
If we create the column with the wrong compression setting initially
and then ALTER it, we have to rewrite the table. If it's empty, that's
cheap, but it'd still be better not to do it at all.
I'm not sure it's a good idea for dumpTableSchema() to leave out
specifying the compression method if it happens to be pglz. I think we
definitely shouldn't do it in binary-upgrade mode. What if we changed
the default in a future release? For that matter, even 0002 could make
the current approach unsafe.... I think, anyway.
The changes to pg_dump.h look like they haven't had a visit from
pgindent. You should probably try to do that for the whole patch,
though it's a bit annoying since you'll have to manually remove
unrelated changes to the same files that are being modified by the
patch. Also, why the extra blank line here?
GetAttributeCompression() is hard to understand. I suggest changing
the comment to "resolve column compression specification to an OID"
and somehow rejigger the code so that you aren't using one not-NULL
test and one NULL test on the same variable. Like maybe change the
first part to if (!IsStorageCompressible(typstorage)) { if
(compression == NULL) return InvalidOid; ereport(ERROR, ...); }
It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time. See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values. It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it. I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize(). The whole approach you've taken
here might need to be rethought a bit. I think you are right to want
to avoid copying everything over into a new slot if nothing needs to
be done, and I think we should definitely keep that optimization, but
I think if you need to copy stuff, you have to do the above procedure
and then continue using the other slot instead of the original one.
Some places I think we have functions that return either the original
slot or a different one depending on how it goes; that might be a
useful idea here. But, you also can't just spam-create slots; it's
important that whatever ones we end up with get reused for every
tuple.
Doesn't the change to describeOneTableDetails() require declaring
changing the declaration of char *headers[11] to char *headers[12]?
How does this not fail Assert(cols <= lengthof(headers))?
Why does describeOneTableDetais() arrange to truncate the printed
value? We don't seem to do that for the other column properties, and
it's not like this one is particularly long.
Perhaps the changes to pg_am.dat shouldn't remove the blank line?
I think the comment to pg_attribute.h could be rephrased to stay
something like: "OID of compression AM. Must be InvalidOid if and only
if typstorage is 'a' or 'b'," replacing 'a' and 'b' with whatever the
right letters are. This would be shorter and I think also clearer than
what you have
The first comment change in postgres.h is wrong. You changed
va_extsize to "size in va_extinfo" but the associated structure
definition is unchanged, so the comment shouldn't be changed either.
In toast_internals.h, you end using 30 as a constant several times but
have no #define for it. You do have a #define for RAWSIZEMASK, but
that's really a derived value from 30. Also, it's not a great name
because it's kind of generic. So how about something like:
#define TOAST_RAWSIZE_BITS 30
#define TOAST_RAWSIZE_MASK ((1 << (TOAST_RAW_SIZE_BITS + 1)) - 1)
But then again on second thought, this 30 seems to be the same 30 that
shows up in the changes to postgres.h, and there again 0x3FFFFFFF
shows up too. So maybe we should actually be defining these constants
there, using names like VARLENA_RAWSIZE_BITS and VARLENA_RAWSIZE_MASK
and then having toast_internals.h use those constants as well.
Taken with the email I sent yesterday, I think this is a more or less
complete review of 0001. Although there are a bunch of things to fix
here still, I don't think this is that far from being committable. I
don't at this point see too much in terms of big design problems.
Probably the CompareCompressionMethodAndDecompress() is the closest to
a design-level problem, and certainly something needs to be done about
it, but even that is a fairly localized problem in the context of the
entire patch.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Feb 3, 2021 at 2:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Even more review comments, still looking mostly at 0001:
If there's a reason why parallel_schedule is arranging to run the
compression test in parallel with nothing else, the comment in that
file should explain the reason. If there isn't, it should be added to
a parallel group that doesn't have the maximum number of tests yet,
probably the last such group in the file.serial_schedule should add the test in a position that roughly
corresponds to where it appears in parallel_schedule.I believe it's relatively standard practice to put variable
declarations at the top of the file. compress_lz4.c and
compress_pglz.c instead put those declarations nearer to the point of
use.compressamapi.c has an awful lot of #include directives for the code
it actually contains. I believe that we should cut that down to what
is required by 0001, and other patches can add more later as required.
In fact, it's tempting to just get rid of this .c file altogether and
make the two functions it contains static inline functions in the
header, but I'm not 100% sure that's a good idea.The copyright dates in a number of the file headers are out of date.
binary_upgrade_next_pg_am_oid and the related changes to
CreateAccessMethod don't belong in 0001, because it doesn't support
non-built-in compression methods. These changes and the related
pg_dump change should be moved to the patch that adds support for
that.The comments added to dumpTableSchema() say that "compression is
assigned by ALTER" but don't give a reason. I think they should. I
don't know how much they need to explain about what the code does, but
they definitely need to explain why it does it. Also, isn't this bad?
If we create the column with the wrong compression setting initially
and then ALTER it, we have to rewrite the table. If it's empty, that's
cheap, but it'd still be better not to do it at all.I'm not sure it's a good idea for dumpTableSchema() to leave out
specifying the compression method if it happens to be pglz. I think we
definitely shouldn't do it in binary-upgrade mode. What if we changed
the default in a future release? For that matter, even 0002 could make
the current approach unsafe.... I think, anyway.The changes to pg_dump.h look like they haven't had a visit from
pgindent. You should probably try to do that for the whole patch,
though it's a bit annoying since you'll have to manually remove
unrelated changes to the same files that are being modified by the
patch. Also, why the extra blank line here?GetAttributeCompression() is hard to understand. I suggest changing
the comment to "resolve column compression specification to an OID"
and somehow rejigger the code so that you aren't using one not-NULL
test and one NULL test on the same variable. Like maybe change the
first part to if (!IsStorageCompressible(typstorage)) { if
(compression == NULL) return InvalidOid; ereport(ERROR, ...); }It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time. See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values. It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it. I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize(). The whole approach you've taken
here might need to be rethought a bit. I think you are right to want
to avoid copying everything over into a new slot if nothing needs to
be done, and I think we should definitely keep that optimization, but
I think if you need to copy stuff, you have to do the above procedure
and then continue using the other slot instead of the original one.
Some places I think we have functions that return either the original
slot or a different one depending on how it goes; that might be a
useful idea here. But, you also can't just spam-create slots; it's
important that whatever ones we end up with get reused for every
tuple.Doesn't the change to describeOneTableDetails() require declaring
changing the declaration of char *headers[11] to char *headers[12]?
How does this not fail Assert(cols <= lengthof(headers))?Why does describeOneTableDetais() arrange to truncate the printed
value? We don't seem to do that for the other column properties, and
it's not like this one is particularly long.Perhaps the changes to pg_am.dat shouldn't remove the blank line?
I think the comment to pg_attribute.h could be rephrased to stay
something like: "OID of compression AM. Must be InvalidOid if and only
if typstorage is 'a' or 'b'," replacing 'a' and 'b' with whatever the
right letters are. This would be shorter and I think also clearer than
what you haveThe first comment change in postgres.h is wrong. You changed
va_extsize to "size in va_extinfo" but the associated structure
definition is unchanged, so the comment shouldn't be changed either.In toast_internals.h, you end using 30 as a constant several times but
have no #define for it. You do have a #define for RAWSIZEMASK, but
that's really a derived value from 30. Also, it's not a great name
because it's kind of generic. So how about something like:#define TOAST_RAWSIZE_BITS 30
#define TOAST_RAWSIZE_MASK ((1 << (TOAST_RAW_SIZE_BITS + 1)) - 1)But then again on second thought, this 30 seems to be the same 30 that
shows up in the changes to postgres.h, and there again 0x3FFFFFFF
shows up too. So maybe we should actually be defining these constants
there, using names like VARLENA_RAWSIZE_BITS and VARLENA_RAWSIZE_MASK
and then having toast_internals.h use those constants as well.Taken with the email I sent yesterday, I think this is a more or less
complete review of 0001. Although there are a bunch of things to fix
here still, I don't think this is that far from being committable. I
don't at this point see too much in terms of big design problems.
Probably the CompareCompressionMethodAndDecompress() is the closest to
a design-level problem, and certainly something needs to be done about
it, but even that is a fairly localized problem in the context of the
entire patch.
Thanks, Robert for the detailed review. I will work on these comments
and post the updated patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Feb 3, 2021 at 2:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
While going through your comments, I need some suggestions about the
one except this all other comments looks fine to me.
It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time.
Yeah, actually, I thought I would avoid calling slot_getallattrs if
none of the attributes got decompress. I agree if we call this before
we can avoid calling slot_getattr but slot_getattr
is only called for the attribute which has attlen -1. I agree that if
we call slot_getattr for attnum n then it will deform all the
attributes before that. But then slot_getallattrs only need to deform
the remaining attributes not all. But maybe we can call the
slot_getallattrs as soon as we see the first attribute with attlen -1
and then avoid calling subsequent slot_getattr, maybe that is better
than compared to what I have because we will avoid calling
slot_getattr for many attributes, especially when there are many
verlena.
See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values.
It will not overwrite those values because slot_getallattrs will only
fetch the values for "attnum > slot->tts_nvalid" so whatever we
already fetched will not be overwritten. Just did that at the end to
optimize the normal cases where we are not doing "insert into select *
from" so that those can get away without calling slot_getallattrs at
all. However, maybe calling slot_getattr for each varlena might cost
us extra so I am okay to call slot_getallattrs this early.
It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it.
Yeah, that's completely wrong. I think I missed that part. One
solution can be that we can just detach the tuple from the slot and
then materialize so it will form the tuple with new values and then we
can clear the old tuple. But that seems a bit hacky.
I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize().
Okay, so if we take a new slot then we need to set this slot reference
in the ScanState also otherwise that might point to the old slot. I
haven't yet analyzed where all we might be keeping the reference to
that old slot. Or I am missing something.
Anyway, I will get a better idea once I try to implement this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Feb 4, 2021 at 11:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Yeah, actually, I thought I would avoid calling slot_getallattrs if
none of the attributes got decompress. I agree if we call this before
we can avoid calling slot_getattr but slot_getattr
is only called for the attribute which has attlen -1. I agree that if
we call slot_getattr for attnum n then it will deform all the
attributes before that. But then slot_getallattrs only need to deform
the remaining attributes not all. But maybe we can call the
slot_getallattrs as soon as we see the first attribute with attlen -1
and then avoid calling subsequent slot_getattr, maybe that is better
than compared to what I have because we will avoid calling
slot_getattr for many attributes, especially when there are many
verlena.
I think that if we need to deform at all, we need to deform all
attributes, right? So there's no point in considering e.g.
slot_getsomeattrs(). But just slot_getallattrs() as soon as we know we
need to do it might be worthwhile. Could even have two loops: one that
just figures out whether we need to deform; if not, return. Then
slot_getallattrs(). Then another loop to do the work.
I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize().Okay, so if we take a new slot then we need to set this slot reference
in the ScanState also otherwise that might point to the old slot. I
haven't yet analyzed where all we might be keeping the reference to
that old slot. Or I am missing something.
My guess is you want to leave the ScanState alone so that we keep
fetching into the same slot as before and have an extra slot on the
side someplace.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Feb 5, 2021 at 3:51 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Feb 4, 2021 at 11:39 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Yeah, actually, I thought I would avoid calling slot_getallattrs if
none of the attributes got decompress. I agree if we call this before
we can avoid calling slot_getattr but slot_getattr
is only called for the attribute which has attlen -1. I agree that if
we call slot_getattr for attnum n then it will deform all the
attributes before that. But then slot_getallattrs only need to deform
the remaining attributes not all. But maybe we can call the
slot_getallattrs as soon as we see the first attribute with attlen -1
and then avoid calling subsequent slot_getattr, maybe that is better
than compared to what I have because we will avoid calling
slot_getattr for many attributes, especially when there are many
verlena.I think that if we need to deform at all, we need to deform all
attributes, right?
IMHO that is not true, because we might need to deform the attribute
just to check its stored compression. So for example the first
attribute is varchar and the remaining 100 attributes are interger.
So we just need to deform the first attribute and if the compression
method of that is the same as the target attribute then we are done
and not need to deform the remaining and we can just continue with the
original slot and tuple.
I am not saying this is a very practical example and we have to do it
like this, but I am just making a point that it is not true that if we
deform at all then we have to deform all. However, if we decompress
any then we have to deform all because we need to materialize the
tuple again.
So there's no point in considering e.g.
slot_getsomeattrs(). But just slot_getallattrs() as soon as we know we
need to do it might be worthwhile. Could even have two loops: one that
just figures out whether we need to deform; if not, return. Then
slot_getallattrs(). Then another loop to do the work.I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize().Okay, so if we take a new slot then we need to set this slot reference
in the ScanState also otherwise that might point to the old slot. I
haven't yet analyzed where all we might be keeping the reference to
that old slot. Or I am missing something.My guess is you want to leave the ScanState alone so that we keep
fetching into the same slot as before and have an extra slot on the
side someplace.
Okay, got your point. Thanks.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Feb 3, 2021 at 2:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Even more review comments, still looking mostly at 0001:
If there's a reason why parallel_schedule is arranging to run the
compression test in parallel with nothing else, the comment in that
file should explain the reason. If there isn't, it should be added to
a parallel group that doesn't have the maximum number of tests yet,
probably the last such group in the file.serial_schedule should add the test in a position that roughly
corresponds to where it appears in parallel_schedule.
Done
I believe it's relatively standard practice to put variable
declarations at the top of the file. compress_lz4.c and
compress_pglz.c instead put those declarations nearer to the point of
use.
Do you mean pglz_compress_methods and lz4_compress_methods ? I
followed that style from
heapam_handler.c. If you think that doesn't look good then I can move it up.
compressamapi.c has an awful lot of #include directives for the code
it actually contains. I believe that we should cut that down to what
is required by 0001, and other patches can add more later as required.
In fact, it's tempting to just get rid of this .c file altogether and
make the two functions it contains static inline functions in the
header, but I'm not 100% sure that's a good idea.
I think it looks better to move them to compressamapi.h so done that.
The copyright dates in a number of the file headers are out of date.
Fixed
binary_upgrade_next_pg_am_oid and the related changes to
CreateAccessMethod don't belong in 0001, because it doesn't support
non-built-in compression methods. These changes and the related
pg_dump change should be moved to the patch that adds support for
that.
Fixed
The comments added to dumpTableSchema() say that "compression is
assigned by ALTER" but don't give a reason. I think they should. I
don't know how much they need to explain about what the code does, but
they definitely need to explain why it does it. Also, isn't this bad?
If we create the column with the wrong compression setting initially
and then ALTER it, we have to rewrite the table. If it's empty, that's
cheap, but it'd still be better not to do it at all.
Yeah, actually that part should go in 0003 patch where we implement
the custom compression method.
in that patch we need to alter and set because we want to keep the
preserved method as well
So I will add it there
I'm not sure it's a good idea for dumpTableSchema() to leave out
specifying the compression method if it happens to be pglz. I think we
definitely shouldn't do it in binary-upgrade mode. What if we changed
the default in a future release? For that matter, even 0002 could make
the current approach unsafe.... I think, anyway.
Fixed
The changes to pg_dump.h look like they haven't had a visit from
pgindent. You should probably try to do that for the whole patch,
though it's a bit annoying since you'll have to manually remove
unrelated changes to the same files that are being modified by the
patch. Also, why the extra blank line here?
Fixed, ran pgindent for other files as well.
GetAttributeCompression() is hard to understand. I suggest changing
the comment to "resolve column compression specification to an OID"
and somehow rejigger the code so that you aren't using one not-NULL
test and one NULL test on the same variable. Like maybe change the
first part to if (!IsStorageCompressible(typstorage)) { if
(compression == NULL) return InvalidOid; ereport(ERROR, ...); }
Done
It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time. See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values. It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it. I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize(). The whole approach you've taken
here might need to be rethought a bit. I think you are right to want
to avoid copying everything over into a new slot if nothing needs to
be done, and I think we should definitely keep that optimization, but
I think if you need to copy stuff, you have to do the above procedure
and then continue using the other slot instead of the original one.
Some places I think we have functions that return either the original
slot or a different one depending on how it goes; that might be a
useful idea here. But, you also can't just spam-create slots; it's
important that whatever ones we end up with get reused for every
tuple.
I have changed this algorithm, so now if we have to decompress
anything we will use the new slot and we will stick that new slot to
the ModifyTableState, DR_transientrel for matviews and DR_intorel for
CTAS. Does this looks okay or we need to do something else? If this
logic looks fine then maybe we can think of some more optimization and
cleanup in this function.
Doesn't the change to describeOneTableDetails() require declaring
changing the declaration of char *headers[11] to char *headers[12]?
How does this not fail Assert(cols <= lengthof(headers))?
Fixed
Why does describeOneTableDetais() arrange to truncate the printed
value? We don't seem to do that for the other column properties, and
it's not like this one is particularly long.
Not required, fixed.
Perhaps the changes to pg_am.dat shouldn't remove the blank line?
Fixed
I think the comment to pg_attribute.h could be rephrased to stay
something like: "OID of compression AM. Must be InvalidOid if and only
if typstorage is 'a' or 'b'," replacing 'a' and 'b' with whatever the
right letters are. This would be shorter and I think also clearer than
what you have
Fixed
The first comment change in postgres.h is wrong. You changed
va_extsize to "size in va_extinfo" but the associated structure
definition is unchanged, so the comment shouldn't be changed either.
Yup, not required.
In toast_internals.h, you end using 30 as a constant several times but
have no #define for it. You do have a #define for RAWSIZEMASK, but
that's really a derived value from 30. Also, it's not a great name
because it's kind of generic. So how about something like:#define TOAST_RAWSIZE_BITS 30
#define TOAST_RAWSIZE_MASK ((1 << (TOAST_RAW_SIZE_BITS + 1)) - 1)But then again on second thought, this 30 seems to be the same 30 that
shows up in the changes to postgres.h, and there again 0x3FFFFFFF
shows up too. So maybe we should actually be defining these constants
there, using names like VARLENA_RAWSIZE_BITS and VARLENA_RAWSIZE_MASK
and then having toast_internals.h use those constants as well.
Done, IMHO it should be
#define VARLENA_RAWSIZE_BITS 30
#define VARLENA_RAWSIZE_MASK ((1 << VARLENA_RAWSIZE_BITS) -1 )
Taken with the email I sent yesterday, I think this is a more or less
complete review of 0001. Although there are a bunch of things to fix
here still, I don't think this is that far from being committable. I
don't at this point see too much in terms of big design problems.
Probably the CompareCompressionMethodAndDecompress() is the closest to
a design-level problem, and certainly something needs to be done about
it, but even that is a fairly localized problem in the context of the
entire patch.
0001 is attached, now pending parts are
- Confirm the new design of CompareCompressionMethodAndDecompress
- Performance test, especially lz4 with small varlena
- Rebase other patches atop this patch
- comment in ddl.sgml
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v21-0001-Built-in-compression-method.patchapplication/x-patch; name=v21-0001-Built-in-compression-method.patchDownload
From 608452edde04d6c483d2bf3f65c6cc2a60d4c608 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v21] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++
configure.ac | 17 ++
contrib/test_decoding/expected/ddl.out | 50 ++--
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 +++--
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 160 +++++++++++
.../access/compression/compress_pglz.c | 136 +++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 14 +
src/backend/commands/matview.c | 14 +
src/backend/commands/tablecmds.c | 111 +++++++-
src/backend/executor/nodeModifyTable.c | 124 +++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 ++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/psql/describe.c | 28 +-
src/include/access/compressamapi.h | 99 +++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +-
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 11 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 176 ++++++++++++
src/test/regress/expected/compression_1.out | 167 ++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 +++++-----
.../regress/expected/create_table_like.out | 88 +++---
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 +++++++++---------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 +++++-----
src/test/regress/expected/insert.out | 118 ++++----
src/test/regress/expected/matview.out | 80 +++---
src/test/regress/expected/psql.out | 108 ++++----
src/test/regress/expected/publication.out | 40 +--
.../regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +-
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 73 +++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
78 files changed, 2199 insertions(+), 666 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36999..0895b2f326 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8563,6 +8566,39 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13320,6 +13406,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 07da84d401..63940b78a0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a4625cc..4d7ed698b9 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from parent table's column, however a different compression method can be
+ set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..51a7a977a5 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..5704fd0e78 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +512,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..b04c5a5eb8 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..779f54e785
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..59cf48f884
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,160 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Get maximum size of the compressed data that lz4 compression may output
+ * and allocate the memory for holding the compressed data.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + TOAST_COMPRESS_HDRSZ);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + TOAST_COMPRESS_HDRSZ,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) +
+ VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ /* decompress data using lz4 routine */
+ rawsize = LZ4_decompress_safe(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ /* decompress partial data using lz4 routine */
+ rawsize = LZ4_decompress_safe_partial(TOAST_COMPRESS_RAWDATA(value),
+ VARDATA(result),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ slicelength,
+ TOAST_COMPRESS_RAWSIZE(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..0786673487
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,136 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ TOAST_COMPRESS_HDRSZ);
+
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
+ valsize,
+ TOAST_COMPRESS_RAWDATA(tmp),
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+ SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ TOAST_COMPRESS_SIZE(value),
+ VARDATA(result),
+ TOAST_COMPRESS_RAWSIZE(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(value),
+ VARSIZE(value) - TOAST_COMPRESS_HDRSZ,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..f5bb37b972 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1f55..b53b6b50e6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1cb9172a5f..18a9ee31cf 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..3ad4a61739 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..1d17dc0d6b 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -581,6 +582,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..713fc3fceb 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -486,6 +487,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..dd81d5bf4e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5d90337498..c9c8778228 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,115 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /* nothing to be done, if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(VARFLAGS_4B_C(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then we need to copy the
+ * values/null array from oldslot to the new slot and materialize the new
+ * slot.
+ */
+ if (decompressed_any)
+ {
+ TupleTableSlot *newslot = *outslot;
+
+ /*
+ * If the called has passed an invalid slot then create a new slot.
+ * Otherwise, just clear the existing tuple from the slot. This slot
+ * should be stored by the caller so that it can be reused for
+ * decompressing the subsequent tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+ ExecStoreVirtualTuple(newslot);
+ ExecMaterializeSlot(newslot);
+ ExecClearTuple(slot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2355,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +2996,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18ecb..1338e04409 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf6e8..0605ef3f84 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9fc3c..52d92df25d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..cf4413da64 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..975e3004f1 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5299,6 +5301,50 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(VARFLAGS_4B_C(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a569..5e168a3c04 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d99b61e621..cccf3c0d7e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1290f9659b..66a2374544 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..ba464d463e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..5a8e23d926
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..6a2a420f86 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,31 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
+ *
+ * Since version 14 TOAST_COMPRESS_SET_RAWSIZE also marks compressed
+ * varlenas as custom compressed. Such varlenas will contain 0x02 (0b10) in
+ * two highest bits.
*/
#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
+#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->info & VARLENA_RAWSIZE_MASK)
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
#define TOAST_COMPRESS_RAWDATA(ptr) \
(((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..605684408c 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86faef8..65079f3821 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..797e78b17c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4e0c9be58c..4a13844ce6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f53b..306aabf6c1 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540c94..e5aea8a240 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 758c3ca097..44a2ab1e7a 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -616,5 +616,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d65099c94a..1601e4ba27 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1178,6 +1178,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, incase the target attribute's
+ * compression method doesn't match with the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489c23..20d6f96f62 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a2ca..19d2ba26bf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d2bf..53d378b67d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..baae295444 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -280,8 +281,14 @@ typedef struct
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4622..138df4be93 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..4dea269fc3
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,176 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..a8a48636e9
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,167 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f0719e7..1778c15c7a 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b8de..3105115fee 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be23c..4da878ee1c 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c003e..6268b26562 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e25820bc..89466da603 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca0333a2..dc2af075ff 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef654..7e709916f6 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3b67..3cd4ad9e53 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0cb7..e078818496 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..d6912f2109 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7a4e..71388c197d 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 79002197a7..20ac0012ef 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aaef82..7ebdc30ded 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6173473de9..8546f02c1c 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3035,11 +3035,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3086,11 +3086,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 431b3fa3de..a36b7e6069 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -125,11 +125,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d79f6..58e996fdee 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1bbe7e0323..57fbb262d6 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -330,10 +330,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -350,10 +350,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e491..e9c0fc9760 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416fd80..4d5577bae3 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..f4a04d64c2
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,73 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b2c9..c64b369047 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1d540fe489..afc248a895 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.23.0
Re: [HACKERS] Custom compression methods
On Tue, Feb 2, 2021 at 2:45 AM Robert Haas <robertmhaas@gmail.com> wrote:
Some more review comments:
'git am' barfs on v0001 because it's got a whitespace error.
Fixed
VARFLAGS_4B_C() doesn't seem to be used in any of the patches. I'm OK
with keeping it even if it's not used just because maybe someone will
need it later but, uh, don't we need to use it someplace?
Actually I was using TOAST_COMPRESS_METHOD and that required inclusion
of toast_internal.h so now
I have used VARFLAGS_4B_C and with that we are able to remove the
inclusion of toast_internal.h
in unwanted places.
To avoid moving the goalposts for a basic install, I suggest that
--with-lz4 should default to disabled. Maybe we'll want to rethink
that at some point, but since we're just getting started with this
whole thing, I don't think now is the time.
Done
The change to ddl.sgml doesn't seem to make sense to me. There might
be someplace where we want to explain how properties are inherited in
partitioning hierarchies, but I don't think this is the right place,
and I don't think this explanation is particularly clear.
Not yet done, I thought at the same place we are describing the
storage relationship with the partition so that is the place for the
compression also. Maybe I will have to read ddl.sgml file and find
out the most suitable place. So I kept is as pending.
+ This clause adds the compression method to a column. The Compression + method can be set from available compression methods. The built-in + methods are <literal>pglz</literal> and <literal>lz4</literal>. + If no compression method is specified, then compressible types will have + the default compression method <literal>pglz</literal>.Suggest: This sets the compression method for a column. The supported
compression methods are <literal>pglz</literal> and
<literal>lz4</literal>. <literal>lz4</literal> is available only if
<literal>--with-lz4</literal> was used when building
<productname>PostgreSQL</productname>. The default is
<literal>pglz</literal>.
Done
We should make sure, if you haven't already, that trying to create a
column with LZ4 compression fails at table creation time if the build
does not support LZ4. But, someone could also create a table using a
build that has LZ4 support and then switch to a different set of
binaries that do not have it, so we need the runtime checks also.
However, those runtime checks shouldn't fail simplify from trying to
access a table that is set to use LZ4 compression; they should only
fail if we actually need to decompress an LZ4'd value.
Done, I have cheched the compression method Oid if it LZ4 and if the
lz4 library is not install
then error out. We can also use the handler sepcific check function
but I am not sure does that make
sense to add extra routine for that. In later patch 0006 we have an
check function to verify the
option so during that we can error out and no need to check this outside.
Since indexes don't have TOAST tables, it surprises me that
brin_form_tuple() thinks it can TOAST anything. But I guess that's not
this patch's problem, if it's a problem at all.
it is just trying to compress it not externalize.
I like the fact that you changed the message "compressed data is
corrupt" to indicate the compression method, but I think the resulting
message doesn't follow style guidelines because I don't believe we
normally put something with a colon prefix at the beginning of a
primary error message. So instead of saying "pglz: compressed data is
corrupt" I think you should say something like "compressed pglz data
is corrupt". Also, I suggest that we take this opportunity to switch
to ereport() rather than elog() and set
errcode(ERRCODE_DATA_CORRUPTED).
Done
What testing have you done for performance impacts? Does the patch
slow things down noticeably with pglz? (Hopefully not.) Can you
measure a performance improvement with pglz? (Hopefully so.) Is it
likely to hurt performance that there's no minimum size for lz4
compression as we have for pglz? Seems like that could result in a lot
of wasted cycles trying to compress short strings.
Not sure what to do about this, I will check the performance with
small varlenas and see.
pglz_cmcompress() cancels compression if the resulting value would be
larger than the original one, but it looks like lz4_cmcompress() will
just store the enlarged value. That seems bad.
you mean lz4_cmcompress, Done
pglz_cmcompress() doesn't need to pfree(tmp) before elog(ERROR).
Done
CompressionOidToId(), CompressionIdToOid() and maybe other places need
to remember the message style guidelines. Primary error messages are
not capitalized.
Fixed
Why should we now have to include toast_internals.h in
reorderbuffer.c, which has no other changes? That definitely shouldn't
be necessary. If something in another header file now requires
something from toast_internals.h, then that header file would be
obliged to include toast_internals.h itself. But actually that
shouldn't happen, because the whole point of toast_internals.h is that
it should not be included in very many places at all. If we're adding
stuff there that is going to be broadly needed, we're adding it in the
wrong place.
Done
varlena.c shouldn't need toast_internals.h either, and if it did, it
should be in alphabetical order.
It was the wrong usage, fixed now.
Please refer to the latest patch at
/messages/by-id/CAFiTN-v9Cs1MORnp-3bGZ5QBwr5v3VarSvfaDizHi1acXES5xQ@mail.gmail.com
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
+ * If the called has passed an invalid slot then create a new slot.
*caller
+ * Slot for storing the modified tuple, incase the target attribute's
*in case
Could you comment on the patch I sent on Jan 30 ? I think it would be squished
into 0001.
Subject: [PATCH v21 2/7] Add default_toast_compression GUC
Also, what about the idea to add HIDE_COMPRESSAM ? Right now, your patch
changes a great many regression tests, and I doubt many people are going to try
to look closely to verify the differences, now, or when setting a non-default
compression method.
Also, I think we may want to make enable-lz4 the default *for testing
purposes*, now that the linux and BSD environments include that.
--
Justin
Re: [HACKERS] Custom compression methods
On Fri, Feb 5, 2021 at 10:56 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Could you comment on the patch I sent on Jan 30 ? I think it would be squished
into 0001.
I don't see why we have to do that. Seems fine to have it as a separate patch.
Also, what about the idea to add HIDE_COMPRESSAM ? Right now, your patch
changes a great many regression tests, and I doubt many people are going to try
to look closely to verify the differences, now, or when setting a non-default
compression method.
Personally, my preference is to just update the test outputs. It's not
important whether many people look closely to verify the differences;
we just need to look them over on a one-time basis to see if they seem
OK. After that it's 0 effort, vs. having to maintain HIDE_COMPRESSAM
forever.
Also, I think we may want to make enable-lz4 the default *for testing
purposes*, now that the linux and BSD environments include that.
My guess was that would annoy some hackers whose build environments
got broken. If everyone thinks otherwise I'm willing to be persuaded,
but it's going to take more than 1 vote...
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Feb 5, 2021 at 11:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Personally, my preference is to just update the test outputs. It's not
important whether many people look closely to verify the differences;
we just need to look them over on a one-time basis to see if they seem
OK. After that it's 0 effort, vs. having to maintain HIDE_COMPRESSAM
forever.
Oh, I guess you're thinking about the case where someone wants to run
the tests with a different default. That might be a good reason to
have this. But then those changes should go in 0002.
Regarding 0002, I'm not feeling very excited about having every call
to TupleDescInitEntry() do an extra syscache lookup. It's going to be
the same lookup every time forever to get the same value every time
forever. Now maybe that function can never get hot enough for it to
matter, but can't we find a way to be smarter about this? Like,
suppose we cache the OID in a global variable the first time we look
it up, and then use CacheRegisterSyscacheCallback() to have it zeroed
out if pg_am is updated?
Taking that idea a bit further, suppose you get rid of all the places
where you do get_compression_am_oid(default_toast_compression, false)
and change them to get_default_compression_am_oid(), which is defined
thus:
static Oid
get_default_compression_am_oid(void)
{
if (unlikely(!OidIsValid(cached_default_compression_oid))
// figure it out;
return cached_default_compression_oid;
}
Also, how about removing the debugging leftovers from syscache.c?
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
This fails make-check world for me, and CFBOT will say the same.
Be sure to compile with --enable-tap-tests
cd . && TESTDIR='/home/pryzbyj/src/postgres/src/bin/pg_dump' PATH="/home/pryzbyj/src/postgres/tmp_install/usr/local/pgsql/bin:$PATH" LD_LIBRARY_PATH="/home/pryzbyj/src/postgres/tmp_install/usr/local/pgsql/lib" PGPORT='65432' PG_REGRESS='/home/pryzbyj/src/postgres/src/bin/pg_dump/../../../src/test/regress/pg_regress' REGRESS_SHLIB='/home/pryzbyj/src/postgres/src/test/regress/regress.so' /usr/bin/prove -I ../../../src/test/perl/ -I . t/*.pl
t/001_basic.pl ................ ok
t/002_pg_dump.pl .............. 13/6408
# Failed test 'binary_upgrade: should dump CREATE TABLE test_fifth_table'
# at t/002_pg_dump.pl line 3601.
# Review binary_upgrade results in /home/pryzbyj/src/postgres/src/bin/pg_dump/tmp_check/tmp_test_gw4p
# Failed test 'binary_upgrade: should dump CREATE TABLE test_second_table'
# at t/002_pg_dump.pl line 3601.
# Review binary_upgrade results in /home/pryzbyj/src/postgres/src/bin/pg_dump/tmp_check/tmp_test_gw4p
# Failed test 'binary_upgrade: should dump CREATE TABLE test_table'
# at t/002_pg_dump.pl line 3601.
# Review binary_upgrade results in /home/pryzbyj/src/postgres/src/bin/pg_dump/tmp_check/tmp_test_gw4p
# Failed test 'binary_upgrade: should dump CREATE TABLE test_table_identity'
# at t/002_pg_dump.pl line 3601.
# Review binary_upgrade results in /home/pryzbyj/src/postgres/src/bin/pg_dump/tmp_check/tmp_test_gw4p
t/002_pg_dump.pl .............. 6203/6408 # Looks like you failed 4 tests of 6408.
t/002_pg_dump.pl .............. Dubious, test returned 4 (wstat 1024, 0x400)
Failed 4/6408 subtests
Re: [HACKERS] Custom compression methods
On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Feb 3, 2021 at 2:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Even more review comments, still looking mostly at 0001:
If there's a reason why parallel_schedule is arranging to run the
compression test in parallel with nothing else, the comment in that
file should explain the reason. If there isn't, it should be added to
a parallel group that doesn't have the maximum number of tests yet,
probably the last such group in the file.serial_schedule should add the test in a position that roughly
corresponds to where it appears in parallel_schedule.Done
I believe it's relatively standard practice to put variable
declarations at the top of the file. compress_lz4.c and
compress_pglz.c instead put those declarations nearer to the point of
use.Do you mean pglz_compress_methods and lz4_compress_methods ? I
followed that style from
heapam_handler.c. If you think that doesn't look good then I can move it up.compressamapi.c has an awful lot of #include directives for the code
it actually contains. I believe that we should cut that down to what
is required by 0001, and other patches can add more later as required.
In fact, it's tempting to just get rid of this .c file altogether and
make the two functions it contains static inline functions in the
header, but I'm not 100% sure that's a good idea.I think it looks better to move them to compressamapi.h so done that.
The copyright dates in a number of the file headers are out of date.
Fixed
binary_upgrade_next_pg_am_oid and the related changes to
CreateAccessMethod don't belong in 0001, because it doesn't support
non-built-in compression methods. These changes and the related
pg_dump change should be moved to the patch that adds support for
that.Fixed
The comments added to dumpTableSchema() say that "compression is
assigned by ALTER" but don't give a reason. I think they should. I
don't know how much they need to explain about what the code does, but
they definitely need to explain why it does it. Also, isn't this bad?
If we create the column with the wrong compression setting initially
and then ALTER it, we have to rewrite the table. If it's empty, that's
cheap, but it'd still be better not to do it at all.Yeah, actually that part should go in 0003 patch where we implement
the custom compression method.
in that patch we need to alter and set because we want to keep the
preserved method as well
So I will add it thereI'm not sure it's a good idea for dumpTableSchema() to leave out
specifying the compression method if it happens to be pglz. I think we
definitely shouldn't do it in binary-upgrade mode. What if we changed
the default in a future release? For that matter, even 0002 could make
the current approach unsafe.... I think, anyway.Fixed
The changes to pg_dump.h look like they haven't had a visit from
pgindent. You should probably try to do that for the whole patch,
though it's a bit annoying since you'll have to manually remove
unrelated changes to the same files that are being modified by the
patch. Also, why the extra blank line here?Fixed, ran pgindent for other files as well.
GetAttributeCompression() is hard to understand. I suggest changing
the comment to "resolve column compression specification to an OID"
and somehow rejigger the code so that you aren't using one not-NULL
test and one NULL test on the same variable. Like maybe change the
first part to if (!IsStorageCompressible(typstorage)) { if
(compression == NULL) return InvalidOid; ereport(ERROR, ...); }Done
It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time. See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values. It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it. I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize(). The whole approach you've taken
here might need to be rethought a bit. I think you are right to want
to avoid copying everything over into a new slot if nothing needs to
be done, and I think we should definitely keep that optimization, but
I think if you need to copy stuff, you have to do the above procedure
and then continue using the other slot instead of the original one.
Some places I think we have functions that return either the original
slot or a different one depending on how it goes; that might be a
useful idea here. But, you also can't just spam-create slots; it's
important that whatever ones we end up with get reused for every
tuple.I have changed this algorithm, so now if we have to decompress
anything we will use the new slot and we will stick that new slot to
the ModifyTableState, DR_transientrel for matviews and DR_intorel for
CTAS. Does this looks okay or we need to do something else? If this
logic looks fine then maybe we can think of some more optimization and
cleanup in this function.Doesn't the change to describeOneTableDetails() require declaring
changing the declaration of char *headers[11] to char *headers[12]?
How does this not fail Assert(cols <= lengthof(headers))?Fixed
Why does describeOneTableDetais() arrange to truncate the printed
value? We don't seem to do that for the other column properties, and
it's not like this one is particularly long.Not required, fixed.
Perhaps the changes to pg_am.dat shouldn't remove the blank line?
Fixed
I think the comment to pg_attribute.h could be rephrased to stay
something like: "OID of compression AM. Must be InvalidOid if and only
if typstorage is 'a' or 'b'," replacing 'a' and 'b' with whatever the
right letters are. This would be shorter and I think also clearer than
what you haveFixed
The first comment change in postgres.h is wrong. You changed
va_extsize to "size in va_extinfo" but the associated structure
definition is unchanged, so the comment shouldn't be changed either.Yup, not required.
In toast_internals.h, you end using 30 as a constant several times but
have no #define for it. You do have a #define for RAWSIZEMASK, but
that's really a derived value from 30. Also, it's not a great name
because it's kind of generic. So how about something like:#define TOAST_RAWSIZE_BITS 30
#define TOAST_RAWSIZE_MASK ((1 << (TOAST_RAW_SIZE_BITS + 1)) - 1)But then again on second thought, this 30 seems to be the same 30 that
shows up in the changes to postgres.h, and there again 0x3FFFFFFF
shows up too. So maybe we should actually be defining these constants
there, using names like VARLENA_RAWSIZE_BITS and VARLENA_RAWSIZE_MASK
and then having toast_internals.h use those constants as well.Done, IMHO it should be
#define VARLENA_RAWSIZE_BITS 30
#define VARLENA_RAWSIZE_MASK ((1 << VARLENA_RAWSIZE_BITS) -1 )Taken with the email I sent yesterday, I think this is a more or less
complete review of 0001. Although there are a bunch of things to fix
here still, I don't think this is that far from being committable. I
don't at this point see too much in terms of big design problems.
Probably the CompareCompressionMethodAndDecompress() is the closest to
a design-level problem, and certainly something needs to be done about
it, but even that is a fairly localized problem in the context of the
entire patch.0001 is attached, now pending parts are
- Confirm the new design of CompareCompressionMethodAndDecompress
- Performance test, especially lz4 with small varlena
I have tested the performance, pglz vs lz4
Test1: With a small simple string, pglz doesn't select compression but
lz4 select as no min limit
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 212 ms
Test2: With small incompressible string, pglz don't select compression
lz4 select but can not compress
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 1090 ms (attempt to compress but externalize):
Test3: Test a few columns with large random data
Table: 3 varchar column
Test: Insert 1000 tuple 3 columns size(3500 byes, 4200 bytes, 4900bytes)
pglz: 150 ms (compression ratio: 3.02%),
lz4: 30 ms (compression ratio : 2.3%)
Test4: Test3 with different large random slighly compressible, need to
compress + externalize:
Table: 3 varchar column
Insert: Insert 1000 tuple 3 columns size(8192, 8192, 8192)
CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
Test: insert into t1 select large_val(), large_val(), large_val() from
generate_series(1,1000);
pglz: 2000 ms
lz4: 1500 ms
Conclusion:
1. In most cases lz4 is faster and doing better compression as well.
2. In Test2 when small data is incompressible then lz4 tries to
compress whereas pglz doesn't try so there is some performance loss.
But if we want we can fix
it by setting some minimum limit of size for lz4 as well, maybe the
same size as pglz?
- Rebase other patches atop this patch
- comment in ddl.sgml
Other changes in patch:
- Now we are dumping the default compression method in the
binary-upgrade mode so the pg_dump test needed some change, fixed
that.
- in compress_pglz.c and compress_lz4.c, we were using
toast_internal.h macros so I removed and used varlena macros instead.
While testing, I noticed that if the compressed data are externalized
then pg_column_compression(), doesn't fetch the compression method
from the toast chunk, I think we should do that. I will analyze this
and fix it in the next version.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v22-0001-Built-in-compression-method.patchapplication/octet-stream; name=v22-0001-Built-in-compression-method.patchDownload
From 70f7ab66e1b8190df20e46328a5b56d48d550bd7 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v22] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 +++++++++++
configure.ac | 17 ++
contrib/test_decoding/expected/ddl.out | 50 ++---
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 +++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 72 ++++---
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++---
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 164 +++++++++++++++
src/backend/access/compression/compress_pglz.c | 138 +++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++-
src/backend/executor/nodeModifyTable.c | 124 ++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 46 +++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 28 ++-
src/include/access/compressamapi.h | 99 +++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 19 +-
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 176 ++++++++++++++++
src/test/regress/expected/compression_1.out | 167 +++++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 ++++++-------
src/test/regress/expected/create_table_like.out | 88 ++++----
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 ++++++++++++------------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 ++++++-------
src/test/regress/expected/insert.out | 118 +++++------
src/test/regress/expected/matview.out | 80 ++++----
src/test/regress/expected/psql.out | 108 +++++-----
src/test/regress/expected/publication.out | 40 ++--
src/test/regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +--
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 73 +++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
79 files changed, 2207 insertions(+), 676 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044..6ee0776 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a462..4d7ed69 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from parent table's column, however a different compression method can be
+ set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9..51a7a97 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -606,6 +607,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..5704fd0 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,47 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_handler - get the compression handler routines
*
- * Decompress a compressed version of a varlena datum
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +512,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1d43d5d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..779f54e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..1856cf7
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Get maximum size of the compressed data that lz4 compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress data using lz4 routine */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress partial data using lz4 routine */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..8a4bf42
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,138 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Get maximum size of the compressed data that pglz compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..b53b6b5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1cb9172..18a9ee3 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e..dd81d5b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5d90337..c9c8778 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,115 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /* nothing to be done, if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare with the compression method of the target.
+ */
+ if (targetTupDesc->attrs[i].attcompression !=
+ CompressionIdToOid(VARFLAGS_4B_C(new_value)))
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then we need to copy the
+ * values/null array from oldslot to the new slot and materialize the new
+ * slot.
+ */
+ if (decompressed_any)
+ {
+ TupleTableSlot *newslot = *outslot;
+
+ /*
+ * If the called has passed an invalid slot then create a new slot.
+ * Otherwise, just clear the existing tuple from the slot. This slot
+ * should be stored by the caller so that it can be reused for
+ * decompressing the subsequent tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+ ExecStoreVirtualTuple(newslot);
+ ExecMaterializeSlot(newslot);
+ ExecClearTuple(slot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2355,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +2996,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3af..cf4413d 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..975e300 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,50 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ char *compression;
+ int typlen;
+ struct varlena *varvalue;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ /*
+ * If it is not a varlena type or the attribute is not compressed then
+ * return NULL.
+ */
+ if ((typlen != -1) || !VARATT_IS_COMPRESSED(value))
+ PG_RETURN_NULL();
+
+ varvalue = (struct varlena *) DatumGetPointer(value);
+
+ compression =
+ get_am_name(CompressionIdToOid(VARFLAGS_4B_C(varvalue)));
+
+ PG_RETURN_TEXT_P(cstring_to_text(compression));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a..5e168a3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d99b61e..cccf3c0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1290f96..66a2374 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..ba464d4 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..5a8e23d
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..31ff91a 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,22 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4e0c9be..4a13844 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 758c3ca..44a2ab1 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -616,5 +616,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d65099c..1601e4b 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1178,6 +1178,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, incase the target attribute's
+ * compression method doesn't match with the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..667927f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4..138df4b 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..4dea269
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,176 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..a8a4863
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,167 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f071..1778c15 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b..3105115 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be..4da878e 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c0..6268b26 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e2582..89466da 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca033..dc2af07 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef..7e70991 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3..3cd4ad9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0..e078818 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..d6912f2 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7..71388c1 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 7900219..20ac001 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aae..7ebdc30 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6173473..8546f02 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3035,11 +3035,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3086,11 +3086,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 431b3fa..a36b7e6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -125,11 +125,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d7..58e996f 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1bbe7e0..57fbb26 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -330,10 +330,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -350,10 +350,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..f4a04d6
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,73 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1d540fe..afc248a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Feb 3, 2021 at 2:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Even more review comments, still looking mostly at 0001:
If there's a reason why parallel_schedule is arranging to run the
compression test in parallel with nothing else, the comment in that
file should explain the reason. If there isn't, it should be added to
a parallel group that doesn't have the maximum number of tests yet,
probably the last such group in the file.serial_schedule should add the test in a position that roughly
corresponds to where it appears in parallel_schedule.Done
I believe it's relatively standard practice to put variable
declarations at the top of the file. compress_lz4.c and
compress_pglz.c instead put those declarations nearer to the point of
use.Do you mean pglz_compress_methods and lz4_compress_methods ? I
followed that style from
heapam_handler.c. If you think that doesn't look good then I can move it up.compressamapi.c has an awful lot of #include directives for the code
it actually contains. I believe that we should cut that down to what
is required by 0001, and other patches can add more later as required.
In fact, it's tempting to just get rid of this .c file altogether and
make the two functions it contains static inline functions in the
header, but I'm not 100% sure that's a good idea.I think it looks better to move them to compressamapi.h so done that.
The copyright dates in a number of the file headers are out of date.
Fixed
binary_upgrade_next_pg_am_oid and the related changes to
CreateAccessMethod don't belong in 0001, because it doesn't support
non-built-in compression methods. These changes and the related
pg_dump change should be moved to the patch that adds support for
that.Fixed
The comments added to dumpTableSchema() say that "compression is
assigned by ALTER" but don't give a reason. I think they should. I
don't know how much they need to explain about what the code does, but
they definitely need to explain why it does it. Also, isn't this bad?
If we create the column with the wrong compression setting initially
and then ALTER it, we have to rewrite the table. If it's empty, that's
cheap, but it'd still be better not to do it at all.Yeah, actually that part should go in 0003 patch where we implement
the custom compression method.
in that patch we need to alter and set because we want to keep the
preserved method as well
So I will add it thereI'm not sure it's a good idea for dumpTableSchema() to leave out
specifying the compression method if it happens to be pglz. I think we
definitely shouldn't do it in binary-upgrade mode. What if we changed
the default in a future release? For that matter, even 0002 could make
the current approach unsafe.... I think, anyway.Fixed
The changes to pg_dump.h look like they haven't had a visit from
pgindent. You should probably try to do that for the whole patch,
though it's a bit annoying since you'll have to manually remove
unrelated changes to the same files that are being modified by the
patch. Also, why the extra blank line here?Fixed, ran pgindent for other files as well.
GetAttributeCompression() is hard to understand. I suggest changing
the comment to "resolve column compression specification to an OID"
and somehow rejigger the code so that you aren't using one not-NULL
test and one NULL test on the same variable. Like maybe change the
first part to if (!IsStorageCompressible(typstorage)) { if
(compression == NULL) return InvalidOid; ereport(ERROR, ...); }Done
It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time. See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values. It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it. I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize(). The whole approach you've taken
here might need to be rethought a bit. I think you are right to want
to avoid copying everything over into a new slot if nothing needs to
be done, and I think we should definitely keep that optimization, but
I think if you need to copy stuff, you have to do the above procedure
and then continue using the other slot instead of the original one.
Some places I think we have functions that return either the original
slot or a different one depending on how it goes; that might be a
useful idea here. But, you also can't just spam-create slots; it's
important that whatever ones we end up with get reused for every
tuple.I have changed this algorithm, so now if we have to decompress
anything we will use the new slot and we will stick that new slot to
the ModifyTableState, DR_transientrel for matviews and DR_intorel for
CTAS. Does this looks okay or we need to do something else? If this
logic looks fine then maybe we can think of some more optimization and
cleanup in this function.Doesn't the change to describeOneTableDetails() require declaring
changing the declaration of char *headers[11] to char *headers[12]?
How does this not fail Assert(cols <= lengthof(headers))?Fixed
Why does describeOneTableDetais() arrange to truncate the printed
value? We don't seem to do that for the other column properties, and
it's not like this one is particularly long.Not required, fixed.
Perhaps the changes to pg_am.dat shouldn't remove the blank line?
Fixed
I think the comment to pg_attribute.h could be rephrased to stay
something like: "OID of compression AM. Must be InvalidOid if and only
if typstorage is 'a' or 'b'," replacing 'a' and 'b' with whatever the
right letters are. This would be shorter and I think also clearer than
what you haveFixed
The first comment change in postgres.h is wrong. You changed
va_extsize to "size in va_extinfo" but the associated structure
definition is unchanged, so the comment shouldn't be changed either.Yup, not required.
In toast_internals.h, you end using 30 as a constant several times but
have no #define for it. You do have a #define for RAWSIZEMASK, but
that's really a derived value from 30. Also, it's not a great name
because it's kind of generic. So how about something like:#define TOAST_RAWSIZE_BITS 30
#define TOAST_RAWSIZE_MASK ((1 << (TOAST_RAW_SIZE_BITS + 1)) - 1)But then again on second thought, this 30 seems to be the same 30 that
shows up in the changes to postgres.h, and there again 0x3FFFFFFF
shows up too. So maybe we should actually be defining these constants
there, using names like VARLENA_RAWSIZE_BITS and VARLENA_RAWSIZE_MASK
and then having toast_internals.h use those constants as well.Done, IMHO it should be
#define VARLENA_RAWSIZE_BITS 30
#define VARLENA_RAWSIZE_MASK ((1 << VARLENA_RAWSIZE_BITS) -1 )Taken with the email I sent yesterday, I think this is a more or less
complete review of 0001. Although there are a bunch of things to fix
here still, I don't think this is that far from being committable. I
don't at this point see too much in terms of big design problems.
Probably the CompareCompressionMethodAndDecompress() is the closest to
a design-level problem, and certainly something needs to be done about
it, but even that is a fairly localized problem in the context of the
entire patch.0001 is attached, now pending parts are
- Confirm the new design of CompareCompressionMethodAndDecompress
- Performance test, especially lz4 with small varlenaI have tested the performance, pglz vs lz4
Test1: With a small simple string, pglz doesn't select compression but
lz4 select as no min limit
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 212 msTest2: With small incompressible string, pglz don't select compression
lz4 select but can not compress
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 1090 ms (attempt to compress but externalize):Test3: Test a few columns with large random data
Table: 3 varchar column
Test: Insert 1000 tuple 3 columns size(3500 byes, 4200 bytes, 4900bytes)
pglz: 150 ms (compression ratio: 3.02%),
lz4: 30 ms (compression ratio : 2.3%)Test4: Test3 with different large random slighly compressible, need to
compress + externalize:
Table: 3 varchar column
Insert: Insert 1000 tuple 3 columns size(8192, 8192, 8192)
CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
Test: insert into t1 select large_val(), large_val(), large_val() from
generate_series(1,1000);
pglz: 2000 ms
lz4: 1500 msConclusion:
1. In most cases lz4 is faster and doing better compression as well.
2. In Test2 when small data is incompressible then lz4 tries to
compress whereas pglz doesn't try so there is some performance loss.
But if we want we can fix
it by setting some minimum limit of size for lz4 as well, maybe the
same size as pglz?- Rebase other patches atop this patch
- comment in ddl.sgmlOther changes in patch:
- Now we are dumping the default compression method in the
binary-upgrade mode so the pg_dump test needed some change, fixed
that.
- in compress_pglz.c and compress_lz4.c, we were using
toast_internal.h macros so I removed and used varlena macros instead.While testing, I noticed that if the compressed data are externalized
then pg_column_compression(), doesn't fetch the compression method
from the toast chunk, I think we should do that. I will analyze this
and fix it in the next version.
While trying to fix this, I have realized this problem exists in
CompareCompressionMethodAndDecompress
see below code.
---
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /* nothing to be done, if it is not compressed */
+ if (!VARATT_IS_COMPRESSED(new_value))
+ continue;
---Basically, we are just checking whether the stored value is compressed
or not, but we are clearly ignoring the fact that it might be
compressed and stored externally on disk. So basically if the value
is stored externally we can not know whether the external data were
compressed or not without fetching the values from the toast table, I
think instead of fetching the complete data from toast we can just
fetch the header using 'toast_fetch_datum_slice'.
Any other thoughts on this?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Feb 9, 2021 at 2:08 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Feb 7, 2021 at 5:15 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Fri, Feb 5, 2021 at 8:11 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Feb 3, 2021 at 2:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Even more review comments, still looking mostly at 0001:
If there's a reason why parallel_schedule is arranging to run the
compression test in parallel with nothing else, the comment in that
file should explain the reason. If there isn't, it should be added to
a parallel group that doesn't have the maximum number of tests yet,
probably the last such group in the file.serial_schedule should add the test in a position that roughly
corresponds to where it appears in parallel_schedule.Done
I believe it's relatively standard practice to put variable
declarations at the top of the file. compress_lz4.c and
compress_pglz.c instead put those declarations nearer to the point of
use.Do you mean pglz_compress_methods and lz4_compress_methods ? I
followed that style from
heapam_handler.c. If you think that doesn't look good then I can move it up.compressamapi.c has an awful lot of #include directives for the code
it actually contains. I believe that we should cut that down to what
is required by 0001, and other patches can add more later as required.
In fact, it's tempting to just get rid of this .c file altogether and
make the two functions it contains static inline functions in the
header, but I'm not 100% sure that's a good idea.I think it looks better to move them to compressamapi.h so done that.
The copyright dates in a number of the file headers are out of date.
Fixed
binary_upgrade_next_pg_am_oid and the related changes to
CreateAccessMethod don't belong in 0001, because it doesn't support
non-built-in compression methods. These changes and the related
pg_dump change should be moved to the patch that adds support for
that.Fixed
The comments added to dumpTableSchema() say that "compression is
assigned by ALTER" but don't give a reason. I think they should. I
don't know how much they need to explain about what the code does, but
they definitely need to explain why it does it. Also, isn't this bad?
If we create the column with the wrong compression setting initially
and then ALTER it, we have to rewrite the table. If it's empty, that's
cheap, but it'd still be better not to do it at all.Yeah, actually that part should go in 0003 patch where we implement
the custom compression method.
in that patch we need to alter and set because we want to keep the
preserved method as well
So I will add it thereI'm not sure it's a good idea for dumpTableSchema() to leave out
specifying the compression method if it happens to be pglz. I think we
definitely shouldn't do it in binary-upgrade mode. What if we changed
the default in a future release? For that matter, even 0002 could make
the current approach unsafe.... I think, anyway.Fixed
The changes to pg_dump.h look like they haven't had a visit from
pgindent. You should probably try to do that for the whole patch,
though it's a bit annoying since you'll have to manually remove
unrelated changes to the same files that are being modified by the
patch. Also, why the extra blank line here?Fixed, ran pgindent for other files as well.
GetAttributeCompression() is hard to understand. I suggest changing
the comment to "resolve column compression specification to an OID"
and somehow rejigger the code so that you aren't using one not-NULL
test and one NULL test on the same variable. Like maybe change the
first part to if (!IsStorageCompressible(typstorage)) { if
(compression == NULL) return InvalidOid; ereport(ERROR, ...); }Done
It puzzles me that CompareCompressionMethodAndDecompress() calls
slot_getallattrs() just before clearing the slot. It seems like this
ought to happen before we loop over the attributes, so that we don't
need to call slot_getattr() every time. See the comment for that
function. But even if we didn't do that for some reason, why would we
do it here? If it's already been done, it shouldn't do anything, and
if it hasn't been done, it might overwrite some of the values we just
poked into tts_values. It also seems suspicious that we can get away
with clearing the slot and then again marking it valid. I'm not sure
it really works like that. Like, can't clearing the slot invalidate
pointers stored in tts_values[]? For instance, if they are pointers
into an in-memory heap tuple, tts_heap_clear() is going to free the
tuple; if they are pointers into a buffer, tts_buffer_heap_clear() is
going to unpin it. I think the supported procedure for this sort of
thing is to have a second slot, set tts_values, tts_isnull etc. and
then materialize the slot. After materializing the new slot, it's
independent of the old slot, which can then be cleared. See for
example tts_virtual_materialize(). The whole approach you've taken
here might need to be rethought a bit. I think you are right to want
to avoid copying everything over into a new slot if nothing needs to
be done, and I think we should definitely keep that optimization, but
I think if you need to copy stuff, you have to do the above procedure
and then continue using the other slot instead of the original one.
Some places I think we have functions that return either the original
slot or a different one depending on how it goes; that might be a
useful idea here. But, you also can't just spam-create slots; it's
important that whatever ones we end up with get reused for every
tuple.I have changed this algorithm, so now if we have to decompress
anything we will use the new slot and we will stick that new slot to
the ModifyTableState, DR_transientrel for matviews and DR_intorel for
CTAS. Does this looks okay or we need to do something else? If this
logic looks fine then maybe we can think of some more optimization and
cleanup in this function.Doesn't the change to describeOneTableDetails() require declaring
changing the declaration of char *headers[11] to char *headers[12]?
How does this not fail Assert(cols <= lengthof(headers))?Fixed
Why does describeOneTableDetais() arrange to truncate the printed
value? We don't seem to do that for the other column properties, and
it's not like this one is particularly long.Not required, fixed.
Perhaps the changes to pg_am.dat shouldn't remove the blank line?
Fixed
I think the comment to pg_attribute.h could be rephrased to stay
something like: "OID of compression AM. Must be InvalidOid if and only
if typstorage is 'a' or 'b'," replacing 'a' and 'b' with whatever the
right letters are. This would be shorter and I think also clearer than
what you haveFixed
The first comment change in postgres.h is wrong. You changed
va_extsize to "size in va_extinfo" but the associated structure
definition is unchanged, so the comment shouldn't be changed either.Yup, not required.
In toast_internals.h, you end using 30 as a constant several times but
have no #define for it. You do have a #define for RAWSIZEMASK, but
that's really a derived value from 30. Also, it's not a great name
because it's kind of generic. So how about something like:#define TOAST_RAWSIZE_BITS 30
#define TOAST_RAWSIZE_MASK ((1 << (TOAST_RAW_SIZE_BITS + 1)) - 1)But then again on second thought, this 30 seems to be the same 30 that
shows up in the changes to postgres.h, and there again 0x3FFFFFFF
shows up too. So maybe we should actually be defining these constants
there, using names like VARLENA_RAWSIZE_BITS and VARLENA_RAWSIZE_MASK
and then having toast_internals.h use those constants as well.Done, IMHO it should be
#define VARLENA_RAWSIZE_BITS 30
#define VARLENA_RAWSIZE_MASK ((1 << VARLENA_RAWSIZE_BITS) -1 )Taken with the email I sent yesterday, I think this is a more or less
complete review of 0001. Although there are a bunch of things to fix
here still, I don't think this is that far from being committable. I
don't at this point see too much in terms of big design problems.
Probably the CompareCompressionMethodAndDecompress() is the closest to
a design-level problem, and certainly something needs to be done about
it, but even that is a fairly localized problem in the context of the
entire patch.0001 is attached, now pending parts are
- Confirm the new design of CompareCompressionMethodAndDecompress
- Performance test, especially lz4 with small varlenaI have tested the performance, pglz vs lz4
Test1: With a small simple string, pglz doesn't select compression but
lz4 select as no min limit
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 212 msTest2: With small incompressible string, pglz don't select compression
lz4 select but can not compress
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 1090 ms (attempt to compress but externalize):Test3: Test a few columns with large random data
Table: 3 varchar column
Test: Insert 1000 tuple 3 columns size(3500 byes, 4200 bytes, 4900bytes)
pglz: 150 ms (compression ratio: 3.02%),
lz4: 30 ms (compression ratio : 2.3%)Test4: Test3 with different large random slighly compressible, need to
compress + externalize:
Table: 3 varchar column
Insert: Insert 1000 tuple 3 columns size(8192, 8192, 8192)
CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
Test: insert into t1 select large_val(), large_val(), large_val() from
generate_series(1,1000);
pglz: 2000 ms
lz4: 1500 msConclusion:
1. In most cases lz4 is faster and doing better compression as well.
2. In Test2 when small data is incompressible then lz4 tries to
compress whereas pglz doesn't try so there is some performance loss.
But if we want we can fix
it by setting some minimum limit of size for lz4 as well, maybe the
same size as pglz?- Rebase other patches atop this patch
- comment in ddl.sgmlOther changes in patch:
- Now we are dumping the default compression method in the
binary-upgrade mode so the pg_dump test needed some change, fixed
that.
- in compress_pglz.c and compress_lz4.c, we were using
toast_internal.h macros so I removed and used varlena macros instead.While testing, I noticed that if the compressed data are externalized
then pg_column_compression(), doesn't fetch the compression method
from the toast chunk, I think we should do that. I will analyze this
and fix it in the next version.While trying to fix this, I have realized this problem exists in CompareCompressionMethodAndDecompress see below code. --- + new_value = (struct varlena *) + DatumGetPointer(slot->tts_values[attnum - 1]); + + /* nothing to be done, if it is not compressed */ + if (!VARATT_IS_COMPRESSED(new_value)) + continue; ---Basically, we are just checking whether the stored value is compressed
or not, but we are clearly ignoring the fact that it might be
compressed and stored externally on disk. So basically if the value
is stored externally we can not know whether the external data were
compressed or not without fetching the values from the toast table, I
think instead of fetching the complete data from toast we can just
fetch the header using 'toast_fetch_datum_slice'.Any other thoughts on this?
I think I was partially wrong here. Basically, there is a way to know
whether the external data are compressed or not using
VARATT_EXTERNAL_IS_COMPRESSED macro. However, if it is compressed
then we will have to fetch the toast slice of size
toast_compress_header, to know the compression method.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Please remember to trim unnecessary quoted material.
On Sun, Feb 7, 2021 at 6:45 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
[ a whole lot of quoted stuff ]
I have tested the performance, pglz vs lz4
Test1: With a small simple string, pglz doesn't select compression but
lz4 select as no min limit
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 212 msTest2: With small incompressible string, pglz don't select compression
lz4 select but can not compress
Table: 100 varchar column
Test: Insert 1000 tuple, each column of 25 bytes string (32 is min
limit for pglz)
Result:
pglz: 1030 ms (doesn't attempt compression so externalize),
lz4: 1090 ms (attempt to compress but externalize):Test3: Test a few columns with large random data
Table: 3 varchar column
Test: Insert 1000 tuple 3 columns size(3500 byes, 4200 bytes, 4900bytes)
pglz: 150 ms (compression ratio: 3.02%),
lz4: 30 ms (compression ratio : 2.3%)Test4: Test3 with different large random slighly compressible, need to
compress + externalize:
Table: 3 varchar column
Insert: Insert 1000 tuple 3 columns size(8192, 8192, 8192)
CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
Test: insert into t1 select large_val(), large_val(), large_val() from
generate_series(1,1000);
pglz: 2000 ms
lz4: 1500 msConclusion:
1. In most cases lz4 is faster and doing better compression as well.
2. In Test2 when small data is incompressible then lz4 tries to
compress whereas pglz doesn't try so there is some performance loss.
But if we want we can fix
it by setting some minimum limit of size for lz4 as well, maybe the
same size as pglz?
So my conclusion here is that perhaps there's no real problem. It
looks like externalizing is so expensive compared to compression that
it's worth trying to compress even though it may not always pay off.
If, by trying to compress, we avoid externalizing, it's a huge win
(~5x). If we try to compress and don't manage to avoid externalizing,
it's a small loss (~6%). It's probably reasonable to expect that
compressible data is more common than incompressible data, so not only
is the win a lot bigger than the loss, but we should be able to expect
it to happen a lot more often. It's not impossible that somebody could
get bitten, but it doesn't feel like a huge risk to me.
One thing that does occur to me is that it might be a good idea to
skip compression if it doesn't change the number of chunks that will
be stored into the TOAST table. If we compress the value but still
need to externalize it, and the compression didn't save enough to
reduce the number of chunks, I suppose we ideally would externalize
the uncompressed version. That would save decompression time later,
without really costing anything. However, I suppose that would be a
separate improvement from this patch. Maybe the possibility of
compressing smaller values makes it slightly more important, but I'm
not sure that it's worth getting excited about.
If anyone feels otherwise on either point, it'd be good to hear about it.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
I see the thread got broken somehow (or cfbot thought it did), so I added the
new thread, and this is now passing all tests. (I think using the v22
patches). http://cfbot.cputube.org/dilip-kumar.html
On Fri, Feb 05, 2021 at 11:07:53AM -0500, Robert Haas wrote:
Also, I think we may want to make enable-lz4 the default *for testing
purposes*, now that the linux and BSD environments include that.My guess was that would annoy some hackers whose build environments
got broken. If everyone thinks otherwise I'm willing to be persuaded,
but it's going to take more than 1 vote...
I think you misunderstood: I mean that the WIP patch should default to
--enable-lz4, to exercise on a few CI. It's hardly useful to run CI with the
feature disabled. I assume that the patch would be committed with default
--disable-lz4.
On Fri, Feb 05, 2021 at 11:23:46AM -0500, Robert Haas wrote:
On Fri, Feb 5, 2021 at 11:07 AM Robert Haas <robertmhaas@gmail.com> wrote:
Personally, my preference is to just update the test outputs. It's not
important whether many people look closely to verify the differences;
we just need to look them over on a one-time basis to see if they seem
OK. After that it's 0 effort, vs. having to maintain HIDE_COMPRESSAM
forever.Oh, I guess you're thinking about the case where someone wants to run
the tests with a different default. That might be a good reason to
have this. But then those changes should go in 0002.
Right, it's not one-time, it's also whenever setting a non-default compression
method. I say it should go into 0001 to avoid a whole bunch of churn in
src/test/regress, and then more churn (and rebase conflicts in other patches)
while adding HIDE_COMPRESSAM in 0002.
--
Justin
Re: [HACKERS] Custom compression methods
On Tue, Feb 9, 2021 at 3:37 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think you misunderstood: I mean that the WIP patch should default to
--enable-lz4, to exercise on a few CI. It's hardly useful to run CI with the
feature disabled. I assume that the patch would be committed with default
--disable-lz4.
Oh, I see. I guess we could do that.
Right, it's not one-time, it's also whenever setting a non-default compression
method. I say it should go into 0001 to avoid a whole bunch of churn in
src/test/regress, and then more churn (and rebase conflicts in other patches)
while adding HIDE_COMPRESSAM in 0002.
Hmm, I guess that makes some sense, too.
I'm not sure either one is completely critical, but it does make sense
to me now.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Feb 10, 2021 at 1:42 AM Robert Haas <robertmhaas@gmail.com> wrote:
Please remember to trim unnecessary quoted material.
Okay, I will.
On Sun, Feb 7, 2021 at 6:45 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
[ a whole lot of quoted stuff ]
Conclusion:
1. In most cases lz4 is faster and doing better compression as well.
2. In Test2 when small data is incompressible then lz4 tries to
compress whereas pglz doesn't try so there is some performance loss.
But if we want we can fix
it by setting some minimum limit of size for lz4 as well, maybe the
same size as pglz?So my conclusion here is that perhaps there's no real problem. It
looks like externalizing is so expensive compared to compression that
it's worth trying to compress even though it may not always pay off.
If, by trying to compress, we avoid externalizing, it's a huge win
(~5x). If we try to compress and don't manage to avoid externalizing,
it's a small loss (~6%). It's probably reasonable to expect that
compressible data is more common than incompressible data, so not only
is the win a lot bigger than the loss, but we should be able to expect
it to happen a lot more often. It's not impossible that somebody could
get bitten, but it doesn't feel like a huge risk to me.
I agree with this. That said maybe we could test the performance of
pglz also by lowering/removing the min compression limit but maybe
that should be an independent change.
One thing that does occur to me is that it might be a good idea to
skip compression if it doesn't change the number of chunks that will
be stored into the TOAST table. If we compress the value but still
need to externalize it, and the compression didn't save enough to
reduce the number of chunks, I suppose we ideally would externalize
the uncompressed version. That would save decompression time later,
without really costing anything. However, I suppose that would be a
separate improvement from this patch.
Yeah, this seems like a good idea and we can work on that in a different thread.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Feb 9, 2021 at 6:14 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Basically, we are just checking whether the stored value is compressed
or not, but we are clearly ignoring the fact that it might be
compressed and stored externally on disk. So basically if the value
is stored externally we can not know whether the external data were
compressed or not without fetching the values from the toast table, I
think instead of fetching the complete data from toast we can just
fetch the header using 'toast_fetch_datum_slice'.Any other thoughts on this?
I think I was partially wrong here. Basically, there is a way to know
whether the external data are compressed or not using
VARATT_EXTERNAL_IS_COMPRESSED macro. However, if it is compressed
then we will have to fetch the toast slice of size
toast_compress_header, to know the compression method.
I have fixed this issue, so now we will be able to detect the
compression method of the externalized compressed data as well. I
have also added a test case for this. I have rebased other patches
also on top of this patch. I have fixed the doc compilation issue in
patch 0004 raised by Justin. I still could not figure out what is the
right place for inheriting the compression method related change in
the "ddl.sgml" so that is still there, any suggestions on that?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v23-0003-Add-support-for-PRESERVE.patchtext/x-patch; charset=UTF-8; name=v23-0003-Add-support-for-PRESERVE.patchDownload
From 2c7910f57adc36928196a4523fa203385b8d1f8c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 11:49:23 +0530
Subject: [PATCH v23 3/6] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. For
supporting this the column will maintain the dependency with all
the supported compression methods. So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 300 ++++++++++++++++++++++++++++
src/backend/commands/tablecmds.c | 126 ++++++------
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 ++++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 101 ++++++++++
src/bin/pg_dump/pg_dump.h | 15 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 +++-
src/test/regress/expected/compression_1.out | 39 +++-
src/test/regress/expected/create_index.out | 56 +++---
src/test/regress/sql/compression.sql | 9 +
22 files changed, 739 insertions(+), 108 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 0bd0c1a..c9f443a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -387,7 +387,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
@@ -395,6 +395,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da243..dd37648 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0..a7395ad 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000..4ff75d0
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,300 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribute.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+#ifndef HAVE_LIBLZ4
+ if (cmoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode, just create a dependency on all preserved
+ * methods.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p)));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cb2ad13..fcfe4fd 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -936,6 +938,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
rel = relation_open(relationId, AccessExclusiveLock);
/*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
+ /*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
* parsetrees; we need to transform them to executable expression trees
@@ -2415,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression) != 0)
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4908,7 +4926,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6414,7 +6433,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6589,6 +6609,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6737,6 +6758,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
+/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11867,7 +11910,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11982,6 +12026,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15086,24 +15135,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15136,12 +15182,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
atttableform->attcompression = cmoid;
+
+ atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -17865,42 +17916,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * resolve column compression specification to an OID.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
- Oid amoid;
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression == NULL)
- return InvalidOid;
-
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return DefaultCompressionOid;
-
- amoid = get_compression_am_oid(compression, false);
-
-#ifndef HAVE_LIBLZ4
- if (amoid == LZ4_COMPRESSION_AM_OID)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("not built with lz4 support")));
-#endif
- return amoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 93f63da..ad5ed09 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -44,6 +44,7 @@
#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2068,8 +2069,8 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop over all the attributes in the tuple and check if any attribute is
- * compressed and its compression method is not same as the target
- * atrribute's compression method then decompress it.
+ * compressed and its compression method is not is not supported by the
+ * target attribute then we need to decompress
*/
for (i = 0; i < natts; i++)
{
@@ -2094,12 +2095,13 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
DatumGetPointer(slot->tts_values[attnum - 1]);
/*
- * Get the compression method Oid stored in the toast header and
- * compare it with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
cmoid = toast_get_compression_oid(new_value);
if (OidIsValid(cmoid) &&
- targetTupDesc->attrs[i].attcompression != cmoid)
+ !IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1338e04..6a11f8e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,7 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2986,6 +2986,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5675,6 +5687,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f359200..26a9b85 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,7 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2620,6 +2620,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
}
static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
+static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
COMPARE_SCALAR_FIELD(contype);
@@ -3724,6 +3734,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0605ef3..b584a58 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,7 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2882,6 +2882,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
}
static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
+static void
_outTypeName(StringInfo str, const TypeName *node)
{
WRITE_NODE_TYPE("TYPENAME");
@@ -4258,6 +4268,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 30acfe6..9eb2b04 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,7 +596,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2309,12 +2311,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3437,7 +3439,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3492,13 +3494,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index cf4413d..45f4724 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index cccf3c0..075e1cf 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9037,6 +9037,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16343,6 +16417,33 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 66a2374..f02d9ce 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,7 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all
+ compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -356,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index a8d3b33..e34fbdc 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2103,6 +2103,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e5aea8a..bd53f9b 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index b3d30ac..e6c98e6 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -97,5 +97,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 20d6f96..24deaad 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -481,6 +481,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f9a87de..ce0913e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -624,6 +624,20 @@ typedef struct RangeTableSample
} RangeTableSample;
/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
+/*
* ColumnDef - column definition (used in various creates)
*
* If the column has a default value, we may have the value expression
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4ac4c51..174d447 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -225,12 +225,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 2b37f44..6fb6df5 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -204,12 +204,49 @@ SELECT pg_column_compression(f1) FROM cmpart;
ERROR: relation "cmpart" does not exist
LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
^
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index ce734f7..b61c627 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index d268d46..fbbdbd7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -94,6 +94,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v23-0004-Create-custom-compression-methods.patchtext/x-patch; charset=US-ASCII; name=v23-0004-Create-custom-compression-methods.patchDownload
From 07d0ce50ed87dbade9ea7d773da9906f25d3dcbd Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 16:27:04 +0530
Subject: [PATCH v23 4/6] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 6 ++-
doc/src/sgml/ref/create_access_method.sgml | 12 +++---
doc/src/sgml/ref/create_table.sgml | 6 ++-
src/backend/access/common/detoast.c | 51 ++++++++++++++++++++++----
src/backend/access/common/toast_internals.c | 19 ++++++++--
src/backend/access/compression/compress_lz4.c | 21 ++++++-----
src/backend/access/compression/compress_pglz.c | 20 +++++-----
src/backend/access/index/amapi.c | 51 ++++++++++++++++++++------
src/backend/commands/amcmds.c | 21 ++++++++++-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 +++++
src/bin/pg_dump/pg_dump.c | 8 ++++
src/bin/psql/tab-complete.c | 6 +++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 44 ++++++++++++++++++++--
src/include/access/toast_internals.h | 16 ++++++++
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 ++
src/include/postgres.h | 8 ++++
src/test/regress/expected/compression.out | 40 +++++++++++++++++++-
src/test/regress/sql/compression.sql | 10 +++++
21 files changed, 300 insertions(+), 57 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c9f443a..49c43df 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -391,8 +391,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ This sets the compression method for a column. The Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The supported buit-in
+ compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
The <literal>PRESERVE</literal> list contains a list of compression
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43db..c5ef8b7 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,8 +61,8 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
- are supported at present.
+ Currently, <literal>TABLE</literal>, <literal>INDEX</literal> and
+ <literal>COMPRESSION</literal> access methods are supported.
</para>
</listitem>
</varlistentry>
@@ -77,11 +77,13 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
+ is described in <xref linkend="tableam"/>, the index access method
API is described in <xref linkend="indexam"/>.
</para>
</listitem>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 51a7a97..b645495 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -997,8 +997,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ This sets the compression method for a column. The Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The supported buit-in
+ compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>. The default
is <literal>pglz</literal>.
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index b78d491..399d0d5 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -462,10 +462,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*
* Returns the Oid of the compression method stored in the compressed data. If
* the varlena is not compressed then returns InvalidOid.
+ *
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
*/
Oid
toast_get_compression_oid(struct varlena *attr)
{
+ CompressionId cmid;
+
if (VARATT_IS_EXTERNAL_ONDISK(attr))
{
struct varatt_external toast_pointer;
@@ -485,7 +492,21 @@ toast_get_compression_oid(struct varlena *attr)
else if (!VARATT_IS_COMPRESSED(attr))
return InvalidOid;
- return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the built-in
+ * compression id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
}
/* ----------
@@ -494,7 +515,7 @@ toast_get_compression_oid(struct varlena *attr)
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -508,10 +529,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "invalid compression method id %d", cmid);
}
@@ -527,9 +559,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -543,16 +577,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index b04c5a5..a353906 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 1856cf7..a35c6e4 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -46,10 +46,10 @@ lz4_cmcompress(const struct varlena *value)
* and allocate the memory for holding the compressed data and the header.
*/
max_size = LZ4_compressBound(valsize);
- tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + VARHDRSZ_COMPRESS,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
elog(ERROR, "could not compress data with lz4");
@@ -61,7 +61,7 @@ lz4_cmcompress(const struct varlena *value)
return NULL;
}
- SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -73,7 +73,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -87,9 +87,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
/* decompress data using lz4 routine */
- rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARSIZE(value) - header_size,
VARRAWSIZE_4B_C(value));
if (rawsize < 0)
ereport(ERROR,
@@ -109,7 +109,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -123,9 +124,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
/* decompress partial data using lz4 routine */
- rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARSIZE(value) - header_size,
slicelength,
VARRAWSIZE_4B_C(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 8a4bf42..7f6e742 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -26,7 +26,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -47,11 +47,11 @@ pglz_cmcompress(const struct varlena *value)
* and allocate the memory for holding the compressed data and the header.
*/
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- VARHDRSZ_COMPRESS);
+ header_size);
len = pglz_compress(VARDATA_ANY(value),
valsize,
- (char *) tmp + VARHDRSZ_COMPRESS,
+ (char *) tmp + header_size,
NULL);
if (len < 0)
{
@@ -59,7 +59,7 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
}
- SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -70,15 +70,15 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
VARRAWSIZE_4B_C(value), true);
if (rawsize < 0)
@@ -97,7 +97,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -105,8 +105,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
if (rawsize < 0)
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43..16f8fab 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
@@ -46,14 +47,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +65,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +94,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 3ad4a61..d72cd4e 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -34,6 +34,8 @@
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -83,7 +85,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* amoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -227,6 +241,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -264,6 +280,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9eb2b04..b22ce81 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5305,6 +5305,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95..f026104 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -129,6 +129,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
}
Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
+Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
text *extName;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 075e1cf..8b9f480 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13136,6 +13136,11 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -13146,6 +13151,9 @@ dumpAccessMethod(Archive *fout, AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index e34fbdc..60ecb2f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb..0f7b5b0 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -285,6 +285,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 5a8e23d..11a2f3d 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -15,9 +15,12 @@
#include "postgres.h"
+#include "fmgr.h"
+#include "access/amapi.h"
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
@@ -26,18 +29,25 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Use default compression method if it is not specified. */
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -96,4 +106,30 @@ CompressionIdToOid(CompressionId cmid)
}
}
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+static inline CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 31ff91a..ac28f9e 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -28,9 +28,22 @@ typedef struct toast_compress_header
} toast_compress_header;
/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
+/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
do { \
@@ -38,6 +51,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7..9d65bae 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4a13844..64c9c6e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10777,6 +10777,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '2137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 667927f..ede3b11 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 174d447..aab4375 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -259,13 +259,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index fbbdbd7..0beeadc 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -103,6 +103,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v23-0001-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v23-0001-Built-in-compression-method.patchDownload
From bbc0d0106e48da9327b50e3724b4fb3c38f7fd7f Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v23 1/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 +++++++++++
configure.ac | 17 ++
contrib/test_decoding/expected/ddl.out | 50 ++---
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 +++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 103 +++++++---
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++---
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 164 +++++++++++++++
src/backend/access/compression/compress_pglz.c | 138 +++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++-
src/backend/executor/nodeModifyTable.c | 122 +++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 28 ++-
src/include/access/compressamapi.h | 99 +++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 19 +-
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/alter_table.out | 10 +-
src/test/regress/expected/compression.out | 199 ++++++++++++++++++
src/test/regress/expected/compression_1.out | 187 +++++++++++++++++
src/test/regress/expected/copy2.out | 8 +-
src/test/regress/expected/create_table.out | 142 ++++++-------
src/test/regress/expected/create_table_like.out | 88 ++++----
src/test/regress/expected/domain.out | 16 +-
src/test/regress/expected/foreign_data.out | 258 ++++++++++++------------
src/test/regress/expected/identity.out | 16 +-
src/test/regress/expected/inherit.out | 138 ++++++-------
src/test/regress/expected/insert.out | 118 +++++------
src/test/regress/expected/matview.out | 80 ++++----
src/test/regress/expected/psql.out | 108 +++++-----
src/test/regress/expected/publication.out | 40 ++--
src/test/regress/expected/replica_identity.out | 14 +-
src/test/regress/expected/rowsecurity.out | 16 +-
src/test/regress/expected/rules.out | 30 +--
src/test/regress/expected/stats_ext.out | 10 +-
src/test/regress/expected/update.out | 16 +-
src/test/regress/output/tablespace.source | 16 +-
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 81 ++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
80 files changed, 2291 insertions(+), 676 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044..6ee0776 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a462..4d7ed69 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from parent table's column, however a different compression method can be
+ set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9..51a7a97 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -606,6 +607,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..b78d491 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,78 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Just fetch the toast compress header to know the compression method
+ * in the compressed data.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +543,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1d43d5d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..779f54e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..1856cf7
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Get maximum size of the compressed data that lz4 compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress data using lz4 routine */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress partial data using lz4 routine */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..8a4bf42
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,138 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Get maximum size of the compressed data that pglz compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..b53b6b5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1cb9172..18a9ee3 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e..dd81d5b 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 5d90337..d2da34a 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,113 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then we need to copy the
+ * values/null array from oldslot to the new slot and materialize the new
+ * slot.
+ */
+ if (decompressed_any)
+ {
+ TupleTableSlot *newslot = *outslot;
+
+ /*
+ * If the called has passed an invalid slot then create a new slot.
+ * Otherwise, just clear the existing tuple from the slot. This slot
+ * should be stored by the caller so that it can be reused for
+ * decompressing the subsequent tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+ ExecStoreVirtualTuple(newslot);
+ ExecMaterializeSlot(newslot);
+ ExecClearTuple(slot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2353,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +2994,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3af..cf4413d 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 9d0056a..5e168a3 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index d99b61e..cccf3c0 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1290f96..66a2374 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..ba464d4 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..5a8e23d
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..31ff91a 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,22 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4e0c9be..4a13844 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 758c3ca..44a2ab1 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -616,5 +616,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index d65099c..1601e4b 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1178,6 +1178,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, incase the target attribute's
+ * compression method doesn't match with the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..667927f 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/alter_table.out b/src/test/regress/expected/alter_table.out
index 0ce6ee4..138df4b 100644
--- a/src/test/regress/expected/alter_table.out
+++ b/src/test/regress/expected/alter_table.out
@@ -2196,11 +2196,11 @@ where oid = 'test_storage'::regclass;
create index test_storage_idx on test_storage (b, a);
alter table test_storage alter column a set storage external;
\d+ test_storage
- Table "public.test_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | external | |
- b | integer | | | 0 | plain | |
+ Table "public.test_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | external | pglz | |
+ b | integer | | | 0 | plain | | |
Indexes:
"test_storage_idx" btree (b, a)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..ea487e6
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,199 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..1e0864d
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,187 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
diff --git a/src/test/regress/expected/copy2.out b/src/test/regress/expected/copy2.out
index c64f071..1778c15 100644
--- a/src/test/regress/expected/copy2.out
+++ b/src/test/regress/expected/copy2.out
@@ -511,10 +511,10 @@ begin
end $$ language plpgsql immutable;
alter table check_con_tbl add check (check_con_function(check_con_tbl.*));
\d+ check_con_tbl
- Table "public.check_con_tbl"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
+ Table "public.check_con_tbl"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
Check constraints:
"check_con_tbl_check" CHECK (check_con_function(check_con_tbl.*))
diff --git a/src/test/regress/expected/create_table.out b/src/test/regress/expected/create_table.out
index ed8c01b..3105115 100644
--- a/src/test/regress/expected/create_table.out
+++ b/src/test/regress/expected/create_table.out
@@ -498,11 +498,11 @@ Partition key: RANGE (a oid_ops, plusone(b), c, d COLLATE "C")
Number of partitions: 0
\d+ partitioned2
- Partitioned table "public.partitioned2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Partitioned table "public.partitioned2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition key: RANGE (((a + 1)), substr(b, 1, 5))
Number of partitions: 0
@@ -511,11 +511,11 @@ ERROR: no partition of relation "partitioned2" found for row
DETAIL: Partition key of the failing row contains ((a + 1), substr(b, 1, 5)) = (2, hello).
CREATE TABLE part2_1 PARTITION OF partitioned2 FOR VALUES FROM (-1, 'aaaaa') TO (100, 'ccccc');
\d+ part2_1
- Table "public.part2_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | integer | | | | plain | |
- b | text | | | | extended | |
+ Table "public.part2_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | text | | | | extended | pglz | |
Partition of: partitioned2 FOR VALUES FROM ('-1', 'aaaaa') TO (100, 'ccccc')
Partition constraint: (((a + 1) IS NOT NULL) AND (substr(b, 1, 5) IS NOT NULL) AND (((a + 1) > '-1'::integer) OR (((a + 1) = '-1'::integer) AND (substr(b, 1, 5) >= 'aaaaa'::text))) AND (((a + 1) < 100) OR (((a + 1) = 100) AND (substr(b, 1, 5) < 'ccccc'::text))))
@@ -552,11 +552,11 @@ select * from partitioned where partitioned = '(1,2)'::partitioned;
(2 rows)
\d+ partitioned1
- Table "public.partitioned1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.partitioned1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Partition of: partitioned FOR VALUES IN ('(1,2)')
Partition constraint: (((partitioned1.*)::partitioned IS DISTINCT FROM NULL) AND ((partitioned1.*)::partitioned = '(1,2)'::partitioned))
@@ -609,10 +609,10 @@ CREATE TABLE part_p2 PARTITION OF list_parted FOR VALUES IN (2);
CREATE TABLE part_p3 PARTITION OF list_parted FOR VALUES IN ((2+1));
CREATE TABLE part_null PARTITION OF list_parted FOR VALUES IN (null);
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Partitions: part_null FOR VALUES IN (NULL),
part_p1 FOR VALUES IN (1),
@@ -1049,21 +1049,21 @@ create table test_part_coll_cast2 partition of test_part_coll_posix for values f
drop table test_part_coll_posix;
-- Partition bound in describe output
\d+ part_b
- Table "public.part_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 1 | plain | |
+ Table "public.part_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 1 | plain | | |
Partition of: parted FOR VALUES IN ('b')
Partition constraint: ((a IS NOT NULL) AND (a = 'b'::text))
-- Both partition bound and partition key in describe output
\d+ part_c
- Partitioned table "public.part_c"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Partitioned table "public.part_c"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: parted FOR VALUES IN ('c')
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text))
Partition key: RANGE (b)
@@ -1071,11 +1071,11 @@ Partitions: part_c_1_10 FOR VALUES FROM (1) TO (10)
-- a level-2 partition's constraint will include the parent's expressions
\d+ part_c_1_10
- Table "public.part_c_1_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | not null | 0 | plain | |
+ Table "public.part_c_1_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | not null | 0 | plain | | |
Partition of: part_c FOR VALUES FROM (1) TO (10)
Partition constraint: ((a IS NOT NULL) AND (a = 'c'::text) AND (b IS NOT NULL) AND (b >= 1) AND (b < 10))
@@ -1104,46 +1104,46 @@ Number of partitions: 3 (Use \d+ to list them.)
CREATE TABLE range_parted4 (a int, b int, c int) PARTITION BY RANGE (abs(a), abs(b), c);
CREATE TABLE unbounded_range_part PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE);
\d+ unbounded_range_part
- Table "public.unbounded_range_part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.unbounded_range_part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (MAXVALUE, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL))
DROP TABLE unbounded_range_part;
CREATE TABLE range_parted4_1 PARTITION OF range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE);
\d+ range_parted4_1
- Table "public.range_parted4_1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (MINVALUE, MINVALUE, MINVALUE) TO (1, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND (abs(a) <= 1))
CREATE TABLE range_parted4_2 PARTITION OF range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE);
\d+ range_parted4_2
- Table "public.range_parted4_2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (3, 4, 5) TO (6, 7, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 3) OR ((abs(a) = 3) AND (abs(b) > 4)) OR ((abs(a) = 3) AND (abs(b) = 4) AND (c >= 5))) AND ((abs(a) < 6) OR ((abs(a) = 6) AND (abs(b) <= 7))))
CREATE TABLE range_parted4_3 PARTITION OF range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE);
\d+ range_parted4_3
- Table "public.range_parted4_3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
+ Table "public.range_parted4_3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
Partition of: range_parted4 FOR VALUES FROM (6, 8, MINVALUE) TO (9, MAXVALUE, MAXVALUE)
Partition constraint: ((abs(a) IS NOT NULL) AND (abs(b) IS NOT NULL) AND (c IS NOT NULL) AND ((abs(a) > 6) OR ((abs(a) = 6) AND (abs(b) >= 8))) AND (abs(a) <= 9))
@@ -1175,11 +1175,11 @@ SELECT obj_description('parted_col_comment'::regclass);
(1 row)
\d+ parted_col_comment
- Partitioned table "public.parted_col_comment"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+---------------
- a | integer | | | | plain | | Partition key
- b | text | | | | extended | |
+ Partitioned table "public.parted_col_comment"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+---------------
+ a | integer | | | | plain | | | Partition key
+ b | text | | | | extended | pglz | |
Partition key: LIST (a)
Number of partitions: 0
@@ -1188,10 +1188,10 @@ DROP TABLE parted_col_comment;
CREATE TABLE arrlp (a int[]) PARTITION BY LIST (a);
CREATE TABLE arrlp12 PARTITION OF arrlp FOR VALUES IN ('{1}', '{2}');
\d+ arrlp12
- Table "public.arrlp12"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- a | integer[] | | | | extended | |
+ Table "public.arrlp12"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | integer[] | | | | extended | pglz | |
Partition of: arrlp FOR VALUES IN ('{1}', '{2}')
Partition constraint: ((a IS NOT NULL) AND ((a = '{1}'::integer[]) OR (a = '{2}'::integer[])))
@@ -1201,10 +1201,10 @@ create table boolspart (a bool) partition by list (a);
create table boolspart_t partition of boolspart for values in (true);
create table boolspart_f partition of boolspart for values in (false);
\d+ boolspart
- Partitioned table "public.boolspart"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | boolean | | | | plain | |
+ Partitioned table "public.boolspart"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | boolean | | | | plain | | |
Partition key: LIST (a)
Partitions: boolspart_f FOR VALUES IN (false),
boolspart_t FOR VALUES IN (true)
diff --git a/src/test/regress/expected/create_table_like.out b/src/test/regress/expected/create_table_like.out
index 10d17be..4da878e 100644
--- a/src/test/regress/expected/create_table_like.out
+++ b/src/test/regress/expected/create_table_like.out
@@ -325,32 +325,32 @@ CREATE TABLE ctlt4 (a text, c text);
ALTER TABLE ctlt4 ALTER COLUMN c SET STORAGE EXTERNAL;
CREATE TABLE ctlt12_storage (LIKE ctlt1 INCLUDING STORAGE, LIKE ctlt2 INCLUDING STORAGE);
\d+ ctlt12_storage
- Table "public.ctlt12_storage"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt12_storage"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
CREATE TABLE ctlt12_comments (LIKE ctlt1 INCLUDING COMMENTS, LIKE ctlt2 INCLUDING COMMENTS);
\d+ ctlt12_comments
- Table "public.ctlt12_comments"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | extended | | A
- b | text | | | | extended | | B
- c | text | | | | extended | | C
+ Table "public.ctlt12_comments"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | extended | pglz | | A
+ b | text | | | | extended | pglz | | B
+ c | text | | | | extended | pglz | | C
CREATE TABLE ctlt1_inh (LIKE ctlt1 INCLUDING CONSTRAINTS INCLUDING COMMENTS) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
NOTICE: merging column "b" with inherited definition
NOTICE: merging constraint "ctlt1_a_check" with inherited definition
\d+ ctlt1_inh
- Table "public.ctlt1_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt1_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
Inherits: ctlt1
@@ -364,12 +364,12 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt13_inh () INHERITS (ctlt1, ctlt3);
NOTICE: merging multiple inherited definitions of column "a"
\d+ ctlt13_inh
- Table "public.ctlt13_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | |
- b | text | | | | extended | |
- c | text | | | | external | |
+ Table "public.ctlt13_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | |
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | |
Check constraints:
"ctlt1_a_check" CHECK (length(a) > 2)
"ctlt3_a_check" CHECK (length(a) < 5)
@@ -380,12 +380,12 @@ Inherits: ctlt1,
CREATE TABLE ctlt13_like (LIKE ctlt3 INCLUDING CONSTRAINTS INCLUDING INDEXES INCLUDING COMMENTS INCLUDING STORAGE) INHERITS (ctlt1);
NOTICE: merging column "a" with inherited definition
\d+ ctlt13_like
- Table "public.ctlt13_like"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A3
- b | text | | | | extended | |
- c | text | | | | external | | C
+ Table "public.ctlt13_like"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A3
+ b | text | | | | extended | pglz | |
+ c | text | | | | external | pglz | | C
Indexes:
"ctlt13_like_expr_idx" btree ((a || c))
Check constraints:
@@ -402,11 +402,11 @@ SELECT description FROM pg_description, pg_constraint c WHERE classoid = 'pg_con
CREATE TABLE ctlt_all (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt_all
- Table "public.ctlt_all"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.ctlt_all"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt_all_pkey" PRIMARY KEY, btree (a)
"ctlt_all_b_idx" btree (b)
@@ -440,11 +440,11 @@ DETAIL: MAIN versus EXTENDED
-- Check that LIKE isn't confused by a system catalog of the same name
CREATE TABLE pg_attrdef (LIKE ctlt1 INCLUDING ALL);
\d+ public.pg_attrdef
- Table "public.pg_attrdef"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "public.pg_attrdef"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"pg_attrdef_pkey" PRIMARY KEY, btree (a)
"pg_attrdef_b_idx" btree (b)
@@ -461,11 +461,11 @@ CREATE SCHEMA ctl_schema;
SET LOCAL search_path = ctl_schema, public;
CREATE TABLE ctlt1 (LIKE ctlt1 INCLUDING ALL);
\d+ ctlt1
- Table "ctl_schema.ctlt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- a | text | | not null | | main | | A
- b | text | | | | extended | | B
+ Table "ctl_schema.ctlt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | not null | | main | pglz | | A
+ b | text | | | | extended | pglz | | B
Indexes:
"ctlt1_pkey" PRIMARY KEY, btree (a)
"ctlt1_b_idx" btree (b)
diff --git a/src/test/regress/expected/domain.out b/src/test/regress/expected/domain.out
index 411d5c0..6268b26 100644
--- a/src/test/regress/expected/domain.out
+++ b/src/test/regress/expected/domain.out
@@ -266,10 +266,10 @@ explain (verbose, costs off)
create rule silly as on delete to dcomptable do instead
update dcomptable set d1.r = (d1).r - 1, d1.i = (d1).i + 1 where (d1).i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-----------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptype | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-----------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptype | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
@@ -403,10 +403,10 @@ create rule silly as on delete to dcomptable do instead
update dcomptable set d1[1].r = d1[1].r - 1, d1[1].i = d1[1].i + 1
where d1[1].i > 0;
\d+ dcomptable
- Table "public.dcomptable"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------------+-----------+----------+---------+----------+--------------+-------------
- d1 | dcomptypea | | | | extended | |
+ Table "public.dcomptable"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ d1 | dcomptypea | | | | extended | pglz | |
Indexes:
"dcomptable_d1_key" UNIQUE CONSTRAINT, btree (d1)
Rules:
diff --git a/src/test/regress/expected/foreign_data.out b/src/test/regress/expected/foreign_data.out
index b9e2582..89466da 100644
--- a/src/test/regress/expected/foreign_data.out
+++ b/src/test/regress/expected/foreign_data.out
@@ -1383,12 +1383,12 @@ CREATE TABLE fd_pt1 (
CREATE FOREIGN TABLE ft2 () INHERITS (fd_pt1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1404,12 +1404,12 @@ Inherits: fd_pt1
DROP FOREIGN TABLE ft2;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
CREATE FOREIGN TABLE ft2 (
c1 integer NOT NULL,
@@ -1428,12 +1428,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1471,12 +1471,12 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1496,17 +1496,17 @@ ALTER TABLE fd_pt1 ADD COLUMN c6 integer;
ALTER TABLE fd_pt1 ADD COLUMN c7 integer NOT NULL;
ALTER TABLE fd_pt1 ADD COLUMN c8 integer;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1528,17 +1528,17 @@ Child tables: ct3,
ft3
\d+ ct3
- Table "public.ct3"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | | plain | |
- c5 | integer | | | 0 | plain | |
- c6 | integer | | | | plain | |
- c7 | integer | | not null | | plain | |
- c8 | integer | | | | plain | |
+ Table "public.ct3"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | | plain | | |
+ c5 | integer | | | 0 | plain | | |
+ c6 | integer | | | | plain | | |
+ c7 | integer | | not null | | plain | | |
+ c8 | integer | | | | plain | | |
Inherits: ft2
\d+ ft3
@@ -1570,17 +1570,17 @@ ALTER TABLE fd_pt1 ALTER COLUMN c1 SET (n_distinct = 100);
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STATISTICS -1;
ALTER TABLE fd_pt1 ALTER COLUMN c8 SET STORAGE EXTERNAL;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
- c4 | integer | | | 0 | plain | |
- c5 | integer | | | | plain | |
- c6 | integer | | not null | | plain | |
- c7 | integer | | | | plain | |
- c8 | text | | | | external | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
+ c4 | integer | | | 0 | plain | | |
+ c5 | integer | | | | plain | | |
+ c6 | integer | | not null | | plain | | |
+ c7 | integer | | | | plain | | |
+ c8 | text | | | | external | pglz | |
Child tables: ft2
\d+ ft2
@@ -1608,12 +1608,12 @@ ALTER TABLE fd_pt1 DROP COLUMN c6;
ALTER TABLE fd_pt1 DROP COLUMN c7;
ALTER TABLE fd_pt1 DROP COLUMN c8;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Child tables: ft2
\d+ ft2
@@ -1645,12 +1645,12 @@ SELECT relname, conname, contype, conislocal, coninhcount, connoinherit
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1692,12 +1692,12 @@ ALTER FOREIGN TABLE ft2 ADD CONSTRAINT fd_pt1chk2 CHECK (c2 <> '');
ALTER FOREIGN TABLE ft2 INHERIT fd_pt1;
-- child does not inherit NO INHERIT constraints
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk1" CHECK (c1 > 0) NO INHERIT
"fd_pt1chk2" CHECK (c2 <> ''::text)
@@ -1723,12 +1723,12 @@ ALTER TABLE fd_pt1 DROP CONSTRAINT fd_pt1chk2 CASCADE;
INSERT INTO fd_pt1 VALUES (1, 'fd_pt1'::text, '1994-01-01'::date);
ALTER TABLE fd_pt1 ADD CONSTRAINT fd_pt1chk3 CHECK (c2 <> '') NOT VALID;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text) NOT VALID
Child tables: ft2
@@ -1750,12 +1750,12 @@ Inherits: fd_pt1
-- VALIDATE CONSTRAINT need do nothing on foreign tables
ALTER TABLE fd_pt1 VALIDATE CONSTRAINT fd_pt1chk3;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | 10000 |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | 10000 |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Check constraints:
"fd_pt1chk3" CHECK (c2 <> ''::text)
Child tables: ft2
@@ -1781,12 +1781,12 @@ ALTER TABLE fd_pt1 RENAME COLUMN c3 TO f3;
-- changes name of a constraint recursively
ALTER TABLE fd_pt1 RENAME CONSTRAINT fd_pt1chk3 TO f2_check;
\d+ fd_pt1
- Table "public.fd_pt1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | not null | | plain | 10000 |
- f2 | text | | | | extended | |
- f3 | date | | | | plain | |
+ Table "public.fd_pt1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | not null | | plain | | 10000 |
+ f2 | text | | | | extended | pglz | |
+ f3 | date | | | | plain | | |
Check constraints:
"f2_check" CHECK (f2 <> ''::text)
Child tables: ft2
@@ -1845,12 +1845,12 @@ CREATE TABLE fd_pt2 (
CREATE FOREIGN TABLE fd_pt2_1 PARTITION OF fd_pt2 FOR VALUES IN (1)
SERVER s0 OPTIONS (delimiter ',', quote '"', "be quoted" 'value');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1890,12 +1890,12 @@ ERROR: table "fd_pt2_1" contains column "c4" not found in parent "fd_pt2"
DETAIL: The new partition may contain only the columns present in parent.
DROP FOREIGN TABLE fd_pt2_1;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -1917,12 +1917,12 @@ FDW options: (delimiter ',', quote '"', "be quoted" 'value')
-- no attach partition validation occurs for foreign tables
ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1945,12 +1945,12 @@ ERROR: cannot add column to a partition
ALTER TABLE fd_pt2_1 ALTER c3 SET NOT NULL;
ALTER TABLE fd_pt2_1 ADD CONSTRAINT p21chk CHECK (c2 <> '');
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Partitions: fd_pt2_1 FOR VALUES IN (1)
@@ -1975,12 +1975,12 @@ ERROR: column "c1" is marked NOT NULL in parent table
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ALTER c2 SET NOT NULL;
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Number of partitions: 0
@@ -2003,12 +2003,12 @@ ALTER TABLE fd_pt2 ATTACH PARTITION fd_pt2_1 FOR VALUES IN (1);
ALTER TABLE fd_pt2 DETACH PARTITION fd_pt2_1;
ALTER TABLE fd_pt2 ADD CONSTRAINT fd_pt2chk1 CHECK (c1 > 0);
\d+ fd_pt2
- Partitioned table "public.fd_pt2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- c1 | integer | | not null | | plain | |
- c2 | text | | not null | | extended | |
- c3 | date | | | | plain | |
+ Partitioned table "public.fd_pt2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ c1 | integer | | not null | | plain | | |
+ c2 | text | | not null | | extended | pglz | |
+ c3 | date | | | | plain | | |
Partition key: LIST (c1)
Check constraints:
"fd_pt2chk1" CHECK (c1 > 0)
diff --git a/src/test/regress/expected/identity.out b/src/test/regress/expected/identity.out
index fbca033..dc2af07 100644
--- a/src/test/regress/expected/identity.out
+++ b/src/test/regress/expected/identity.out
@@ -498,14 +498,14 @@ TABLE itest8;
(2 rows)
\d+ itest8
- Table "public.itest8"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+----------------------------------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | not null | generated always as identity | plain | |
- f3 | integer | | not null | generated by default as identity | plain | |
- f4 | bigint | | not null | generated always as identity | plain | |
- f5 | bigint | | | | plain | |
+ Table "public.itest8"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+----------------------------------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | not null | generated always as identity | plain | | |
+ f3 | integer | | not null | generated by default as identity | plain | | |
+ f4 | bigint | | not null | generated always as identity | plain | | |
+ f5 | bigint | | | | plain | | |
\d itest8_f2_seq
Sequence "public.itest8_f2_seq"
diff --git a/src/test/regress/expected/inherit.out b/src/test/regress/expected/inherit.out
index 2b68aef..7e70991 100644
--- a/src/test/regress/expected/inherit.out
+++ b/src/test/regress/expected/inherit.out
@@ -1052,13 +1052,13 @@ ALTER TABLE inhts RENAME aa TO aaa; -- to be failed
ERROR: cannot rename inherited column "aa"
ALTER TABLE inhts RENAME d TO dd;
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aa | integer | | | | plain | |
- b | integer | | | | plain | |
- c | integer | | | | plain | |
- dd | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ dd | integer | | | | plain | | |
Inherits: inht1,
inhs1
@@ -1071,14 +1071,14 @@ NOTICE: merging multiple inherited definitions of column "aa"
NOTICE: merging multiple inherited definitions of column "b"
ALTER TABLE inht1 RENAME aa TO aaa;
\d+ inht4
- Table "public.inht4"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- y | integer | | | | plain | |
- z | integer | | | | plain | |
+ Table "public.inht4"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ y | integer | | | | plain | | |
+ z | integer | | | | plain | | |
Inherits: inht2,
inht3
@@ -1088,14 +1088,14 @@ ALTER TABLE inht1 RENAME aaa TO aaaa;
ALTER TABLE inht1 RENAME b TO bb; -- to be failed
ERROR: cannot rename inherited column "b"
\d+ inhts
- Table "public.inhts"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- aaaa | integer | | | | plain | |
- b | integer | | | | plain | |
- x | integer | | | | plain | |
- c | integer | | | | plain | |
- d | integer | | | | plain | |
+ Table "public.inhts"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ aaaa | integer | | | | plain | | |
+ b | integer | | | | plain | | |
+ x | integer | | | | plain | | |
+ c | integer | | | | plain | | |
+ d | integer | | | | plain | | |
Inherits: inht2,
inhs1
@@ -1135,33 +1135,33 @@ drop cascades to table inht4
CREATE TABLE test_constraints (id int, val1 varchar, val2 int, UNIQUE(val1, val2));
CREATE TABLE test_constraints_inh () INHERITS (test_constraints);
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Indexes:
"test_constraints_val1_val2_key" UNIQUE CONSTRAINT, btree (val1, val2)
Child tables: test_constraints_inh
ALTER TABLE ONLY test_constraints DROP CONSTRAINT test_constraints_val1_val2_key;
\d+ test_constraints
- Table "public.test_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Child tables: test_constraints_inh
\d+ test_constraints_inh
- Table "public.test_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- id | integer | | | | plain | |
- val1 | character varying | | | | extended | |
- val2 | integer | | | | plain | |
+ Table "public.test_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ id | integer | | | | plain | | |
+ val1 | character varying | | | | extended | pglz | |
+ val2 | integer | | | | plain | | |
Inherits: test_constraints
DROP TABLE test_constraints_inh;
@@ -1172,27 +1172,27 @@ CREATE TABLE test_ex_constraints (
);
CREATE TABLE test_ex_constraints_inh () INHERITS (test_ex_constraints);
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Indexes:
"test_ex_constraints_c_excl" EXCLUDE USING gist (c WITH &&)
Child tables: test_ex_constraints_inh
ALTER TABLE test_ex_constraints DROP CONSTRAINT test_ex_constraints_c_excl;
\d+ test_ex_constraints
- Table "public.test_ex_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Child tables: test_ex_constraints_inh
\d+ test_ex_constraints_inh
- Table "public.test_ex_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+---------+--------------+-------------
- c | circle | | | | plain | |
+ Table "public.test_ex_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+---------+-------------+--------------+-------------
+ c | circle | | | | plain | | |
Inherits: test_ex_constraints
DROP TABLE test_ex_constraints_inh;
@@ -1202,37 +1202,37 @@ CREATE TABLE test_primary_constraints(id int PRIMARY KEY);
CREATE TABLE test_foreign_constraints(id1 int REFERENCES test_primary_constraints(id));
CREATE TABLE test_foreign_constraints_inh () INHERITS (test_foreign_constraints);
\d+ test_primary_constraints
- Table "public.test_primary_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id | integer | | not null | | plain | |
+ Table "public.test_primary_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id | integer | | not null | | plain | | |
Indexes:
"test_primary_constraints_pkey" PRIMARY KEY, btree (id)
Referenced by:
TABLE "test_foreign_constraints" CONSTRAINT "test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Foreign-key constraints:
"test_foreign_constraints_id1_fkey" FOREIGN KEY (id1) REFERENCES test_primary_constraints(id)
Child tables: test_foreign_constraints_inh
ALTER TABLE test_foreign_constraints DROP CONSTRAINT test_foreign_constraints_id1_fkey;
\d+ test_foreign_constraints
- Table "public.test_foreign_constraints"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Child tables: test_foreign_constraints_inh
\d+ test_foreign_constraints_inh
- Table "public.test_foreign_constraints_inh"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- id1 | integer | | | | plain | |
+ Table "public.test_foreign_constraints_inh"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ id1 | integer | | | | plain | | |
Inherits: test_foreign_constraints
DROP TABLE test_foreign_constraints_inh;
diff --git a/src/test/regress/expected/insert.out b/src/test/regress/expected/insert.out
index da50ee3..3cd4ad9 100644
--- a/src/test/regress/expected/insert.out
+++ b/src/test/regress/expected/insert.out
@@ -142,11 +142,11 @@ create rule irule3 as on insert to inserttest2 do also
insert into inserttest (f4[1].if1, f4[1].if2[2])
select new.f1, new.f2;
\d+ inserttest2
- Table "public.inserttest2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+--------+-----------+----------+---------+----------+--------------+-------------
- f1 | bigint | | | | plain | |
- f2 | text | | | | extended | |
+ Table "public.inserttest2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+--------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | bigint | | | | plain | | |
+ f2 | text | | | | extended | pglz | |
Rules:
irule1 AS
ON INSERT TO inserttest2 DO INSERT INTO inserttest (f3.if2[1], f3.if2[2])
@@ -448,11 +448,11 @@ from hash_parted order by part;
-- test \d+ output on a table which has both partitioned and unpartitioned
-- partitions
\d+ list_parted
- Partitioned table "public.list_parted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.list_parted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: LIST (lower(a))
Partitions: part_aa_bb FOR VALUES IN ('aa', 'bb'),
part_cc_dd FOR VALUES IN ('cc', 'dd'),
@@ -470,10 +470,10 @@ drop table hash_parted;
create table list_parted (a int) partition by list (a);
create table part_default partition of list_parted default;
\d+ part_default
- Table "public.part_default"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "public.part_default"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: list_parted DEFAULT
No partition constraint
@@ -853,11 +853,11 @@ create table mcrparted6_common_ge_10 partition of mcrparted for values from ('co
create table mcrparted7_gt_common_lt_d partition of mcrparted for values from ('common', maxvalue) to ('d', minvalue);
create table mcrparted8_ge_d partition of mcrparted for values from ('d', minvalue) to (maxvalue, maxvalue);
\d+ mcrparted
- Partitioned table "public.mcrparted"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Partitioned table "public.mcrparted"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition key: RANGE (a, b)
Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE),
mcrparted2_b FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE),
@@ -869,74 +869,74 @@ Partitions: mcrparted1_lt_b FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVAL
mcrparted8_ge_d FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
\d+ mcrparted1_lt_b
- Table "public.mcrparted1_lt_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted1_lt_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM (MINVALUE, MINVALUE) TO ('b', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a < 'b'::text))
\d+ mcrparted2_b
- Table "public.mcrparted2_b"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted2_b"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('b', MINVALUE) TO ('c', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'b'::text) AND (a < 'c'::text))
\d+ mcrparted3_c_to_common
- Table "public.mcrparted3_c_to_common"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted3_c_to_common"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('c', MINVALUE) TO ('common', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'c'::text) AND (a < 'common'::text))
\d+ mcrparted4_common_lt_0
- Table "public.mcrparted4_common_lt_0"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted4_common_lt_0"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MINVALUE) TO ('common', 0)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b < 0))
\d+ mcrparted5_common_0_to_10
- Table "public.mcrparted5_common_0_to_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted5_common_0_to_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 0) TO ('common', 10)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 0) AND (b < 10))
\d+ mcrparted6_common_ge_10
- Table "public.mcrparted6_common_ge_10"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted6_common_ge_10"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', 10) TO ('common', MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a = 'common'::text) AND (b >= 10))
\d+ mcrparted7_gt_common_lt_d
- Table "public.mcrparted7_gt_common_lt_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted7_gt_common_lt_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('common', MAXVALUE) TO ('d', MINVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a > 'common'::text) AND (a < 'd'::text))
\d+ mcrparted8_ge_d
- Table "public.mcrparted8_ge_d"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | integer | | | | plain | |
+ Table "public.mcrparted8_ge_d"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | integer | | | | plain | | |
Partition of: mcrparted FOR VALUES FROM ('d', MINVALUE) TO (MAXVALUE, MAXVALUE)
Partition constraint: ((a IS NOT NULL) AND (b IS NOT NULL) AND (a >= 'd'::text))
diff --git a/src/test/regress/expected/matview.out b/src/test/regress/expected/matview.out
index 4b3a2e0..e078818 100644
--- a/src/test/regress/expected/matview.out
+++ b/src/test/regress/expected/matview.out
@@ -94,11 +94,11 @@ CREATE MATERIALIZED VIEW mvtest_bb AS SELECT * FROM mvtest_tvvmv;
CREATE INDEX mvtest_aa ON mvtest_bb (grandtot);
-- check that plans seem reasonable
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -106,11 +106,11 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvm
- Materialized view "public.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "public.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -118,19 +118,19 @@ View definition:
ORDER BY mvtest_tv.type;
\d+ mvtest_tvvm
- Materialized view "public.mvtest_tvvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tvv.grandtot
FROM mvtest_tvv;
\d+ mvtest_bb
- Materialized view "public.mvtest_bb"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_bb"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_aa" btree (grandtot)
View definition:
@@ -142,10 +142,10 @@ CREATE SCHEMA mvtest_mvschema;
ALTER MATERIALIZED VIEW mvtest_tvm SET SCHEMA mvtest_mvschema;
\d+ mvtest_tvm
\d+ mvtest_tvmm
- Materialized view "public.mvtest_tvmm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- grandtot | numeric | | | | main | |
+ Materialized view "public.mvtest_tvmm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ grandtot | numeric | | | | main | pglz | |
Indexes:
"mvtest_tvmm_expr" UNIQUE, btree ((grandtot > 0::numeric))
"mvtest_tvmm_pred" UNIQUE, btree (grandtot) WHERE grandtot < 0::numeric
@@ -155,11 +155,11 @@ View definition:
SET search_path = mvtest_mvschema, public;
\d+ mvtest_tvm
- Materialized view "mvtest_mvschema.mvtest_tvm"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- type | text | | | | extended | |
- totamt | numeric | | | | main | |
+ Materialized view "mvtest_mvschema.mvtest_tvm"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ type | text | | | | extended | pglz | |
+ totamt | numeric | | | | main | pglz | |
View definition:
SELECT mvtest_tv.type,
mvtest_tv.totamt
@@ -356,11 +356,11 @@ UNION ALL
CREATE MATERIALIZED VIEW mv_test2 AS SELECT moo, 2*moo FROM mvtest_vt2 UNION ALL SELECT moo, 3*moo FROM mvtest_vt2;
\d+ mv_test2
- Materialized view "public.mv_test2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+---------+---------+--------------+-------------
- moo | integer | | | | plain | |
- ?column? | integer | | | | plain | |
+ Materialized view "public.mv_test2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ moo | integer | | | | plain | | |
+ ?column? | integer | | | | plain | | |
View definition:
SELECT mvtest_vt2.moo,
2 * mvtest_vt2.moo
@@ -493,14 +493,14 @@ drop cascades to materialized view mvtest_mv_v_4
CREATE MATERIALIZED VIEW mv_unspecified_types AS
SELECT 42 as i, 42.5 as num, 'foo' as u, 'foo'::unknown as u2, null as n;
\d+ mv_unspecified_types
- Materialized view "public.mv_unspecified_types"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+----------+--------------+-------------
- i | integer | | | | plain | |
- num | numeric | | | | main | |
- u | text | | | | extended | |
- u2 | text | | | | extended | |
- n | text | | | | extended | |
+ Materialized view "public.mv_unspecified_types"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ i | integer | | | | plain | | |
+ num | numeric | | | | main | pglz | |
+ u | text | | | | extended | pglz | |
+ u2 | text | | | | extended | pglz | |
+ n | text | | | | extended | pglz | |
View definition:
SELECT 42 AS i,
42.5 AS num,
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..d6912f2 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -2813,34 +2813,34 @@ CREATE TABLE tbl_heap(f1 int, f2 char(100)) using heap;
CREATE VIEW view_heap_psql AS SELECT f1 from tbl_heap_psql;
CREATE MATERIALIZED VIEW mat_view_heap_psql USING heap_psql AS SELECT f1 from tbl_heap_psql;
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
\set HIDE_TABLEAM off
\d+ tbl_heap_psql
- Table "tableam_display.tbl_heap_psql"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap_psql"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap_psql
\d+ tbl_heap
- Table "tableam_display.tbl_heap"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+----------------+-----------+----------+---------+----------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | character(100) | | | | extended | |
+ Table "tableam_display.tbl_heap"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+----------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | character(100) | | | | extended | pglz | |
Access method: heap
-- AM is displayed for tables, indexes and materialized views.
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/expected/publication.out b/src/test/regress/expected/publication.out
index 63d6ab7..71388c1 100644
--- a/src/test/regress/expected/publication.out
+++ b/src/test/regress/expected/publication.out
@@ -74,11 +74,11 @@ SELECT pubname, puballtables FROM pg_publication WHERE pubname = 'testpub_forall
(1 row)
\d+ testpub_tbl2
- Table "public.testpub_tbl2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl2_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl2_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -187,22 +187,22 @@ ALTER PUBLICATION testpub_default SET TABLE testpub_tbl1;
ALTER PUBLICATION testpub_default ADD TABLE pub_test.testpub_nopk;
ALTER PUBLICATION testpib_ins_trunct ADD TABLE pub_test.testpub_nopk, testpub_tbl1;
\d+ pub_test.testpub_nopk
- Table "pub_test.testpub_nopk"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- foo | integer | | | | plain | |
- bar | integer | | | | plain | |
+ Table "pub_test.testpub_nopk"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ foo | integer | | | | plain | | |
+ bar | integer | | | | plain | | |
Publications:
"testpib_ins_trunct"
"testpub_default"
"testpub_fortbl"
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
@@ -224,11 +224,11 @@ ALTER PUBLICATION testpub_default DROP TABLE testpub_tbl1, pub_test.testpub_nopk
ALTER PUBLICATION testpub_default DROP TABLE pub_test.testpub_nopk;
ERROR: relation "testpub_nopk" is not part of the publication
\d+ testpub_tbl1
- Table "public.testpub_tbl1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | |
- data | text | | | | extended | |
+ Table "public.testpub_tbl1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('testpub_tbl1_id_seq'::regclass) | plain | | |
+ data | text | | | | extended | pglz | |
Indexes:
"testpub_tbl1_pkey" PRIMARY KEY, btree (id)
Publications:
diff --git a/src/test/regress/expected/replica_identity.out b/src/test/regress/expected/replica_identity.out
index 7900219..20ac001 100644
--- a/src/test/regress/expected/replica_identity.out
+++ b/src/test/regress/expected/replica_identity.out
@@ -153,13 +153,13 @@ SELECT relreplident FROM pg_class WHERE oid = 'test_replica_identity'::regclass;
(1 row)
\d+ test_replica_identity
- Table "public.test_replica_identity"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | |
- keya | text | | not null | | extended | |
- keyb | text | | not null | | extended | |
- nonkey | text | | | | extended | |
+ Table "public.test_replica_identity"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('test_replica_identity_id_seq'::regclass) | plain | | |
+ keya | text | | not null | | extended | pglz | |
+ keyb | text | | not null | | extended | pglz | |
+ nonkey | text | | | | extended | pglz | |
Indexes:
"test_replica_identity_pkey" PRIMARY KEY, btree (id)
"test_replica_identity_expr" UNIQUE, btree (keya, keyb, (3))
diff --git a/src/test/regress/expected/rowsecurity.out b/src/test/regress/expected/rowsecurity.out
index 9506aae..7ebdc30 100644
--- a/src/test/regress/expected/rowsecurity.out
+++ b/src/test/regress/expected/rowsecurity.out
@@ -938,14 +938,14 @@ CREATE POLICY pp1 ON part_document AS PERMISSIVE
CREATE POLICY pp1r ON part_document AS RESTRICTIVE TO regress_rls_dave
USING (cid < 55);
\d+ part_document
- Partitioned table "regress_rls_schema.part_document"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
----------+---------+-----------+----------+---------+----------+--------------+-------------
- did | integer | | | | plain | |
- cid | integer | | | | plain | |
- dlevel | integer | | not null | | plain | |
- dauthor | name | | | | plain | |
- dtitle | text | | | | extended | |
+ Partitioned table "regress_rls_schema.part_document"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+---------+---------+-----------+----------+---------+----------+-------------+--------------+-------------
+ did | integer | | | | plain | | |
+ cid | integer | | | | plain | | |
+ dlevel | integer | | not null | | plain | | |
+ dauthor | name | | | | plain | | |
+ dtitle | text | | | | extended | pglz | |
Partition key: RANGE (cid)
Policies:
POLICY "pp1"
diff --git a/src/test/regress/expected/rules.out b/src/test/regress/expected/rules.out
index 6173473..8546f02 100644
--- a/src/test/regress/expected/rules.out
+++ b/src/test/regress/expected/rules.out
@@ -3035,11 +3035,11 @@ select * from rules_log;
create rule r3 as on delete to rules_src do notify rules_src_deletion;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3055,11 +3055,11 @@ Rules:
create rule r4 as on insert to rules_src do instead insert into rules_log AS trgt SELECT NEW.* RETURNING trgt.f1, trgt.f2;
create rule r5 as on update to rules_src do instead UPDATE rules_log AS trgt SET tag = 'updated' WHERE trgt.f1 = new.f1;
\d+ rules_src
- Table "public.rules_src"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rules_src"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
r1 AS
ON UPDATE TO rules_src DO INSERT INTO rules_log (f1, f2, tag) VALUES (old.f1,old.f2,'old'::text), (new.f1,new.f2,'new'::text)
@@ -3086,11 +3086,11 @@ create rule rr as on update to rule_t1 do instead UPDATE rule_dest trgt
SET (f2[1], f1, tag) = (SELECT new.f2, new.f1, 'updated'::varchar)
WHERE trgt.f1 = new.f1 RETURNING new.*;
\d+ rule_t1
- Table "public.rule_t1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- f1 | integer | | | | plain | |
- f2 | integer | | | | plain | |
+ Table "public.rule_t1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+ f2 | integer | | | | plain | | |
Rules:
rr AS
ON UPDATE TO rule_t1 DO INSTEAD UPDATE rule_dest trgt SET (f2[1], f1, tag) = ( SELECT new.f2,
diff --git a/src/test/regress/expected/stats_ext.out b/src/test/regress/expected/stats_ext.out
index 431b3fa..a36b7e6 100644
--- a/src/test/regress/expected/stats_ext.out
+++ b/src/test/regress/expected/stats_ext.out
@@ -125,11 +125,11 @@ SELECT stxname, stxdndistinct, stxddependencies, stxdmcv
ALTER STATISTICS ab1_a_b_stats SET STATISTICS -1;
\d+ ab1
- Table "public.ab1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
- b | integer | | | | plain | |
+ Table "public.ab1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
+ b | integer | | | | plain | | |
Statistics objects:
"public"."ab1_a_b_stats" (ndistinct, dependencies, mcv) ON a, b FROM ab1
diff --git a/src/test/regress/expected/update.out b/src/test/regress/expected/update.out
index bf939d7..58e996f 100644
--- a/src/test/regress/expected/update.out
+++ b/src/test/regress/expected/update.out
@@ -711,14 +711,14 @@ DROP TRIGGER d15_insert_trig ON part_d_15_20;
:init_range_parted;
create table part_def partition of range_parted default;
\d+ part_def
- Table "public.part_def"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+-------------------+-----------+----------+---------+----------+--------------+-------------
- a | text | | | | extended | |
- b | bigint | | | | plain | |
- c | numeric | | | | main | |
- d | integer | | | | plain | |
- e | character varying | | | | extended | |
+ Table "public.part_def"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ a | text | | | | extended | pglz | |
+ b | bigint | | | | plain | | |
+ c | numeric | | | | main | pglz | |
+ d | integer | | | | plain | | |
+ e | character varying | | | | extended | pglz | |
Partition of: range_parted DEFAULT
Partition constraint: (NOT ((a IS NOT NULL) AND (b IS NOT NULL) AND (((a = 'a'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'a'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '1'::bigint) AND (b < '10'::bigint)) OR ((a = 'b'::text) AND (b >= '10'::bigint) AND (b < '20'::bigint)) OR ((a = 'b'::text) AND (b >= '20'::bigint) AND (b < '30'::bigint)))))
diff --git a/src/test/regress/output/tablespace.source b/src/test/regress/output/tablespace.source
index 1bbe7e0..57fbb26 100644
--- a/src/test/regress/output/tablespace.source
+++ b/src/test/regress/output/tablespace.source
@@ -330,10 +330,10 @@ Indexes:
Number of partitions: 2 (Use \d+ to list them.)
\d+ testschema.part
- Partitioned table "testschema.part"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Partitioned table "testschema.part"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition key: LIST (a)
Indexes:
"part_a_idx" btree (a), tablespace "regress_tblspace"
@@ -350,10 +350,10 @@ Indexes:
"part1_a_idx" btree (a), tablespace "regress_tblspace"
\d+ testschema.part1
- Table "testschema.part1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+---------+-----------+----------+---------+---------+--------------+-------------
- a | integer | | | | plain | |
+ Table "testschema.part1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ a | integer | | | | plain | | |
Partition of: testschema.part FOR VALUES IN (1)
Partition constraint: ((a IS NOT NULL) AND (a = 1))
Indexes:
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..9721097
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,81 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 1d540fe..afc248a 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v23-0002-alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v23-0002-alter-table-set-compression.patchDownload
From 399c3f101f3fadba83facc26119b4107b76e3b4f Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v23 2/6] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 ++++++++++++++++++++++------
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 61 ++++++++
src/test/regress/expected/compression_1.out | 48 ++++++
src/test/regress/sql/compression.sql | 21 +++
13 files changed, 350 insertions(+), 55 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0..748ec29 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3f..779b4e5 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dd81d5b..cb2ad13 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7768,6 +7791,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15034,6 +15078,92 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index d2da34a..93f63da 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df..30acfe6 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 5f0e775..a8d3b33 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d..5a314f4 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 44a2ab1..7759b53 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -618,5 +618,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index ea487e6..4ac4c51 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -164,6 +164,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 1e0864d..2b37f44 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -156,6 +156,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 9721097..d268d46 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -73,6 +73,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v23-0005-new-compression-method-extension-for-zlib.patchtext/x-patch; charset=US-ASCII; name=v23-0005-new-compression-method-extension-for-zlib.patchDownload
From d3177514c5a3b82d08397c84b34cdf070fddab19 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v23 5/6] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 ++++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 ++
contrib/cmzlib/expected/cmzlib.out | 53 +++++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index f27e458..9e452d8 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000..5dcb3ff
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000..956fbe7
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000..41f2f95
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000..686a7c7
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000..2eb10f3
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000..2b6fac7
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ pglz
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000..ea8d206
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
1.8.3.1
v23-0006-Support-compression-methods-options.patchtext/x-patch; charset=US-ASCII; name=v23-0006-Support-compression-methods-options.patchDownload
From 6ac7eeef8caa76eeb374ac9560ee644b3b9b9bd5 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 16:29:55 +0530
Subject: [PATCH v23 6/6] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 ++++++++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 7 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 ++++++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 75 +++++++++++++++-
src/backend/access/compression/compress_pglz.c | 116 +++++++++++++++++++++----
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 +++-
src/backend/catalog/index.c | 43 +++++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 ++++++++++++++++-
src/backend/commands/foreigncmds.c | 44 ----------
src/backend/commands/tablecmds.c | 102 +++++++++++++++-------
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 23 ++++-
src/bin/pg_dump/pg_dump.h | 1 +
src/include/access/compressamapi.h | 19 +++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 +++++++++++
src/test/regress/expected/compression_1.out | 55 ++++++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 ++++
36 files changed, 729 insertions(+), 142 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7..d8c2865 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -46,29 +47,83 @@ typedef struct
} zlib_state;
/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
+/*
* zlib_cmcompress - compression routine for zlib compression method
*
* Compresses source into dest using the default compression level.
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 49c43df..0f861e7 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -396,7 +396,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
it can be set to any available compression method. The supported buit-in
compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>.
+ was used when building <productname>PostgreSQL</productname>. If the
+ compression method has options they can be specified with the <literal>WITH
+ </literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index b645495..715fc73 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
This sets the compression method for a column. The Compression method
@@ -1003,7 +1003,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>. The default
- is <literal>pglz</literal>.
+ is <literal>pglz</literal>. If the compression method has options they can
+ be specified with the <literal>WITH</literal> parameter.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712..76f824b 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d..0d3307e 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3e..2dda6c0 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a353906..5ed3312 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index a35c6e4..644c318 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -17,17 +17,81 @@
#endif
#include "access/compressamapi.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ /*
+ * We don't need to check the range for the acceleration parameter
+ * because the LZ4_compress_fast will automatically replace the
+ * values <=0 with LZ4_ACCELERATION_DEFAULT (currently == 1). And,
+ * Values > LZ4_ACCELERATION_MAX with LZ4_ACCELERATION_MAX
+ * (currently == 65537c).
+ */
+ if (strcmp(def->defname, "acceleration") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+#endif
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+#endif
+}
+
+/*
* lz4_cmcompress - compression routine for lz4 compression method
*
* Compresses source into dest using the default strategy. Returns the
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -37,6 +101,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -48,9 +113,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
elog(ERROR, "could not compress data with lz4");
@@ -146,6 +211,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
*/
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 7f6e742..1ca9128 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -14,11 +14,93 @@
#include "postgres.h"
#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -26,42 +108,41 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
- /*
- * Get maximum size of the compressed data that pglz compression may output
- * and allocate the memory for holding the compressed data and the header.
- */
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
header_size);
- len = pglz_compress(VARDATA_ANY(value),
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
- if (len < 0)
+ strategy);
+
+ if (len >= 0)
{
- pfree(tmp);
- return NULL;
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
}
- SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ pfree(tmp);
- return tmp;
+ return NULL;
}
/*
@@ -125,10 +206,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
*/
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9..c9d44c6 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004..a43e0e1 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b53b6b5..18b0481 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -741,6 +741,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -796,6 +797,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -843,6 +849,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -861,7 +868,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -893,7 +901,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1160,6 +1168,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1418,7 +1427,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 18a9ee3..5869163 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -108,10 +108,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -273,7 +275,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -334,6 +337,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -350,6 +356,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -490,7 +513,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -508,7 +531,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -721,6 +745,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -876,6 +901,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -884,7 +911,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -975,7 +1003,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481..348b7d3 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 096a06f..0ad946d 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -699,6 +699,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index 4ff75d0..c313ed2 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -29,6 +29,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -181,7 +182,7 @@ BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -215,6 +216,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
errmsg("not built with lz4 support")));
#endif
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -286,15 +301,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttribute's compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103f..ae9ae2c 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fcfe4fd..a18c0ca 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -609,6 +609,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -813,6 +814,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -866,8 +869,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -921,8 +925,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2432,16 +2439,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2478,8 +2483,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2729,14 +2733,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6271,6 +6270,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6434,6 +6434,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6444,7 +6445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7840,13 +7841,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* the respective input values are valid.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7882,7 +7890,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
if (newstorage != '\0')
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7973,7 +8003,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15145,9 +15175,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15182,7 +15214,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15192,8 +15225,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15201,8 +15243,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6a11f8e..991340e 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2993,6 +2993,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 26a9b85..57a2975 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2624,6 +2624,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b584a58..bc5c64d 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2888,6 +2888,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b22ce81..00fe3b7 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3503,11 +3504,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3515,14 +3522,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 45f4724..3a33bce 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 8b9f480..93b997f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8709,10 +8709,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8765,6 +8772,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8794,6 +8802,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15959,7 +15968,8 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -16005,6 +16015,10 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
@@ -16436,6 +16450,9 @@ dumpTableSchema(Archive *fout, TableInfo *tbinfo)
appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, "\nWITH (%s) ", tbinfo->attcmoptions[j]);
+
if (cminfo->nitems > 0)
{
appendPQExpBuffer(q, "\nPRESERVE (");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index f02d9ce..470532b 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,7 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
+ char **attcmoptions; /* per-attribute compression options */
struct _attrCompressionInfo **attcompression; /* per-attribute all
compression data */
/*
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 11a2f3d..b6913e1 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -19,6 +19,7 @@
#include "access/amapi.h"
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "nodes/pg_list.h"
/*
@@ -40,8 +41,11 @@ typedef enum CompressionId
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -52,14 +56,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc3..fe8de40 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index ac28f9e..864de31 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -54,7 +54,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b..5ecb6f3 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 797e78b..7c74bc3 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -178,6 +178,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bd53f9b..8271e97 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -134,6 +134,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -146,9 +148,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ce0913e..c3c878b 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index aab4375..26e6f0c 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -296,6 +296,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6fb6df5..477eda1 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -240,6 +240,61 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ERROR: not built with lz4 support
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+HINT: use "pg_column_compression" function for list of compression methods
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 9ebe28a..4e3b0c1 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -99,6 +99,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -109,7 +110,7 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
-- system catalogs without primary keys
--
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0beeadc..14132b2 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -113,6 +113,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Feb 10, 2021 at 9:52 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
[ new patches ]
I think that in both varattrib_4b and toast_internals.h it would be
better to pick a less generic field name. In toast_internals.h it's
just info; in postgres.h it's va_info. But:
[rhaas pgsql]$ git grep info | wc -l
24552
There are no references in the current source tree to va_info, so at
least that one is greppable, but it's still not very descriptive. I
suggest info -> tcinfo and va_info -> va_tcinfo, where "tc" stands for
"TOAST compression". Looking through 24552 references to info to find
the ones that pertain to this feature might take longer than searching
the somewhat shorter list of references to tcinfo, which prepatch is
just:
[rhaas pgsql]$ git grep tcinfo | wc -l
0
I don't see why we should allow for datum_decompress to be optional,
as toast_decompress_datum_slice does. Likely every serious compression
method will support that anyway. If not, the compression AM can deal
with the problem, rather than having the core code do it. That will
save some tiny amount of performance, too.
src/backend/access/compression/Makefile is missing a copyright header.
It's really sad that lz4_cmdecompress_slice allocates
VARRAWSIZE_4B_C(value) + VARHDRSZ rather than slicelength + VARHDRSZ
as pglz_cmdecompress_slice() does. Is that a mistake, or is that
necessary for some reason? If it's a mistake, let's fix it. If it's
necessary, let's add a comment about why, probably starting with
"Unfortunately, ....".
I think you have a fairly big problem with row types. Consider this example:
create table t1 (a int, b text compression pglz);
create table t2 (a int, b text compression lz4);
create table t3 (x t1);
insert into t1 values (1, repeat('foo', 1000));
insert into t2 values (1, repeat('foo', 1000));
insert into t3 select t1 from t1;
insert into t3 select row(a, b)::t1 from t2;
rhaas=# select pg_column_compression((t3.x).b) from t3;
pg_column_compression
-----------------------
pglz
lz4
(2 rows)
That's not good, because now
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Feb 10, 2021 at 3:06 PM Robert Haas <robertmhaas@gmail.com> wrote:
I think you have a fairly big problem with row types. Consider this example:
create table t1 (a int, b text compression pglz);
create table t2 (a int, b text compression lz4);
create table t3 (x t1);
insert into t1 values (1, repeat('foo', 1000));
insert into t2 values (1, repeat('foo', 1000));
insert into t3 select t1 from t1;
insert into t3 select row(a, b)::t1 from t2;rhaas=# select pg_column_compression((t3.x).b) from t3;
pg_column_compression
-----------------------
pglz
lz4
(2 rows)That's not good, because now
...because now I hit send too soon. Also, because now column b has
implicit dependencies on both compression AMs and the rest of the
system has no idea. I think we probably should have a rule that
nothing except pglz is allowed inside of a record, just to keep it
simple. The record overall can be toasted so it's not clear why we
should also be toasting the original columns at all, but I think
precedent probably argues for continuing to allow PGLZ, as it can
already be like that on disk and pg_upgrade is a thing. The same kind
of issue probably exists for arrays and range types.
I poked around a bit trying to find ways of getting data compressed
with one compression method into a column that was marked with another
compression method, but wasn't able to break it.
In CompareCompressionMethodAndDecompress, I think this is still
playing a bit fast and loose with the rules around slots. I think we
can do better. Suppose that at the point where we discover that we
need to decompress at least one attribute, we create the new slot
right then, and also memcpy tts_values and tts_isnull. Then, for that
attribute and any future attributes that need decompression, we reset
tts_values in the *new* slot, leaving the old one untouched. Then,
after finishing all the attributes, the if (decompressed_any) block,
you just have a lot less stuff to do. The advantage of this is that
you haven't tainted the old slot; it's still got whatever contents it
had before, and is in a clean state, which seems better to me.
It's unclear to me whether this function actually needs to
ExecMaterializeSlot(newslot). It definitely does need to
ExecStoreVirtualTuple(newslot) and I think it's a very good idea, if
not absolutely mandatory, for it not to modify anything about the old
slot. But what's the argument that the new slot needs to be
materialized at this point? It may be needed, if the old slot would've
had to be materialized at this point. But it's something to think
about.
The CREATE TABLE documentation says that COMPRESSION is a kind of
column constraint, but that's wrong. For example, you can't write
CREATE TABLE a (b int4 CONSTRAINT thunk COMPRESSION lz4), for example,
contrary to what the syntax summary implies. When you fix this so that
the documentation matches the grammar change, you may also need to
move the longer description further up in create_table.sgml so the
order matches.
The use of VARHDRSZ_COMPRESS in toast_get_compression_oid() appears to
be incorrect. VARHDRSZ_COMPRESS is offsetof(varattrib_4b,
va_compressed.va_data). But what gets externalized in the case of a
compressed datum is just VARDATA(dval), which excludes the length
word, unlike VARHDRSZ_COMPRESS, which does not. This has no
consequences since we're only going to fetch 1 chunk either way, but I
think we should make it correct.
TOAST_COMPRESS_SET_SIZE_AND_METHOD() could Assert something about cm_method.
Small delta patch with a few other suggested changes attached.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
fixups.patchapplication/octet-stream; name=fixups.patchDownload
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 4d7ed698b9..1e9a4625cc 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,9 +3762,6 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
- By default, each column in a partition inherits the compression method
- from parent table's column, however a different compression method can be
- set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 51a7a977a5..a4b297340f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -997,10 +997,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
- <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>. The default
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
is <literal>pglz</literal>.
</para>
</listitem>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index b78d49167b..95d5b1c12a 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -477,8 +477,8 @@ toast_get_compression_oid(struct varlena *attr)
return InvalidOid;
/*
- * Just fetch the toast compress header to know the compression method
- * in the compressed data.
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
*/
attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
}
@@ -503,7 +503,6 @@ toast_get_compression_handler(struct varlena *attr)
cmid = TOAST_COMPRESS_METHOD(attr);
- /* Get the handler routines for the compression method */
switch (cmid)
{
case PGLZ_COMPRESSION_ID:
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 1856cf7df7..3079eff7eb 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -23,7 +23,7 @@
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
- * Compresses source into dest using the default strategy. Returns the
+ * Compresses source into dest using the LZ4 defaults. Returns the
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
@@ -42,8 +42,8 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
/*
- * Get maximum size of the compressed data that lz4 compression may output
- * and allocate the memory for holding the compressed data and the header.
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
*/
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
@@ -83,10 +83,10 @@ lz4_cmdecompress(const struct varlena *value)
int32 rawsize;
struct varlena *result;
- /* allocate memory for holding the uncompressed data */
+ /* allocate memory for the uncompressed data */
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- /* decompress data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18e8ecfd90..d8483c36b3 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1183,8 +1183,8 @@ typedef struct ModifyTableState
TupleTableSlot *mt_root_tuple_slot;
/*
- * Slot for storing the modified tuple, incase the target attribute's
- * compression method doesn't match with the source table.
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
*/
TupleTableSlot *mt_decompress_tuple_slot;
Re: [HACKERS] Custom compression methods
On Thu, Feb 11, 2021 at 3:26 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Feb 10, 2021 at 3:06 PM Robert Haas <robertmhaas@gmail.com> wrote:
I think you have a fairly big problem with row types. Consider this example:
create table t1 (a int, b text compression pglz);
create table t2 (a int, b text compression lz4);
create table t3 (x t1);
insert into t1 values (1, repeat('foo', 1000));
insert into t2 values (1, repeat('foo', 1000));
insert into t3 select t1 from t1;
insert into t3 select row(a, b)::t1 from t2;rhaas=# select pg_column_compression((t3.x).b) from t3;
pg_column_compression
-----------------------
pglz
lz4
(2 rows)That's not good, because now
Yeah, that's really bad.
...because now I hit send too soon. Also, because now column b has
implicit dependencies on both compression AMs and the rest of the
system has no idea. I think we probably should have a rule that
nothing except pglz is allowed inside of a record, just to keep it
simple. The record overall can be toasted so it's not clear why we
should also be toasting the original columns at all, but I think
precedent probably argues for continuing to allow PGLZ, as it can
already be like that on disk and pg_upgrade is a thing. The same kind
of issue probably exists for arrays and range types.
While constructing a row type from the tuple we flatten the external
data I think that would be the place to decompress the data if they
are not compressed with PGLZ. For array-type, we are already
detoasting/decompressing the source attribute see "construct_md_array"
so the array type doesn't have this problem. I haven't yet checked
the range type. Based on my analysis it appeared that the different
data types are getting constructed at different paths so maybe we
should find some centralized place or we need to make some function
call in all such places so that we can decompress the attribute if
required before forming the tuple for the composite type.
I have quickly hacked the code and after that, your test case is working fine.
postgres[55841]=# select pg_column_compression((t3.x).b) from t3;
pg_column_compression
-----------------------
pglz
(2 rows)
-> now the attribute 'b' inside the second tuple is decompressed
(because it was not compressed with PGLZ) so the compression method of
b is NULL
postgres[55841]=# select pg_column_compression((t3.x)) from t3;
pg_column_compression
-----------------------
pglz
(2 rows)
--> but the second row itself is compressed back using the local
compression method of t3
W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even
attempt to detoast if there is no external field in the tuple, in POC
I have got rid of that check, but I think we might need to do better.
Maybe we can add a flag in infomask to detect whether the tuple has
any compressed data or not as we have for detecting the external data
(HEAP_HASEXTERNAL).
So I will do some more analysis in this area and try to come up with a
clean solution.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
POC_fix_compression_in_rowtype.patchapplication/octet-stream; name=POC_fix_compression_in_rowtype.patchDownload
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..0aeea55 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -587,11 +587,28 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
if (!isnull[i] && TupleDescAttr(tupleDesc, i)->attlen == -1)
{
struct varlena *new_value;
+ bool changed = false;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
+
if (VARATT_IS_EXTERNAL(new_value))
{
new_value = detoast_external_attr(new_value);
+ changed = true;
+ }
+
+ if (VARATT_IS_COMPRESSED(new_value) &&
+ VARFLAGS_4B_C(new_value) != PGLZ_COMPRESSION_ID)
+ {
+ struct varlena *tmp = new_value;
+
+ new_value = detoast_attr(tmp);
+ if (changed)
+ pfree(tmp);
+ changed = true;
+ }
+ if (changed)
+ {
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..f115464 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
Re: [HACKERS] Custom compression methods
On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even
attempt to detoast if there is no external field in the tuple, in POC
I have got rid of that check, but I think we might need to do better.
Maybe we can add a flag in infomask to detect whether the tuple has
any compressed data or not as we have for detecting the external data
(HEAP_HASEXTERNAL).
No. This feature isn't close to being important enough to justify
consuming an infomask bit.
I don't really see why we need it anyway. If array construction
already categorically detoasts, why can't we do the same thing here?
Would it really cost that much? In what case? Having compressed values
in a record we're going to store on disk actually seems like a pretty
dumb idea. We might end up trying to recompress something parts of
which have already been compressed.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Feb 11, 2021 at 8:17 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even
attempt to detoast if there is no external field in the tuple, in POC
I have got rid of that check, but I think we might need to do better.
Maybe we can add a flag in infomask to detect whether the tuple has
any compressed data or not as we have for detecting the external data
(HEAP_HASEXTERNAL).No. This feature isn't close to being important enough to justify
consuming an infomask bit.
Okay,
I don't really see why we need it anyway. If array construction
already categorically detoasts, why can't we do the same thing here?
Would it really cost that much? In what case? Having compressed values
in a record we're going to store on disk actually seems like a pretty
dumb idea. We might end up trying to recompress something parts of
which have already been compressed.
If we refer the comments atop function "toast_flatten_tuple_to_datum"
---------------
* We have a general rule that Datums of container types (rows, arrays,
* ranges, etc) must not contain any external TOAST pointers. Without
* this rule, we'd have to look inside each Datum when preparing a tuple
* for storage, which would be expensive and would fail to extend cleanly
* to new sorts of container types.
*
* However, we don't want to say that tuples represented as HeapTuples
* can't contain toasted fields, so instead this routine should be called
* when such a HeapTuple is being converted into a Datum.
*
* While we're at it, we decompress any compressed fields too. This is not
* necessary for correctness, but reflects an expectation that compression
* will be more effective if applied to the whole tuple not individual
* fields. We are not so concerned about that that we want to deconstruct
* and reconstruct tuples just to get rid of compressed fields, however.
* So callers typically won't call this unless they see that the tuple has
* at least one external field.
----------------
It appears that the general rule we want to follow is that while
creating the composite type we want to flatten any external pointer,
but while doing that we also decompress any compressed field with the
assumption that compressing the whole row/array will be a better idea
instead of keeping them compressed individually. However, if there
are no external toast pointers then we don't want to make an effort to
just decompress the compressed data.
Having said that I don't think this rule is followed throughout the
code for example
1. "ExecEvalRow" is calling HeapTupleHeaderGetDatum only if there is
any external field and which is calling "toast_flatten_tuple_to_datum"
so this is following the rule.
2. "ExecEvalWholeRowVar" is calling "toast_build_flattened_tuple", but
this is just flattening the external toast pointer but not doing
anything to the compressed data.
3. "ExecEvalArrayExpr" is calling "construct_md_array", which will
detoast the attribute if attlen is -1, so this will decompress any
compressed data even though there is no external toast pointer.
So in 1 we are following the rule but in 2 and 3 we are not.
IMHO, for the composite data types we should make common a rule and we
should follow that everywhere. As you said it will be good if we can
always detoast any external/compressed data, that will help in getting
better compression as well as fetching the data will be faster because
we can avoid multi level detoasting/decompression. I will analyse
this further and post a patch for the same.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Feb 10, 2021 at 04:56:17PM -0500, Robert Haas wrote:
Small delta patch with a few other suggested changes attached.
Robert's fixup patch caused the CI to fail, since it 1) was called *.patch;
and, 2) didn't include the previous patches.
This includes a couple proposals of mine as separate patches.
. psql: Add HIDE_COMPRESSAM for regress testing; I'm proposing this as part of
0001 (I excluded the reression test changes from 0001 which would've been
reverted by this patch).
. Add default_toast_compression GUC with ideas from Robert; I'm proposing
something like this as part of 0001. Actually, the full complexity with syscache
lookups and invalidation aren't needed in 0001, but are needed in the
"Custom compression" patch.
. default --enable-lz4 (this not meant to be merged so you could leave this as
a separate patch)
I'm not including the whole patch series, since this patch is failing
pg_restore tests:
|Create custom compression methods
--
Justin
Attachments:
v24-0001-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From 28b99449da0942f945e3762b86ac977a15076932 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v24 1/7] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++
configure.ac | 17 ++
contrib/test_decoding/expected/ddl.out | 50 ++---
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 +++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 103 ++++++---
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++---
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++
.../access/compression/compress_pglz.c | 138 ++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++-
src/backend/executor/nodeModifyTable.c | 122 +++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 28 ++-
src/include/access/compressamapi.h | 99 +++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 19 +-
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 ++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 ++++++++++++++++
src/test/regress/expected/psql.out | 68 +++---
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 85 ++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
63 files changed, 1771 insertions(+), 148 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36999..0895b2f326 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8563,6 +8566,39 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13320,6 +13406,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 07da84d401..63940b78a0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/test_decoding/expected/ddl.out b/contrib/test_decoding/expected/ddl.out
index 4ff0044c78..6ee077678d 100644
--- a/contrib/test_decoding/expected/ddl.out
+++ b/contrib/test_decoding/expected/ddl.out
@@ -438,12 +438,12 @@ CREATE TABLE replication_metadata (
WITH (user_catalog_table = true)
;
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -452,12 +452,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('foo', ARRAY['a', 'b']);
ALTER TABLE replication_metadata RESET (user_catalog_table);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
@@ -465,12 +465,12 @@ INSERT INTO replication_metadata(relation, options)
VALUES ('bar', ARRAY['a', 'b']);
ALTER TABLE replication_metadata SET (user_catalog_table = true);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=true
@@ -483,13 +483,13 @@ ALTER TABLE replication_metadata ALTER COLUMN rewritemeornot TYPE text;
ERROR: cannot rewrite table "replication_metadata" used as a catalog table
ALTER TABLE replication_metadata SET (user_catalog_table = false);
\d+ replication_metadata
- Table "public.replication_metadata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
-----------------+---------+-----------+----------+--------------------------------------------------+----------+--------------+-------------
- id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | |
- relation | name | | not null | | plain | |
- options | text[] | | | | extended | |
- rewritemeornot | integer | | | | plain | |
+ Table "public.replication_metadata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+----------------+---------+-----------+----------+--------------------------------------------------+----------+-------------+--------------+-------------
+ id | integer | | not null | nextval('replication_metadata_id_seq'::regclass) | plain | | |
+ relation | name | | not null | | plain | | |
+ options | text[] | | | | extended | pglz | |
+ rewritemeornot | integer | | | | plain | | |
Indexes:
"replication_metadata_pkey" PRIMARY KEY, btree (id)
Options: user_catalog_table=false
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a4625cc..4d7ed698b9 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from parent table's column, however a different compression method can be
+ set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..51a7a977a5 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..b78d49167b 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,78 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Just fetch the toast compress header to know the compression method
+ * in the compressed data.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +543,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..b04c5a5eb8 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..779f54e785
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..1856cf7df7
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Get maximum size of the compressed data that lz4 compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress data using lz4 routine */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress partial data using lz4 routine */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..8a4bf427cf
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,138 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Get maximum size of the compressed data that pglz compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..f5bb37b972 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1f55..b53b6b50e6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1514937748..d1d6bdf470 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..3ad4a61739 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..1d17dc0d6b 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -581,6 +582,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..713fc3fceb 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -486,6 +487,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..dd81d5bf4e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba43e3..f77deee399 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,113 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then we need to copy the
+ * values/null array from oldslot to the new slot and materialize the new
+ * slot.
+ */
+ if (decompressed_any)
+ {
+ TupleTableSlot *newslot = *outslot;
+
+ /*
+ * If the called has passed an invalid slot then create a new slot.
+ * Otherwise, just clear the existing tuple from the slot. This slot
+ * should be stored by the caller so that it can be reused for
+ * decompressing the subsequent tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+ ExecStoreVirtualTuple(newslot);
+ ExecMaterializeSlot(newslot);
+ ExecClearTuple(slot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2353,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +2994,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18ecb..1338e04409 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf6e8..0605ef3f84 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9fc3c..52d92df25d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..cf4413da64 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..a35abe5f1a 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5299,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30a79..7332f93a25 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7eb4..46044cb92a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213fb06..1789e18f46 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e46464a..97791f8dbe 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..ba464d463e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..5a8e23d926
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..6cdc37542d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..31ff91a09c 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,22 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..605684408c 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86faef8..65079f3821 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..797e78b17c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4e0c9be58c..4a13844ce6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f53b..306aabf6c1 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540c94..e5aea8a240 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363d54..6495162a33 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index b6a88ff76b..18e8ecfd90 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1182,6 +1182,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, incase the target attribute's
+ * compression method doesn't match with the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489c23..20d6f96f62 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a2ca..19d2ba26bf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d2bf..53d378b67d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..667927fd7c 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..167878e78b
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM off
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ x | text | | | | extended | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM on
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..329e7881b0
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM off
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM on
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..14f3fa3fc8 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e491..e9c0fc9760 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416fd80..4d5577bae3 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..450416ecb4
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,85 @@
+\set HIDE_COMPRESSAM off
+
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM on
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b2c9..c64b369047 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3adb3..46cac11d39 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
v24-0002-psql-Add-HIDE_COMPRESSAM-for-regress-testing.patchtext/x-diff; charset=us-asciiDownload
From cc36d0fdd8bbed4887d902ba592a15e29b72c44e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 20:48:51 -0600
Subject: [PATCH v24 2/7] psql: Add HIDE_COMPRESSAM for regress testing
This avoids churn in regression output, and allows testing with alternate
compression AMs.
---
doc/src/sgml/ref/psql-ref.sgml | 11 +++++++++++
src/bin/psql/describe.c | 1 +
src/bin/psql/help.c | 2 ++
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 +++++++++
src/test/regress/pg_regress_main.c | 4 ++--
6 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edfa4d..66dcb1b33b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3863,6 +3863,17 @@ bar
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index ba464d463e..b835b0cf76 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1897,6 +1897,7 @@ describeOneTableDetails(const char *schemaname,
/* compresssion info */
if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
(tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
tableinfo.relkind == RELKIND_MATVIEW))
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120bf76..a818ee5503 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d65990059d..9755e8eac6 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..554b64367d 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1159,6 +1159,12 @@ show_context_hook(const char *newval)
return true;
}
+static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941c24..07fd3f6a4d 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
--
2.17.0
v24-0003-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From bfec3954dee86ffc2c4cbc7a69dd79489309901c Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH v24 3/7] Add default_toast_compression GUC
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 3 +-
src/backend/commands/amcmds.c | 143 +++++++++++++++++++++++++++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/init/postinit.c | 4 +
src/backend/utils/misc/guc.c | 12 +++
src/include/access/amapi.h | 2 +
src/include/access/compressamapi.h | 12 ++-
8 files changed, 176 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab487..7afaea000b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidOid;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451eaa71..ec3376cf8a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 3ad4a61739..1682afd2a4 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -13,8 +13,11 @@
*/
#include "postgres.h"
+#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
@@ -27,13 +30,20 @@
#include "parser/parse_func.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+#include "utils/guc.h"
+#include "utils/inval.h"
#include "utils/rel.h"
#include "utils/syscache.h"
-
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION; // maybe needs pg_dump support ?
+/* Invalid means need to lookup the text value in the catalog */
+static Oid default_toast_compression_oid = InvalidOid;
+
+static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue);
/*
* CreateAccessMethod
@@ -277,3 +287,134 @@ lookup_am_handler_func(List *handler_name, char amtype)
return handlerOid;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/*
+ * assign_default_toast_compression: GUC assign_hook for default_toast_compression
+ */
+void
+assign_default_toast_compression(const char *newval, void *extra)
+{
+ /*
+ * Invalidate setting, forcing it to be looked up as needed.
+ * This avoids trying to do database access during GUC initialization,
+ * or outside a transaction.
+ */
+
+ default_toast_compression = NULL;
+fprintf(stderr, "set to null\n");
+}
+
+
+/*
+ * GetDefaultToastCompression -- get the OID of the current toast compression
+ *
+ * This exists to hide and optimize the use of the default_toast_compression
+ * GUC variable.
+ */
+Oid
+GetDefaultToastCompression(void)
+{
+ /* Avoid catalog access during bootstrap, and for default compression */
+ if (strcmp(default_toast_compression, DEFAULT_TOAST_COMPRESSION) == 0)
+ return PGLZ_COMPRESSION_AM_OID;
+
+ /* Cannot call get_compression_am_oid this early */
+ // if (IsBootstrapProcessingMode())
+ // return PGLZ_COMPRESSION_AM_OID;
+ Assert(!IsBootstrapProcessingMode());
+
+ /*
+ * If cached value isn't valid, look up the current default value, caching
+ * the result
+ */
+ if (!OidIsValid(default_toast_compression_oid))
+ default_toast_compression_oid =
+ get_am_type_oid(default_toast_compression, AMTYPE_COMPRESSION,
+ false);
+
+ return default_toast_compression_oid;
+}
+
+/*
+ * InitializeAccessMethods: initialize module during InitPostgres.
+ *
+ * This is called after we are up enough to be able to do catalog lookups.
+ */
+void
+InitializeAccessMethods(void)
+{
+ if (IsBootstrapProcessingMode())
+ return;
+
+ /*
+ * In normal mode, arrange for a callback on any syscache invalidation
+ * of pg_am rows.
+ */
+ CacheRegisterSyscacheCallback(AMOID,
+ AccessMethodCallback,
+ (Datum) 0);
+ /* Force cached default access method to be recomputed on next use */
+ // default_toast_compression_oid = InvalidOid;
+}
+
+/*
+ * AccessMethodCallback
+ * Syscache inval callback function
+ */
+static void
+AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue)
+{
+ /* Force look up of compression oid on next use */
+ default_toast_compression_oid = false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dd81d5bf4e..72ba017814 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidOid;
@@ -17762,7 +17762,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return GetDefaultToastCompression();
amoid = get_compression_am_oid(compression, false);
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc517..6a02bbd377 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -25,6 +25,7 @@
#include "access/session.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/amapi.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -1060,6 +1061,9 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
/* set default namespace search path */
InitializeSearchPath();
+ /* set callback for changes to pg_am */
+ InitializeAccessMethods();
+
/* initialize client encoding */
InitializeClientEncoding();
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index eafdb1118e..3a2b33fcd1 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, assign_default_toast_compression, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb559..1513cafcf4 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -287,4 +287,6 @@ typedef struct IndexAmRoutine
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
+void InitializeAccessMethods(void);
+
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 5a8e23d926..d75a8e9df2 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +30,17 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* Use default compression method if it is not specified. */
+/* Default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+
+/* GUC */
+extern char *default_toast_compression;
+extern void assign_default_toast_compression(const char *newval, void *extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+extern Oid GetDefaultToastCompression(void);
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
--
2.17.0
v24-0004-default-to-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From f9b0b50c0b68eb5bc4299747ca0c7d144345cdd9 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v24 4/7] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure.ac | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/configure.ac b/configure.ac
index 63940b78a0..1fb0f07323 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(without, lz4, no, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
2.17.0
v24-0005-fixups.patch.patchtext/x-diff; charset=us-asciiDownload
From 5bd3d2b7b19ec40d70d04476a72b765e6a2c2b8c Mon Sep 17 00:00:00 2001
From: Robert Haas <robertmhaas@gmail.com>
Date: Wed, 10 Feb 2021 16:56:17 -0500
Subject: [PATCH v24 5/7] fixups.patch
---
doc/src/sgml/ddl.sgml | 3 ---
doc/src/sgml/ref/create_table.sgml | 15 +++++++++++----
src/backend/access/common/detoast.c | 5 ++---
src/backend/access/compression/compress_lz4.c | 10 +++++-----
src/backend/executor/nodeModifyTable.c | 2 +-
src/include/nodes/execnodes.h | 4 ++--
6 files changed, 21 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 4d7ed698b9..1e9a4625cc 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,9 +3762,6 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
- By default, each column in a partition inherits the compression method
- from parent table's column, however a different compression method can be
- set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 51a7a977a5..a4b297340f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -997,10 +997,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
- <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>. The default
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
is <literal>pglz</literal>.
</para>
</listitem>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index b78d49167b..95d5b1c12a 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -477,8 +477,8 @@ toast_get_compression_oid(struct varlena *attr)
return InvalidOid;
/*
- * Just fetch the toast compress header to know the compression method
- * in the compressed data.
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
*/
attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
}
@@ -503,7 +503,6 @@ toast_get_compression_handler(struct varlena *attr)
cmid = TOAST_COMPRESS_METHOD(attr);
- /* Get the handler routines for the compression method */
switch (cmid)
{
case PGLZ_COMPRESSION_ID:
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 1856cf7df7..3079eff7eb 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -23,7 +23,7 @@
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
- * Compresses source into dest using the default strategy. Returns the
+ * Compresses source into dest using the LZ4 defaults. Returns the
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
@@ -42,8 +42,8 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
/*
- * Get maximum size of the compressed data that lz4 compression may output
- * and allocate the memory for holding the compressed data and the header.
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
*/
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
@@ -83,10 +83,10 @@ lz4_cmdecompress(const struct varlena *value)
int32 rawsize;
struct varlena *result;
- /* allocate memory for holding the uncompressed data */
+ /* allocate memory for the uncompressed data */
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- /* decompress data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index f77deee399..920d9dd0d5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2117,7 +2117,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot *newslot = *outslot;
/*
- * If the called has passed an invalid slot then create a new slot.
+ * If the caller has passed an invalid slot then create a new slot.
* Otherwise, just clear the existing tuple from the slot. This slot
* should be stored by the caller so that it can be reused for
* decompressing the subsequent tuples.
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18e8ecfd90..d8483c36b3 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1183,8 +1183,8 @@ typedef struct ModifyTableState
TupleTableSlot *mt_root_tuple_slot;
/*
- * Slot for storing the modified tuple, incase the target attribute's
- * compression method doesn't match with the source table.
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
*/
TupleTableSlot *mt_decompress_tuple_slot;
--
2.17.0
v24-0006-alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From b0788c502b5f7a6733935ec7b81e7fb717741f80 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v24 6/7] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v24-0007-Add-support-for-PRESERVE.patchtext/x-diff; charset=iso-8859-1Download
From 34436a06ec53febcf7bf49ca7e1a82f21a5bc55a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 11:49:23 +0530
Subject: [PATCH v24 7/7] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. Â For
supporting this the column will maintain the dependency with all
the supported compression methods. Â So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. Â If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 300 ++++++++++++++++++++
src/backend/commands/tablecmds.c | 126 ++++----
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 101 +++++++
src/bin/pg_dump/pg_dump.h | 15 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/compression_1.out | 39 ++-
src/test/regress/expected/create_index.out | 56 ++--
src/test/regress/sql/compression.sql | 9 +
22 files changed, 739 insertions(+), 108 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 0bd0c1a503..c9f443a59c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -387,7 +387,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
@@ -395,6 +395,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da24322d..dd376484b7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..fd6db24e7f
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,300 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribute.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return GetDefaultToastCompression();
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+#ifndef HAVE_LIBLZ4
+ if (cmoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode, just create a dependency on all preserved
+ * methods.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p)));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 586a92f0c1..2a1841c353 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +937,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2415,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression) != 0)
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4908,7 +4926,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6414,7 +6433,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6589,6 +6609,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6736,6 +6757,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11867,7 +11910,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11982,6 +12026,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15086,24 +15135,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15136,11 +15182,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17865,42 +17916,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * resolve column compression specification to an OID.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
- Oid amoid;
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression == NULL)
- return InvalidOid;
-
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return GetDefaultToastCompression();
-
- amoid = get_compression_am_oid(compression, false);
-
-#ifndef HAVE_LIBLZ4
- if (amoid == LZ4_COMPRESSION_AM_OID)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("not built with lz4 support")));
-#endif
- return amoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ea82a05591..90d092671e 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -44,6 +44,7 @@
#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2068,8 +2069,8 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop over all the attributes in the tuple and check if any attribute is
- * compressed and its compression method is not same as the target
- * atrribute's compression method then decompress it.
+ * compressed and its compression method is not is not supported by the
+ * target attribute then we need to decompress
*/
for (i = 0; i < natts; i++)
{
@@ -2094,12 +2095,13 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
DatumGetPointer(slot->tts_values[attnum - 1]);
/*
- * Get the compression method Oid stored in the toast header and
- * compare it with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
cmoid = toast_get_compression_oid(new_value);
if (OidIsValid(cmoid) &&
- targetTupDesc->attrs[i].attcompression != cmoid)
+ !IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1338e04409..6a11f8eb60 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,7 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2986,6 +2986,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5675,6 +5687,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f3592003da..26a9b85974 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,7 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2619,6 +2619,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3724,6 +3734,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0605ef3f84..b584a58ba3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,7 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2881,6 +2881,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4258,6 +4268,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 30acfe615d..9eb2b04d58 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,7 +596,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2309,12 +2311,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3437,7 +3439,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3492,13 +3494,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index cf4413da64..45f4724a13 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 46044cb92a..7bf345a4ac 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9037,6 +9037,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16343,6 +16417,33 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1789e18f46..a829528cd0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,7 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all
+ compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -356,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ffa8d05edf..869fd3676d 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2103,6 +2103,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e5aea8a240..bd53f9bb0f 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index b3d30acc35..e6c98e65d4 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -97,5 +97,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 20d6f96f62..24deaad253 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -481,6 +481,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f9a87dee02..ce0913e18a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 21c1b451d2..3ed33b6534 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -226,12 +226,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 64c5855bf7..36a5f8ba5e 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -205,12 +205,49 @@ SELECT pg_column_compression(f1) FROM cmpart;
ERROR: relation "cmpart" does not exist
LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
^
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..8f984510ac 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b9daa33b74..5774b55d82 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -96,6 +96,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
Re: [HACKERS] Custom compression methods
On Sun, Feb 14, 2021 at 12:49:40PM -0600, Justin Pryzby wrote:
On Wed, Feb 10, 2021 at 04:56:17PM -0500, Robert Haas wrote:
Small delta patch with a few other suggested changes attached.
Robert's fixup patch caused the CI to fail, since it 1) was called *.patch;
and, 2) didn't include the previous patches.This includes a couple proposals of mine as separate patches.
CIs failed on BSD and linux due to a test in contrib/, but others passed.
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.127551
https://cirrus-ci.com/task/6087701947482112
https://cirrus-ci.com/task/6650651900903424
https://cirrus-ci.com/task/5524751994060800
Resending with fixes to configure.ac and missed autoconf run. I think this is
expected to fail on mac, due to missing LZ4.
BTW, compressamapi.h doesn't need to be included in any of these, at least in
the 0001 patch:
src/backend/access/common/indextuple.c | 2 +-
src/backend/catalog/heap.c | 2 +-
src/backend/catalog/index.c | 2 +-
src/backend/parser/parse_utilcmd.c | 2 +-
It's pretty unfriendly that this requires quoting the integer to be
syntactically valid:
|postgres=# create table j(q text compression pglz with (level 1) );
|2021-01-30 01:26:33.554 CST [31814] ERROR: syntax error at or near "1" at character 52
|2021-01-30 01:26:33.554 CST [31814] STATEMENT: create table j(q text compression pglz with (level 1) );
|ERROR: syntax error at or near "1"
|LINE 1: create table j(q text compression pglz with (level 1) );
Attachments:
v24-0001-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From ee4c361801dea3c60e16a3675b24fa677b6bf6b7 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v24 01/10] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++
configure.ac | 17 ++
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 +++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 103 ++++++---
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++---
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++
.../access/compression/compress_pglz.c | 138 ++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++-
src/backend/executor/nodeModifyTable.c | 122 +++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 28 ++-
src/include/access/compressamapi.h | 99 +++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 19 +-
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 ++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 ++++++++++++++++
src/test/regress/expected/psql.out | 68 +++---
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 85 ++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
62 files changed, 1746 insertions(+), 123 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36999..0895b2f326 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8563,6 +8566,39 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13320,6 +13406,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 07da84d401..63940b78a0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a4625cc..4d7ed698b9 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from parent table's column, however a different compression method can be
+ set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 569f4c9da7..51a7a977a5 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..b78d49167b 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,78 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Just fetch the toast compress header to know the compression method
+ * in the compressed data.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +543,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..b04c5a5eb8 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..779f54e785
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..1856cf7df7
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Get maximum size of the compressed data that lz4 compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress data using lz4 routine */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress partial data using lz4 routine */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..8a4bf427cf
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,138 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Get maximum size of the compressed data that pglz compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..f5bb37b972 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1f55..b53b6b50e6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 1514937748..d1d6bdf470 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..3ad4a61739 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..1d17dc0d6b 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -581,6 +582,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..713fc3fceb 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -486,6 +487,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 420991e315..dd81d5bf4e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba43e3..f77deee399 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,113 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then we need to copy the
+ * values/null array from oldslot to the new slot and materialize the new
+ * slot.
+ */
+ if (decompressed_any)
+ {
+ TupleTableSlot *newslot = *outslot;
+
+ /*
+ * If the called has passed an invalid slot then create a new slot.
+ * Otherwise, just clear the existing tuple from the slot. This slot
+ * should be stored by the caller so that it can be reused for
+ * decompressing the subsequent tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+ ExecStoreVirtualTuple(newslot);
+ ExecMaterializeSlot(newslot);
+ ExecClearTuple(slot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2353,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +2994,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18ecb..1338e04409 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf6e8..0605ef3f84 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9fc3c..52d92df25d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index b31f3afa03..cf4413da64 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..a35abe5f1a 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5299,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30a79..7332f93a25 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7eb4..46044cb92a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213fb06..1789e18f46 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e46464a..97791f8dbe 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..ba464d463e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..5a8e23d926
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..6cdc37542d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..31ff91a09c 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,22 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..605684408c 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86faef8..65079f3821 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..797e78b17c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4e0c9be58c..4a13844ce6 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f53b..306aabf6c1 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540c94..e5aea8a240 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363d54..6495162a33 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index b6a88ff76b..18e8ecfd90 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1182,6 +1182,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, incase the target attribute's
+ * compression method doesn't match with the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489c23..20d6f96f62 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a2ca..19d2ba26bf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d2bf..53d378b67d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..667927fd7c 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..167878e78b
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM off
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ x | text | | | | extended | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM on
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..329e7881b0
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM off
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM on
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..14f3fa3fc8 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e491..e9c0fc9760 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416fd80..4d5577bae3 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..450416ecb4
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,85 @@
+\set HIDE_COMPRESSAM off
+
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM on
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b2c9..c64b369047 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3adb3..46cac11d39 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
v24-0002-psql-Add-HIDE_COMPRESSAM-for-regress-testing.patchtext/x-diff; charset=us-asciiDownload
From 06b853ab1afa359b3dfc35dcba130f08e955a620 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 20:48:51 -0600
Subject: [PATCH v24 02/10] psql: Add HIDE_COMPRESSAM for regress testing
This avoids churn in regression output, and allows testing with alternate
compression AMs.
---
doc/src/sgml/ref/psql-ref.sgml | 11 +++++++++++
src/bin/psql/describe.c | 1 +
src/bin/psql/help.c | 2 ++
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 +++++++++
src/test/regress/pg_regress_main.c | 4 ++--
6 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edfa4d..66dcb1b33b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3863,6 +3863,17 @@ bar
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index ba464d463e..b835b0cf76 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1897,6 +1897,7 @@ describeOneTableDetails(const char *schemaname,
/* compresssion info */
if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
(tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
tableinfo.relkind == RELKIND_MATVIEW))
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120bf76..a818ee5503 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d65990059d..9755e8eac6 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..554b64367d 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1159,6 +1159,12 @@ show_context_hook(const char *newval)
return true;
}
+static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941c24..07fd3f6a4d 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
--
2.17.0
v24-0003-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From e5054fb34179baa9c06f370d95c8dc2d06daeac2 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH v24 03/10] Add default_toast_compression GUC
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 3 +-
src/backend/commands/amcmds.c | 143 +++++++++++++++++++++++++++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/init/postinit.c | 4 +
src/backend/utils/misc/guc.c | 12 +++
src/include/access/amapi.h | 2 +
src/include/access/compressamapi.h | 12 ++-
8 files changed, 176 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab487..7afaea000b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidOid;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451eaa71..ec3376cf8a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 3ad4a61739..1682afd2a4 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -13,8 +13,11 @@
*/
#include "postgres.h"
+#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
@@ -27,13 +30,20 @@
#include "parser/parse_func.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+#include "utils/guc.h"
+#include "utils/inval.h"
#include "utils/rel.h"
#include "utils/syscache.h"
-
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION; // maybe needs pg_dump support ?
+/* Invalid means need to lookup the text value in the catalog */
+static Oid default_toast_compression_oid = InvalidOid;
+
+static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue);
/*
* CreateAccessMethod
@@ -277,3 +287,134 @@ lookup_am_handler_func(List *handler_name, char amtype)
return handlerOid;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/*
+ * assign_default_toast_compression: GUC assign_hook for default_toast_compression
+ */
+void
+assign_default_toast_compression(const char *newval, void *extra)
+{
+ /*
+ * Invalidate setting, forcing it to be looked up as needed.
+ * This avoids trying to do database access during GUC initialization,
+ * or outside a transaction.
+ */
+
+ default_toast_compression = NULL;
+fprintf(stderr, "set to null\n");
+}
+
+
+/*
+ * GetDefaultToastCompression -- get the OID of the current toast compression
+ *
+ * This exists to hide and optimize the use of the default_toast_compression
+ * GUC variable.
+ */
+Oid
+GetDefaultToastCompression(void)
+{
+ /* Avoid catalog access during bootstrap, and for default compression */
+ if (strcmp(default_toast_compression, DEFAULT_TOAST_COMPRESSION) == 0)
+ return PGLZ_COMPRESSION_AM_OID;
+
+ /* Cannot call get_compression_am_oid this early */
+ // if (IsBootstrapProcessingMode())
+ // return PGLZ_COMPRESSION_AM_OID;
+ Assert(!IsBootstrapProcessingMode());
+
+ /*
+ * If cached value isn't valid, look up the current default value, caching
+ * the result
+ */
+ if (!OidIsValid(default_toast_compression_oid))
+ default_toast_compression_oid =
+ get_am_type_oid(default_toast_compression, AMTYPE_COMPRESSION,
+ false);
+
+ return default_toast_compression_oid;
+}
+
+/*
+ * InitializeAccessMethods: initialize module during InitPostgres.
+ *
+ * This is called after we are up enough to be able to do catalog lookups.
+ */
+void
+InitializeAccessMethods(void)
+{
+ if (IsBootstrapProcessingMode())
+ return;
+
+ /*
+ * In normal mode, arrange for a callback on any syscache invalidation
+ * of pg_am rows.
+ */
+ CacheRegisterSyscacheCallback(AMOID,
+ AccessMethodCallback,
+ (Datum) 0);
+ /* Force cached default access method to be recomputed on next use */
+ // default_toast_compression_oid = InvalidOid;
+}
+
+/*
+ * AccessMethodCallback
+ * Syscache inval callback function
+ */
+static void
+AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue)
+{
+ /* Force look up of compression oid on next use */
+ default_toast_compression_oid = false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index dd81d5bf4e..72ba017814 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidOid;
@@ -17762,7 +17762,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return GetDefaultToastCompression();
amoid = get_compression_am_oid(compression, false);
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc517..6a02bbd377 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -25,6 +25,7 @@
#include "access/session.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/amapi.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -1060,6 +1061,9 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
/* set default namespace search path */
InitializeSearchPath();
+ /* set callback for changes to pg_am */
+ InitializeAccessMethods();
+
/* initialize client encoding */
InitializeClientEncoding();
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index eafdb1118e..3a2b33fcd1 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, assign_default_toast_compression, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb559..1513cafcf4 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -287,4 +287,6 @@ typedef struct IndexAmRoutine
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
+void InitializeAccessMethods(void);
+
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 5a8e23d926..d75a8e9df2 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +30,17 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* Use default compression method if it is not specified. */
+/* Default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+
+/* GUC */
+extern char *default_toast_compression;
+extern void assign_default_toast_compression(const char *newval, void *extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+extern Oid GetDefaultToastCompression(void);
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
--
2.17.0
v24-0004-default-to-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From 80068f5f033529334f1dd63cc82346a86ca1fd7e Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v24 04/10] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f326..79e9d226ff 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b78a0..62180c5424 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
2.17.0
v24-0005-fixups.patch.patchtext/x-diff; charset=us-asciiDownload
From b1b684ce962c24431a927274774a5e06f513f5e7 Mon Sep 17 00:00:00 2001
From: Robert Haas <robertmhaas@gmail.com>
Date: Wed, 10 Feb 2021 16:56:17 -0500
Subject: [PATCH v24 05/10] fixups.patch
---
doc/src/sgml/ddl.sgml | 3 ---
doc/src/sgml/ref/create_table.sgml | 15 +++++++++++----
src/backend/access/common/detoast.c | 5 ++---
src/backend/access/compression/compress_lz4.c | 10 +++++-----
src/backend/executor/nodeModifyTable.c | 2 +-
src/include/nodes/execnodes.h | 4 ++--
6 files changed, 21 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 4d7ed698b9..1e9a4625cc 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,9 +3762,6 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
- By default, each column in a partition inherits the compression method
- from parent table's column, however a different compression method can be
- set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 51a7a977a5..a4b297340f 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -997,10 +997,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
- <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>. The default
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
is <literal>pglz</literal>.
</para>
</listitem>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index b78d49167b..95d5b1c12a 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -477,8 +477,8 @@ toast_get_compression_oid(struct varlena *attr)
return InvalidOid;
/*
- * Just fetch the toast compress header to know the compression method
- * in the compressed data.
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
*/
attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
}
@@ -503,7 +503,6 @@ toast_get_compression_handler(struct varlena *attr)
cmid = TOAST_COMPRESS_METHOD(attr);
- /* Get the handler routines for the compression method */
switch (cmid)
{
case PGLZ_COMPRESSION_ID:
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 1856cf7df7..3079eff7eb 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -23,7 +23,7 @@
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
- * Compresses source into dest using the default strategy. Returns the
+ * Compresses source into dest using the LZ4 defaults. Returns the
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
@@ -42,8 +42,8 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
/*
- * Get maximum size of the compressed data that lz4 compression may output
- * and allocate the memory for holding the compressed data and the header.
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
*/
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
@@ -83,10 +83,10 @@ lz4_cmdecompress(const struct varlena *value)
int32 rawsize;
struct varlena *result;
- /* allocate memory for holding the uncompressed data */
+ /* allocate memory for the uncompressed data */
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- /* decompress data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index f77deee399..920d9dd0d5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2117,7 +2117,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot *newslot = *outslot;
/*
- * If the called has passed an invalid slot then create a new slot.
+ * If the caller has passed an invalid slot then create a new slot.
* Otherwise, just clear the existing tuple from the slot. This slot
* should be stored by the caller so that it can be reused for
* decompressing the subsequent tuples.
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 18e8ecfd90..d8483c36b3 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1183,8 +1183,8 @@ typedef struct ModifyTableState
TupleTableSlot *mt_root_tuple_slot;
/*
- * Slot for storing the modified tuple, incase the target attribute's
- * compression method doesn't match with the source table.
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
*/
TupleTableSlot *mt_decompress_tuple_slot;
--
2.17.0
v24-0006-alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From 216f3bb0f488559cf23146438371c43acd40c01c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH v24 06/10] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 72ba017814..586a92f0c1 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 1e1c315bae..ffa8d05edf 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2098,7 +2098,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v24-0007-Add-support-for-PRESERVE.patchtext/x-diff; charset=iso-8859-1Download
From 30bcf9b82ea030e623dca771d5f503fe367bcf3f Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 11:49:23 +0530
Subject: [PATCH v24 07/10] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. Â For
supporting this the column will maintain the dependency with all
the supported compression methods. Â So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. Â If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 300 ++++++++++++++++++++
src/backend/commands/tablecmds.c | 126 ++++----
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 101 +++++++
src/bin/pg_dump/pg_dump.h | 15 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/compression_1.out | 39 ++-
src/test/regress/expected/create_index.out | 56 ++--
src/test/regress/sql/compression.sql | 9 +
22 files changed, 739 insertions(+), 108 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 0bd0c1a503..c9f443a59c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -387,7 +387,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
@@ -395,6 +395,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da24322d..dd376484b7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..fd6db24e7f
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,300 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribute.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return GetDefaultToastCompression();
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+#ifndef HAVE_LIBLZ4
+ if (cmoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode, just create a dependency on all preserved
+ * methods.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p)));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 586a92f0c1..2a1841c353 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +937,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2415,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression) != 0)
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4908,7 +4926,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6414,7 +6433,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6589,6 +6609,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6736,6 +6757,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11867,7 +11910,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11982,6 +12026,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15086,24 +15135,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15136,11 +15182,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17865,42 +17916,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * resolve column compression specification to an OID.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
- Oid amoid;
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression == NULL)
- return InvalidOid;
-
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return GetDefaultToastCompression();
-
- amoid = get_compression_am_oid(compression, false);
-
-#ifndef HAVE_LIBLZ4
- if (amoid == LZ4_COMPRESSION_AM_OID)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("not built with lz4 support")));
-#endif
- return amoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ea82a05591..90d092671e 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -44,6 +44,7 @@
#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2068,8 +2069,8 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop over all the attributes in the tuple and check if any attribute is
- * compressed and its compression method is not same as the target
- * atrribute's compression method then decompress it.
+ * compressed and its compression method is not is not supported by the
+ * target attribute then we need to decompress
*/
for (i = 0; i < natts; i++)
{
@@ -2094,12 +2095,13 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
DatumGetPointer(slot->tts_values[attnum - 1]);
/*
- * Get the compression method Oid stored in the toast header and
- * compare it with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
cmoid = toast_get_compression_oid(new_value);
if (OidIsValid(cmoid) &&
- targetTupDesc->attrs[i].attcompression != cmoid)
+ !IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1338e04409..6a11f8eb60 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,7 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2986,6 +2986,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5675,6 +5687,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f3592003da..26a9b85974 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,7 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2619,6 +2619,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3724,6 +3734,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0605ef3f84..b584a58ba3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,7 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2881,6 +2881,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4258,6 +4268,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 30acfe615d..9eb2b04d58 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,7 +596,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2309,12 +2311,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3437,7 +3439,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3492,13 +3494,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index cf4413da64..45f4724a13 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 46044cb92a..7bf345a4ac 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9037,6 +9037,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16343,6 +16417,33 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1789e18f46..a829528cd0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,7 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all
+ compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -356,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ffa8d05edf..869fd3676d 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2103,6 +2103,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e5aea8a240..bd53f9bb0f 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index b3d30acc35..e6c98e65d4 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -97,5 +97,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 20d6f96f62..24deaad253 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -481,6 +481,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f9a87dee02..ce0913e18a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 21c1b451d2..3ed33b6534 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -226,12 +226,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 64c5855bf7..36a5f8ba5e 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -205,12 +205,49 @@ SELECT pg_column_compression(f1) FROM cmpart;
ERROR: relation "cmpart" does not exist
LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
^
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..8f984510ac 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b9daa33b74..5774b55d82 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -96,6 +96,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v24-0008-Create-custom-compression-methods.patchtext/x-diff; charset=us-asciiDownload
From cb57f7985774d87d16401a94e7801a8f09396cbf Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 16:27:04 +0530
Subject: [PATCH v24 08/10] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
XXX: this is failing pg_dump tests
---
doc/src/sgml/ref/alter_table.sgml | 6 ++-
doc/src/sgml/ref/create_access_method.sgml | 12 +++--
doc/src/sgml/ref/create_table.sgml | 9 ++--
src/backend/access/common/detoast.c | 51 ++++++++++++++++---
src/backend/access/common/toast_internals.c | 19 +++++--
src/backend/access/compression/compress_lz4.c | 21 ++++----
.../access/compression/compress_pglz.c | 20 ++++----
src/backend/access/index/amapi.c | 51 ++++++++++++++-----
src/backend/commands/amcmds.c | 22 +++++++-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 ++++
src/bin/pg_dump/pg_dump.c | 8 +++
src/bin/psql/tab-complete.c | 6 +++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 45 ++++++++++++++--
src/include/access/toast_internals.h | 16 ++++++
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 ++
src/include/postgres.h | 8 +++
src/test/regress/expected/compression.out | 40 ++++++++++++++-
src/test/regress/sql/compression.sql | 10 ++++
21 files changed, 304 insertions(+), 58 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c9f443a59c..49c43df1c1 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -391,8 +391,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ This sets the compression method for a column. The Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The supported buit-in
+ compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
The <literal>PRESERVE</literal> list contains a list of compression
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..c5ef8b738d 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,8 +61,8 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
- are supported at present.
+ Currently, <literal>TABLE</literal>, <literal>INDEX</literal> and
+ <literal>COMPRESSION</literal> access methods are supported.
</para>
</listitem>
</varlistentry>
@@ -77,11 +77,13 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
+ is described in <xref linkend="tableam"/>, the index access method
API is described in <xref linkend="indexam"/>.
</para>
</listitem>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index a4b297340f..196352ef42 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -1004,11 +1004,14 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
column storage types.) Setting this property for a partitioned table
has no direct effect, because such tables have no storage of their own,
but the configured value is inherited by newly-created partitions.
- The supported compression methods are <literal>pglz</literal> and
+ The supported built-in compression methods are <literal>pglz</literal> and
<literal>lz4</literal>. <literal>lz4</literal> is available only if
<literal>--with-lz4</literal> was used when building
- <productname>PostgreSQL</productname>. The default
- is <literal>pglz</literal>.
+ <productname>PostgreSQL</productname>.
+ Compression methods can be created with <xref
+ linkend="sql-create-access-method"/> or it can be set to any available
+ compression method.
+ The default is <literal>pglz</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 95d5b1c12a..fd7c42c95e 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -462,10 +462,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*
* Returns the Oid of the compression method stored in the compressed data. If
* the varlena is not compressed then returns InvalidOid.
+ *
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
*/
Oid
toast_get_compression_oid(struct varlena *attr)
{
+ CompressionId cmid;
+
if (VARATT_IS_EXTERNAL_ONDISK(attr))
{
struct varatt_external toast_pointer;
@@ -485,7 +492,21 @@ toast_get_compression_oid(struct varlena *attr)
else if (!VARATT_IS_COMPRESSED(attr))
return InvalidOid;
- return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the built-in
+ * compression id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
}
/* ----------
@@ -494,7 +515,7 @@ toast_get_compression_oid(struct varlena *attr)
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -507,10 +528,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "invalid compression method id %d", cmid);
}
@@ -526,9 +558,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -542,16 +576,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index b04c5a5eb8..a3539065d3 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 3079eff7eb..2a3c162836 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -46,10 +46,10 @@ lz4_cmcompress(const struct varlena *value)
* that will be needed for varlena overhead, and allocate that amount.
*/
max_size = LZ4_compressBound(valsize);
- tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + VARHDRSZ_COMPRESS,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
elog(ERROR, "could not compress data with lz4");
@@ -61,7 +61,7 @@ lz4_cmcompress(const struct varlena *value)
return NULL;
}
- SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -73,7 +73,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -87,9 +87,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
/* decompress the data */
- rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARSIZE(value) - header_size,
VARRAWSIZE_4B_C(value));
if (rawsize < 0)
ereport(ERROR,
@@ -109,7 +109,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -123,9 +124,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
/* decompress partial data using lz4 routine */
- rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARSIZE(value) - header_size,
slicelength,
VARRAWSIZE_4B_C(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 8a4bf427cf..7f6e7429fe 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -26,7 +26,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -47,11 +47,11 @@ pglz_cmcompress(const struct varlena *value)
* and allocate the memory for holding the compressed data and the header.
*/
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- VARHDRSZ_COMPRESS);
+ header_size);
len = pglz_compress(VARDATA_ANY(value),
valsize,
- (char *) tmp + VARHDRSZ_COMPRESS,
+ (char *) tmp + header_size,
NULL);
if (len < 0)
{
@@ -59,7 +59,7 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
}
- SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -70,15 +70,15 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
VARRAWSIZE_4B_C(value), true);
if (rawsize < 0)
@@ -97,7 +97,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -105,8 +105,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
if (rawsize < 0)
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43514..16f8fab359 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
@@ -46,14 +47,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +65,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +94,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 1682afd2a4..0ac86eee02 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -45,6 +45,9 @@ static Oid default_toast_compression_oid = InvalidOid;
static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
+
/*
* CreateAccessMethod
* Registers a new access method.
@@ -93,7 +96,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* amoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -237,6 +252,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -274,6 +291,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9eb2b04d58..b22ce818cd 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5305,6 +5305,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95079..f026104c01 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -128,6 +128,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7bf345a4ac..88fa7e1ed3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13136,6 +13136,11 @@ dumpAccessMethod(Archive *fout, const AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -13146,6 +13151,9 @@ dumpAccessMethod(Archive *fout, const AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 869fd3676d..8ca2b832a6 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 1513cafcf4..bc1e2437f0 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -285,6 +285,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
void InitializeAccessMethods(void);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index d75a8e9df2..bac7b47cfc 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -15,10 +15,13 @@
#include "postgres.h"
+#include "fmgr.h"
+#include "access/amapi.h"
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
#include "utils/guc.h"
+
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
@@ -27,7 +30,9 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Default compression method if not specified. */
@@ -41,13 +46,19 @@ extern bool check_default_toast_compression(char **newval, void **extra, GucSour
extern Oid GetDefaultToastCompression(void);
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -106,4 +117,30 @@ CompressionIdToOid(CompressionId cmid)
}
}
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+static inline CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 31ff91a09c..ac28f9ed55 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,10 +27,23 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
do { \
@@ -38,6 +51,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7ac5..9d65bae238 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4a13844ce6..64c9c6e538 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10777,6 +10777,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '2137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 667927fd7c..ede3b11ef3 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3ed33b6534..e52e272d39 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -260,13 +260,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 5774b55d82..9d2e72b784 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
v24-0009-new-compression-method-extension-for-zlib.patchtext/x-diff; charset=us-asciiDownload
From 1ecfe0c77335e893b1072cbbf725a1cae3ab902c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH v24 09/10] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index f27e458482..9e452d8dd0 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..2b6fac7e0b
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ pglz
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..ea8d206625
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.17.0
v24-0010-Support-compression-methods-options.patchtext/x-diff; charset=us-asciiDownload
From 88b0b4773afafb50173453fd243687b2b4a21d52 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 16:29:55 +0530
Subject: [PATCH v24 10/10] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 +++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 6 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 ++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 75 ++++++++++-
.../access/compression/compress_pglz.c | 116 +++++++++++++++---
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 +++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++-
src/backend/commands/foreigncmds.c | 44 -------
src/backend/commands/tablecmds.c | 102 ++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 23 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/include/access/compressamapi.h | 18 ++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 ++++++++
src/test/regress/expected/compression_1.out | 55 +++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 +++
36 files changed, 728 insertions(+), 141 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..d8c2865f21 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 49c43df1c1..0f861e78f9 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -396,7 +396,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
it can be set to any available compression method. The supported buit-in
compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>.
+ was used when building <productname>PostgreSQL</productname>. If the
+ compression method has options they can be specified with the <literal>WITH
+ </literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 196352ef42..8529782d79 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
The <literal>COMPRESSION</literal> clause sets the compression method
@@ -1011,6 +1011,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
Compression methods can be created with <xref
linkend="sql-create-access-method"/> or it can be set to any available
compression method.
+ If the compression method has options they can be specified with the
+ <literal>WITH </literal> parameter.
The default is <literal>pglz</literal>.
</para>
</listitem>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712c71..76f824bc6f 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d2ff..0d3307e94f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3ee9e..2dda6c038b 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a3539065d3..5ed3312f39 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 2a3c162836..7c422fc534 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -17,9 +17,73 @@
#endif
#include "access/compressamapi.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ /*
+ * We don't need to check the range for the acceleration parameter
+ * because the LZ4_compress_fast will automatically replace the
+ * values <=0 with LZ4_ACCELERATION_DEFAULT (currently == 1). And,
+ * Values > LZ4_ACCELERATION_MAX with LZ4_ACCELERATION_MAX
+ * (currently == 65537c).
+ */
+ if (strcmp(def->defname, "acceleration") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+#endif
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+#endif
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -27,7 +91,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -37,6 +101,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -48,9 +113,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
elog(ERROR, "could not compress data with lz4");
@@ -146,6 +211,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
*/
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 7f6e7429fe..1ca912831b 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -14,11 +14,93 @@
#include "postgres.h"
#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -26,42 +108,41 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
- /*
- * Get maximum size of the compressed data that pglz compression may output
- * and allocate the memory for holding the compressed data and the header.
- */
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
header_size);
- len = pglz_compress(VARDATA_ANY(value),
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
- if (len < 0)
+ strategy);
+
+ if (len >= 0)
{
- pfree(tmp);
- return NULL;
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
}
- SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ pfree(tmp);
- return tmp;
+ return NULL;
}
/*
@@ -125,10 +206,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
*/
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9c3e..c9d44c62fa 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004e1b..a43e0e197c 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b53b6b50e6..18b04816f7 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -741,6 +741,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -796,6 +797,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -843,6 +849,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -861,7 +868,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -893,7 +901,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1160,6 +1168,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1418,7 +1427,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index d1d6bdf470..f5441ce24a 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -108,10 +108,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -273,7 +275,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -334,6 +337,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -350,6 +356,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -490,7 +513,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -508,7 +531,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -721,6 +745,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -876,6 +901,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -884,7 +911,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -975,7 +1003,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481557..348b7d3a37 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 096a06f7b3..0ad946d802 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -699,6 +699,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index fd6db24e7f..160d64ad32 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -29,6 +29,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -181,7 +182,7 @@ BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -215,6 +216,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
errmsg("not built with lz4 support")));
#endif
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -286,15 +301,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttribute's compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103fd3b..ae9ae2c51b 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 2a1841c353..afb9f1190f 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -609,6 +609,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -813,6 +814,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -866,8 +869,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -921,8 +925,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2432,16 +2439,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2478,8 +2483,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2729,14 +2733,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6271,6 +6270,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6434,6 +6434,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6444,7 +6445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7840,13 +7841,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* the respective input values are valid.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7882,7 +7890,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
if (newstorage != '\0')
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7973,7 +8003,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15145,9 +15175,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15182,7 +15214,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15192,8 +15225,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15201,8 +15243,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6a11f8eb60..991340e2ad 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2993,6 +2993,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 26a9b85974..57a2975da1 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2624,6 +2624,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b584a58ba3..bc5c64df1e 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2888,6 +2888,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b22ce818cd..00fe3b7a5f 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3503,11 +3504,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3515,14 +3522,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 45f4724a13..3a33bce506 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 88fa7e1ed3..f9517549f9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8709,10 +8709,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8765,6 +8772,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8794,6 +8802,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15959,7 +15968,8 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -16005,6 +16015,10 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
@@ -16436,6 +16450,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, "\nWITH (%s) ", tbinfo->attcmoptions[j]);
+
if (cminfo->nitems > 0)
{
appendPQExpBuffer(q, "\nPRESERVE (");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a829528cd0..da47f3173f 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,7 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
+ char **attcmoptions; /* per-attribute compression options */
struct _attrCompressionInfo **attcompression; /* per-attribute all
compression data */
/*
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index bac7b47cfc..7f3346b2a6 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -51,8 +51,11 @@ extern Oid GetDefaultToastCompression(void);
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -63,14 +66,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc37f3..fe8de405ac 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index ac28f9ed55..864de31739 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -54,7 +54,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b49c..5ecb6f3653 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 797e78b17c..7c74bc314e 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -178,6 +178,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bd53f9bb0f..8271e97dea 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -134,6 +134,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -146,9 +148,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ce0913e18a..c3c878b80e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index e52e272d39..4af72bdbdc 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -297,6 +297,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 36a5f8ba5e..40b0b33202 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -241,6 +241,61 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ERROR: not built with lz4 support
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+HINT: use "pg_column_compression" function for list of compression methods
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 9ebe28a78d..4e3b0c13c1 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -99,6 +99,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -109,7 +110,7 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
-- system catalogs without primary keys
--
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 9d2e72b784..17320c9062 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -115,6 +115,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
Re: [HACKERS] Custom compression methods
On Mon, Feb 15, 2021 at 1:58 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Sun, Feb 14, 2021 at 12:49:40PM -0600, Justin Pryzby wrote:
On Wed, Feb 10, 2021 at 04:56:17PM -0500, Robert Haas wrote:
Small delta patch with a few other suggested changes attached.
Robert's fixup patch caused the CI to fail, since it 1) was called *.patch;
and, 2) didn't include the previous patches.This includes a couple proposals of mine as separate patches.
CIs failed on BSD and linux due to a test in contrib/, but others passed.
https://ci.appveyor.com/project/postgresql-cfbot/postgresql/build/1.0.127551
https://cirrus-ci.com/task/6087701947482112
https://cirrus-ci.com/task/6650651900903424
https://cirrus-ci.com/task/5524751994060800Resending with fixes to configure.ac and missed autoconf run. I think this is
expected to fail on mac, due to missing LZ4.BTW, compressamapi.h doesn't need to be included in any of these, at least in
the 0001 patch:src/backend/access/common/indextuple.c | 2 +-
src/backend/catalog/heap.c | 2 +-
src/backend/catalog/index.c | 2 +-
src/backend/parser/parse_utilcmd.c | 2 +-It's pretty unfriendly that this requires quoting the integer to be
syntactically valid:|postgres=# create table j(q text compression pglz with (level 1) );
|2021-01-30 01:26:33.554 CST [31814] ERROR: syntax error at or near "1" at character 52
|2021-01-30 01:26:33.554 CST [31814] STATEMENT: create table j(q text compression pglz with (level 1) );
|ERROR: syntax error at or near "1"
|LINE 1: create table j(q text compression pglz with (level 1) );
Thanks for the review and patch for HIDE_COMPRESSAM, I will merge
this into the main patch. And work on other comments after fixing the
issue related to compressed data in composite types.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Feb 13, 2021 at 8:14 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Feb 11, 2021 at 8:17 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Feb 11, 2021 at 7:36 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
W.R.T the attached patch, In HeapTupleHeaderGetDatum, we don't even
attempt to detoast if there is no external field in the tuple, in POC
I have got rid of that check, but I think we might need to do better.
Maybe we can add a flag in infomask to detect whether the tuple has
any compressed data or not as we have for detecting the external data
(HEAP_HASEXTERNAL).No. This feature isn't close to being important enough to justify
consuming an infomask bit.Okay,
I don't really see why we need it anyway. If array construction
already categorically detoasts, why can't we do the same thing here?
Would it really cost that much? In what case? Having compressed values
in a record we're going to store on disk actually seems like a pretty
dumb idea. We might end up trying to recompress something parts of
which have already been compressed.If we refer the comments atop function "toast_flatten_tuple_to_datum"
---------------
* We have a general rule that Datums of container types (rows, arrays,
* ranges, etc) must not contain any external TOAST pointers. Without
* this rule, we'd have to look inside each Datum when preparing a tuple
* for storage, which would be expensive and would fail to extend cleanly
* to new sorts of container types.
*
* However, we don't want to say that tuples represented as HeapTuples
* can't contain toasted fields, so instead this routine should be called
* when such a HeapTuple is being converted into a Datum.
*
* While we're at it, we decompress any compressed fields too. This is not
* necessary for correctness, but reflects an expectation that compression
* will be more effective if applied to the whole tuple not individual
* fields. We are not so concerned about that that we want to deconstruct
* and reconstruct tuples just to get rid of compressed fields, however.
* So callers typically won't call this unless they see that the tuple has
* at least one external field.
----------------It appears that the general rule we want to follow is that while
creating the composite type we want to flatten any external pointer,
but while doing that we also decompress any compressed field with the
assumption that compressing the whole row/array will be a better idea
instead of keeping them compressed individually. However, if there
are no external toast pointers then we don't want to make an effort to
just decompress the compressed data.Having said that I don't think this rule is followed throughout the
code for example1. "ExecEvalRow" is calling HeapTupleHeaderGetDatum only if there is
any external field and which is calling "toast_flatten_tuple_to_datum"
so this is following the rule.
2. "ExecEvalWholeRowVar" is calling "toast_build_flattened_tuple", but
this is just flattening the external toast pointer but not doing
anything to the compressed data.
3. "ExecEvalArrayExpr" is calling "construct_md_array", which will
detoast the attribute if attlen is -1, so this will decompress any
compressed data even though there is no external toast pointer.So in 1 we are following the rule but in 2 and 3 we are not.
IMHO, for the composite data types we should make common a rule and we
should follow that everywhere. As you said it will be good if we can
always detoast any external/compressed data, that will help in getting
better compression as well as fetching the data will be faster because
we can avoid multi level detoasting/decompression. I will analyse
this further and post a patch for the same.
I have done further analysis of this issue and came up with the
attached patch. So with this patch, like external toast posiners we
will not allow any compressed data also in the composite types. The
problem is that now we will be processing all the tuple while forming
the composite type irrespective of the source of the tuple, I mean if
user is directly inserting into the array type and not selecting from
another table then there will not be any compressed data so now
checking each field of tuple for compressed data is unnecessary but I
am not sure how to distinguish between those cases.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v1-0001-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v1-0001-Disallow-compressed-data-inside-container-types.patchDownload
From 59a8520a5278c43f712315035ed2261d83e310ca Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 16 Feb 2021 19:24:38 +0530
Subject: [PATCH v1] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
src/backend/access/common/heaptuple.c | 28 ++++++----------------------
src/backend/access/heap/heaptoast.c | 16 +++++++++-------
src/backend/executor/execTuples.c | 11 -----------
src/include/access/heaptoast.h | 4 ++--
4 files changed, 17 insertions(+), 42 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 24a27e3..5c3194f 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,30 +983,14 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
- /*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
- */
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
- tuple->t_len,
- tupleDesc);
-
/*
- * Fast path for easy case: just make a palloc'd copy and insert the
- * correct composite-Datum header fields (since those may not be set if
- * the given tuple came from disk, rather than from heap_form_tuple).
+ * The tuple contains compressed/external TOAST pointers, so we have
+ * to inline those fields to meet the conventions for composite-type
+ * Datums.
*/
- td = (HeapTupleHeader) palloc(tuple->t_len);
- memcpy((char *) td, (char *) tuple->t_data, tuple->t_len);
-
- HeapTupleHeaderSetDatumLength(td, tuple->t_len);
- HeapTupleHeaderSetTypeId(td, tupleDesc->tdtypeid);
- HeapTupleHeaderSetTypMod(td, tupleDesc->tdtypmod);
-
- return PointerGetDatum(td);
+ return toast_flatten_tuple_to_datum(tuple->t_data,
+ tuple->t_len,
+ tupleDesc);
}
/*
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..dd162da 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -549,14 +549,15 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
/* ----------
* toast_build_flattened_tuple -
*
- * Build a tuple containing no out-of-line toasted fields.
- * (This does not eliminate compressed or short-header datums.)
+ * Build a tuple containing no compressed/out-of-line toasted fields.
+ * (This does not eliminate short-header datums.)
*
* This is essentially just like heap_form_tuple, except that it will
- * expand any external-data pointers beforehand.
+ * expand any compressed/external-data pointers beforehand.
*
- * It's not very clear whether it would be preferable to decompress
- * in-line compressed datums while at it. For now, we don't.
+ * It is not necessary to decompress the compressed data for the
+ * correctness, but reflects an expectation that compression will be more
+ * effective if applied to the whole tuple not individual fields.
* ----------
*/
HeapTuple
@@ -589,9 +590,10 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) ||
+ VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..ca7fbed 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2189,13 +2189,6 @@ BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values)
* memory context. Beware of code that changes context between the initial
* heap_form_tuple/etc call and calling HeapTuple(Header)GetDatum.
*
- * For performance-critical callers, it could be worthwhile to take extra
- * steps to ensure that there aren't TOAST pointers in the output of
- * heap_form_tuple to begin with. It's likely however that the costs of the
- * typcache lookup and tuple disassembly/reassembly are swamped by TOAST
- * dereference costs, so that the benefits of such extra effort would be
- * minimal.
- *
* XXX it would likely be better to create wrapper functions that produce
* a composite Datum from the field values in one step. However, there's
* enough code using the existing APIs that we couldn't get rid of this
@@ -2207,10 +2200,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/include/access/heaptoast.h b/src/include/access/heaptoast.h
index 8b29f1a..c80af45 100644
--- a/src/include/access/heaptoast.h
+++ b/src/include/access/heaptoast.h
@@ -128,8 +128,8 @@ extern Datum toast_flatten_tuple_to_datum(HeapTupleHeader tup,
/* ----------
* toast_build_flattened_tuple -
*
- * Build a tuple containing no out-of-line toasted fields.
- * (This does not eliminate compressed or short-header datums.)
+ * Build a tuple containing no compressed/out-of-line toasted fields.
+ * (This does not eliminate short-header datums.)
* ----------
*/
extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
--
1.8.3.1
Re: [HACKERS] Custom compression methods
@cfbot: Resending with fixes to regression tests.
I'm hoping to see check-world pass --with-lz4 on an environment other than my
own PC.
Attachments:
0001-Disallow-compressed-data-inside-container-types.patchtext/x-diff; charset=us-asciiDownload
From 0fa26749845a94b1dc437c57f698b4612c931c8b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Tue, 16 Feb 2021 19:24:38 +0530
Subject: [PATCH 01/12] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
src/backend/access/common/heaptuple.c | 28 ++++++---------------------
src/backend/access/heap/heaptoast.c | 16 ++++++++-------
src/backend/executor/execTuples.c | 11 -----------
src/include/access/heaptoast.h | 4 ++--
4 files changed, 17 insertions(+), 42 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 24a27e387d..5c3194f96d 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,30 +983,14 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
- /*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
- */
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
- tuple->t_len,
- tupleDesc);
-
/*
- * Fast path for easy case: just make a palloc'd copy and insert the
- * correct composite-Datum header fields (since those may not be set if
- * the given tuple came from disk, rather than from heap_form_tuple).
+ * The tuple contains compressed/external TOAST pointers, so we have
+ * to inline those fields to meet the conventions for composite-type
+ * Datums.
*/
- td = (HeapTupleHeader) palloc(tuple->t_len);
- memcpy((char *) td, (char *) tuple->t_data, tuple->t_len);
-
- HeapTupleHeaderSetDatumLength(td, tuple->t_len);
- HeapTupleHeaderSetTypeId(td, tupleDesc->tdtypeid);
- HeapTupleHeaderSetTypMod(td, tupleDesc->tdtypmod);
-
- return PointerGetDatum(td);
+ return toast_flatten_tuple_to_datum(tuple->t_data,
+ tuple->t_len,
+ tupleDesc);
}
/*
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d584..dd162daab9 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -549,14 +549,15 @@ toast_flatten_tuple_to_datum(HeapTupleHeader tup,
/* ----------
* toast_build_flattened_tuple -
*
- * Build a tuple containing no out-of-line toasted fields.
- * (This does not eliminate compressed or short-header datums.)
+ * Build a tuple containing no compressed/out-of-line toasted fields.
+ * (This does not eliminate short-header datums.)
*
* This is essentially just like heap_form_tuple, except that it will
- * expand any external-data pointers beforehand.
+ * expand any compressed/external-data pointers beforehand.
*
- * It's not very clear whether it would be preferable to decompress
- * in-line compressed datums while at it. For now, we don't.
+ * It is not necessary to decompress the compressed data for the
+ * correctness, but reflects an expectation that compression will be more
+ * effective if applied to the whole tuple not individual fields.
* ----------
*/
HeapTuple
@@ -589,9 +590,10 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) ||
+ VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df9c9..ca7fbed576 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2189,13 +2189,6 @@ BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values)
* memory context. Beware of code that changes context between the initial
* heap_form_tuple/etc call and calling HeapTuple(Header)GetDatum.
*
- * For performance-critical callers, it could be worthwhile to take extra
- * steps to ensure that there aren't TOAST pointers in the output of
- * heap_form_tuple to begin with. It's likely however that the costs of the
- * typcache lookup and tuple disassembly/reassembly are swamped by TOAST
- * dereference costs, so that the benefits of such extra effort would be
- * minimal.
- *
* XXX it would likely be better to create wrapper functions that produce
* a composite Datum from the field values in one step. However, there's
* enough code using the existing APIs that we couldn't get rid of this
@@ -2207,10 +2200,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/include/access/heaptoast.h b/src/include/access/heaptoast.h
index 8b29f1a986..c80af45012 100644
--- a/src/include/access/heaptoast.h
+++ b/src/include/access/heaptoast.h
@@ -128,8 +128,8 @@ extern Datum toast_flatten_tuple_to_datum(HeapTupleHeader tup,
/* ----------
* toast_build_flattened_tuple -
*
- * Build a tuple containing no out-of-line toasted fields.
- * (This does not eliminate compressed or short-header datums.)
+ * Build a tuple containing no compressed/out-of-line toasted fields.
+ * (This does not eliminate short-header datums.)
* ----------
*/
extern HeapTuple toast_build_flattened_tuple(TupleDesc tupleDesc,
--
2.17.0
0002-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From 174c64fc8c40331fee571bd4b568a72e0f67c610 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH 02/12] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++
configure.ac | 17 ++
doc/src/sgml/ddl.sgml | 3 +
doc/src/sgml/ref/create_table.sgml | 25 +++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 103 ++++++---
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 +++---
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 ++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++
.../access/compression/compress_pglz.c | 138 ++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 +
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++-
src/backend/executor/nodeModifyTable.c | 122 +++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 28 ++-
src/include/access/compressamapi.h | 99 +++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 19 +-
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 ++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 ++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 ++++++++++++++++
src/test/regress/expected/psql.out | 68 +++---
src/test/regress/parallel_schedule | 2 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 85 ++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
62 files changed, 1746 insertions(+), 123 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36999..0895b2f326 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8563,6 +8566,39 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13320,6 +13406,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 07da84d401..63940b78a0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 1e9a4625cc..4d7ed698b9 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,6 +3762,9 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
+ By default, each column in a partition inherits the compression method
+ from parent table's column, however a different compression method can be
+ set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227683..3374012940 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,6 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -605,6 +606,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
@@ -981,6 +993,19 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="sql-createtable-exclude">
<term><literal>EXCLUDE [ USING <replaceable class="parameter">index_method</replaceable> ] ( <replaceable class="parameter">exclude_element</replaceable> WITH <replaceable class="parameter">operator</replaceable> [, ... ] ) <replaceable class="parameter">index_parameters</replaceable> [ WHERE ( <replaceable class="parameter">predicate</replaceable> ) ]</literal></term>
<listitem>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a8bb..ba08bdd63e 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..b78d49167b 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,78 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Just fetch the toast compress header to know the compression method
+ * in the compressed data.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /* Get the handler routines for the compression method */
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- return result;
+ return cmroutine;
+}
+
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +543,16 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
-
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * If the handler supports the slice decompression then decompress the
+ * slice otherwise decompress complete data.
+ */
+ if (cmroutine->datum_decompress_slice)
+ return cmroutine->datum_decompress_slice(attr, slicelength);
+ else
+ return cmroutine->datum_decompress(attr);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1d43d5d2ff 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..b04c5a5eb8 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..ca26fab487 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000000..779f54e785
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile--
+# Makefile for access/compression
+#
+# IDENTIFICATION
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000000..1856cf7df7
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Get maximum size of the compressed data that lz4 compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress data using lz4 routine */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for holding the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress partial data using lz4 routine */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000000..8a4bf427cf
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,138 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Get maximum size of the compressed data that pglz compression may output
+ * and allocate the memory for holding the compressed data and the header.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..9b451eaa71 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..f5bb37b972 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1f55..b53b6b50e6 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4ab0b88ad..0db2a7430f 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..a549481557 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535ed0..3ad4a61739 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -175,6 +175,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
return get_am_type_oid(amname, AMTYPE_TABLE, missing_ok);
}
+/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce882012e..1d17dc0d6b 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -581,6 +582,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
/* Nothing to insert if WITH NO DATA is specified. */
if (!myState->into->skipData)
{
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce11d..713fc3fceb 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -486,6 +487,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
{
DR_transientrel *myState = (DR_transientrel *) self;
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
/*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b2457a6924..811dbc3741 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba43e3..f77deee399 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,113 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ new_value = detoast_attr(new_value);
+ slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then we need to copy the
+ * values/null array from oldslot to the new slot and materialize the new
+ * slot.
+ */
+ if (decompressed_any)
+ {
+ TupleTableSlot *newslot = *outslot;
+
+ /*
+ * If the called has passed an invalid slot then create a new slot.
+ * Otherwise, just clear the existing tuple from the slot. This slot
+ * should be stored by the caller so that it can be reused for
+ * decompressing the subsequent tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+ ExecStoreVirtualTuple(newslot);
+ ExecMaterializeSlot(newslot);
+ ExecClearTuple(slot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2353,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +2994,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18ecb..1338e04409 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf6e8..0605ef3f84 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9fc3c..52d92df25d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266caeb4..e262ec5bcb 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d606..fe133c76c2 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..a35abe5f1a 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5299,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30a79..7332f93a25 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7eb4..46044cb92a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213fb06..1789e18f46 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e46464a..97791f8dbe 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..ba464d463e 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,20 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2035,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2116,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000000..5a8e23d926
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Portions Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..6cdc37542d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..dca0bc37f3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..31ff91a09c 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,22 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 info; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e6a8..605684408c 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86faef8..65079f3821 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..797e78b17c 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710d59..4b3d34ae1a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f53b..306aabf6c1 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540c94..e5aea8a240 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363d54..6495162a33 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 943931f65d..ac3884b3a8 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, incase the target attribute's
+ * compression method doesn't match with the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489c23..20d6f96f62 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a2ca..19d2ba26bf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d2bf..53d378b67d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..667927fd7c 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_info in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_info & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_info >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..167878e78b
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM off
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ x | text | | | | extended | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM on
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..329e7881b0
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM off
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
+--------+------+-----------+----------+---------+----------+--------------+-------------
+ f1 | text | | | | extended | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM on
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb0b4..14f3fa3fc8 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e491..e9c0fc9760 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416fd80..4d5577bae3 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..450416ecb4
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,85 @@
+\set HIDE_COMPRESSAM off
+
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM on
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b2c9..c64b369047 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3adb3..46cac11d39 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
0003-psql-Add-HIDE_COMPRESSAM-for-regress-testing.patchtext/x-diff; charset=us-asciiDownload
From 61b674c345ab155e3edcedebd77f7380f22aacca Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 20:48:51 -0600
Subject: [PATCH 03/12] psql: Add HIDE_COMPRESSAM for regress testing
This avoids churn in regression output, and allows testing with alternate
compression AMs.
---
doc/src/sgml/ref/psql-ref.sgml | 11 +++++++++++
src/bin/psql/describe.c | 1 +
src/bin/psql/help.c | 2 ++
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 +++++++++
src/test/regress/pg_regress_main.c | 4 ++--
6 files changed, 26 insertions(+), 2 deletions(-)
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edfa4d..66dcb1b33b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3863,6 +3863,17 @@ bar
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index ba464d463e..b835b0cf76 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1897,6 +1897,7 @@ describeOneTableDetails(const char *schemaname,
/* compresssion info */
if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
(tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
tableinfo.relkind == RELKIND_MATVIEW))
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120bf76..a818ee5503 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d65990059d..9755e8eac6 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..554b64367d 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1159,6 +1159,12 @@ show_context_hook(const char *newval)
return true;
}
+static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941c24..07fd3f6a4d 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
--
2.17.0
0004-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From 570692f03eea1c28fa2b097533aa6a0c5c251237 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH 04/12] Add default_toast_compression GUC
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 3 +-
src/backend/commands/amcmds.c | 143 +++++++++++++++++++++++++++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/init/postinit.c | 4 +
src/backend/utils/misc/guc.c | 12 +++
src/include/access/amapi.h | 2 +
src/include/access/compressamapi.h | 12 ++-
8 files changed, 176 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab487..7afaea000b 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidOid;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451eaa71..ec3376cf8a 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 3ad4a61739..1682afd2a4 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -13,8 +13,11 @@
*/
#include "postgres.h"
+#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
@@ -27,13 +30,20 @@
#include "parser/parse_func.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+#include "utils/guc.h"
+#include "utils/inval.h"
#include "utils/rel.h"
#include "utils/syscache.h"
-
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION; // maybe needs pg_dump support ?
+/* Invalid means need to lookup the text value in the catalog */
+static Oid default_toast_compression_oid = InvalidOid;
+
+static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue);
/*
* CreateAccessMethod
@@ -277,3 +287,134 @@ lookup_am_handler_func(List *handler_name, char amtype)
return handlerOid;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/*
+ * assign_default_toast_compression: GUC assign_hook for default_toast_compression
+ */
+void
+assign_default_toast_compression(const char *newval, void *extra)
+{
+ /*
+ * Invalidate setting, forcing it to be looked up as needed.
+ * This avoids trying to do database access during GUC initialization,
+ * or outside a transaction.
+ */
+
+ default_toast_compression = NULL;
+fprintf(stderr, "set to null\n");
+}
+
+
+/*
+ * GetDefaultToastCompression -- get the OID of the current toast compression
+ *
+ * This exists to hide and optimize the use of the default_toast_compression
+ * GUC variable.
+ */
+Oid
+GetDefaultToastCompression(void)
+{
+ /* Avoid catalog access during bootstrap, and for default compression */
+ if (strcmp(default_toast_compression, DEFAULT_TOAST_COMPRESSION) == 0)
+ return PGLZ_COMPRESSION_AM_OID;
+
+ /* Cannot call get_compression_am_oid this early */
+ // if (IsBootstrapProcessingMode())
+ // return PGLZ_COMPRESSION_AM_OID;
+ Assert(!IsBootstrapProcessingMode());
+
+ /*
+ * If cached value isn't valid, look up the current default value, caching
+ * the result
+ */
+ if (!OidIsValid(default_toast_compression_oid))
+ default_toast_compression_oid =
+ get_am_type_oid(default_toast_compression, AMTYPE_COMPRESSION,
+ false);
+
+ return default_toast_compression_oid;
+}
+
+/*
+ * InitializeAccessMethods: initialize module during InitPostgres.
+ *
+ * This is called after we are up enough to be able to do catalog lookups.
+ */
+void
+InitializeAccessMethods(void)
+{
+ if (IsBootstrapProcessingMode())
+ return;
+
+ /*
+ * In normal mode, arrange for a callback on any syscache invalidation
+ * of pg_am rows.
+ */
+ CacheRegisterSyscacheCallback(AMOID,
+ AccessMethodCallback,
+ (Datum) 0);
+ /* Force cached default access method to be recomputed on next use */
+ // default_toast_compression_oid = InvalidOid;
+}
+
+/*
+ * AccessMethodCallback
+ * Syscache inval callback function
+ */
+static void
+AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue)
+{
+ /* Force look up of compression oid on next use */
+ default_toast_compression_oid = false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 811dbc3741..7adeaedd0e 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidOid;
@@ -17762,7 +17762,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return GetDefaultToastCompression();
amoid = get_compression_am_oid(compression, false);
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc517..6a02bbd377 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -25,6 +25,7 @@
#include "access/session.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/amapi.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -1060,6 +1061,9 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
/* set default namespace search path */
InitializeSearchPath();
+ /* set callback for changes to pg_am */
+ InitializeAccessMethods();
+
/* initialize client encoding */
InitializeClientEncoding();
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 00018abb7d..df2a985900 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, assign_default_toast_compression, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb559..1513cafcf4 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -287,4 +287,6 @@ typedef struct IndexAmRoutine
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
+void InitializeAccessMethods(void);
+
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 5a8e23d926..d75a8e9df2 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +30,17 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* Use default compression method if it is not specified. */
+/* Default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+
+/* GUC */
+extern char *default_toast_compression;
+extern void assign_default_toast_compression(const char *newval, void *extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+extern Oid GetDefaultToastCompression(void);
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
--
2.17.0
0005-default-to-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From 3c48e28acf33864852ba6f9c2e276af2c62e474b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH 05/12] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f326..79e9d226ff 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b78a0..62180c5424 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
2.17.0
0006-fixups.patch.patchtext/x-diff; charset=us-asciiDownload
From 75814aed18a79c6774bf5e522cedc2a0d4f0d263 Mon Sep 17 00:00:00 2001
From: Robert Haas <robertmhaas@gmail.com>
Date: Wed, 10 Feb 2021 16:56:17 -0500
Subject: [PATCH 06/12] fixups.patch
---
doc/src/sgml/ddl.sgml | 3 ---
doc/src/sgml/ref/create_table.sgml | 15 +++++++++++----
src/backend/access/common/detoast.c | 5 ++---
src/backend/access/compression/compress_lz4.c | 10 +++++-----
src/backend/executor/nodeModifyTable.c | 2 +-
src/include/nodes/execnodes.h | 4 ++--
6 files changed, 21 insertions(+), 18 deletions(-)
diff --git a/doc/src/sgml/ddl.sgml b/doc/src/sgml/ddl.sgml
index 4d7ed698b9..1e9a4625cc 100644
--- a/doc/src/sgml/ddl.sgml
+++ b/doc/src/sgml/ddl.sgml
@@ -3762,9 +3762,6 @@ CREATE TABLE measurement (
<productname>PostgreSQL</productname>
tables (or, possibly, foreign tables). It is possible to specify a
tablespace and storage parameters for each partition separately.
- By default, each column in a partition inherits the compression method
- from parent table's column, however a different compression method can be
- set for each partition.
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3374012940..807597e648 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -997,10 +997,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
- <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>. The default
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
is <literal>pglz</literal>.
</para>
</listitem>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index b78d49167b..95d5b1c12a 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -477,8 +477,8 @@ toast_get_compression_oid(struct varlena *attr)
return InvalidOid;
/*
- * Just fetch the toast compress header to know the compression method
- * in the compressed data.
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
*/
attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ_COMPRESS);
}
@@ -503,7 +503,6 @@ toast_get_compression_handler(struct varlena *attr)
cmid = TOAST_COMPRESS_METHOD(attr);
- /* Get the handler routines for the compression method */
switch (cmid)
{
case PGLZ_COMPRESSION_ID:
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 1856cf7df7..3079eff7eb 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -23,7 +23,7 @@
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
- * Compresses source into dest using the default strategy. Returns the
+ * Compresses source into dest using the LZ4 defaults. Returns the
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
@@ -42,8 +42,8 @@ lz4_cmcompress(const struct varlena *value)
valsize = VARSIZE_ANY_EXHDR(value);
/*
- * Get maximum size of the compressed data that lz4 compression may output
- * and allocate the memory for holding the compressed data and the header.
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
*/
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
@@ -83,10 +83,10 @@ lz4_cmdecompress(const struct varlena *value)
int32 rawsize;
struct varlena *result;
- /* allocate memory for holding the uncompressed data */
+ /* allocate memory for the uncompressed data */
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- /* decompress data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index f77deee399..920d9dd0d5 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2117,7 +2117,7 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot *newslot = *outslot;
/*
- * If the called has passed an invalid slot then create a new slot.
+ * If the caller has passed an invalid slot then create a new slot.
* Otherwise, just clear the existing tuple from the slot. This slot
* should be stored by the caller so that it can be reused for
* decompressing the subsequent tuples.
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index ac3884b3a8..182b77a0ff 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1186,8 +1186,8 @@ typedef struct ModifyTableState
TupleTableSlot *mt_root_tuple_slot;
/*
- * Slot for storing the modified tuple, incase the target attribute's
- * compression method doesn't match with the source table.
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
*/
TupleTableSlot *mt_decompress_tuple_slot;
--
2.17.0
0007-alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From 3a19bc428db37772f867de791ae0d92dd59a4866 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 18 Dec 2020 11:31:22 +0530
Subject: [PATCH 07/12] alter table set compression
Add support for changing the compression method associated
with a column, forcing a table rewrite.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 226 +++++++++++++++-----
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/commands/event_trigger.h | 1 +
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 93 ++++++--
src/test/regress/expected/compression_1.out | 56 ++++-
src/test/regress/sql/compression.sql | 21 ++
13 files changed, 370 insertions(+), 75 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index 1d17dc0d6b..748ec29570 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -589,7 +589,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 713fc3fceb..779b4e51cf 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -494,7 +494,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7adeaedd0e..5e24af9cb3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -529,6 +529,8 @@ static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3968,6 +3970,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4495,7 +4498,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4903,6 +4907,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5519,6 +5527,7 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
while (table_scan_getnextslot(scan, ForwardScanDirection, oldslot))
{
+ bool decompressed = false;
TupleTableSlot *insertslot;
if (tab->rewrite > 0)
@@ -5527,11 +5536,25 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ if (tab->rewrite & AT_REWRITE_ALTER_COMPRESSION)
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute then
+ * the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7767,6 +7790,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7782,7 +7866,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7846,47 +7929,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15033,6 +15077,92 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* Prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* Prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* Get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ if (atttableform->attcompression != cmoid)
+ tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 920d9dd0d5..ea82a05591 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2052,7 +2052,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2139,6 +2140,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
*outslot = newslot;
+ if (decompressed != NULL)
+ *decompressed = true;
+
return newslot;
}
@@ -2360,7 +2364,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 52d92df25d..30acfe615d 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b64db82f02..2aedddc1d8 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2107,7 +2107,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/commands/event_trigger.h b/src/include/commands/event_trigger.h
index c11bf2d781..5a314f4c1d 100644
--- a/src/include/commands/event_trigger.h
+++ b/src/include/commands/event_trigger.h
@@ -32,6 +32,7 @@ typedef struct EventTriggerData
#define AT_REWRITE_ALTER_PERSISTENCE 0x01
#define AT_REWRITE_DEFAULT_VAL 0x02
#define AT_REWRITE_COLUMN_REWRITE 0x04
+#define AT_REWRITE_ALTER_COMPRESSION 0x08
/*
* EventTriggerData is the node type that is passed as fmgr "context" info
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 6495162a33..050ef2dcd0 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 167878e78b..21c1b451d2 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -4,20 +4,20 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
\d+ cmdata1
- Table "public.cmdata1"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- try setting compression for incompressible data type
CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
@@ -112,18 +112,18 @@ DROP TABLE cmdata2;
-- test LIKE INCLUDING COMPRESSION
CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
\d+ cmdata2
- Table "public.cmdata2"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
-- test compression with materialized view
CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
\d+ mv
- Materialized view "public.mv"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- x | text | | | | extended | |
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
View definition:
SELECT cmdata1.f1 AS x
FROM cmdata1;
@@ -165,6 +165,67 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 329e7881b0..64c5855bf7 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -4,10 +4,10 @@ CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
INSERT INTO cmdata VALUES(repeat('1234567890',1000));
\d+ cmdata
- Table "public.cmdata"
- Column | Type | Collation | Nullable | Default | Storage | Stats target | Description
---------+------+-----------+----------+---------+----------+--------------+-------------
- f1 | text | | | | extended | |
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
Indexes:
"idx" btree (f1)
@@ -157,6 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmdata1" does not exist
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+REFRESH MATERIALIZED VIEW mv;
+ERROR: relation "mv" does not exist
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart" does not exist
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 450416ecb4..b9daa33b74 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -75,6 +75,27 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method with rewrite
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER TABLE cmdata1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+REFRESH MATERIALIZED VIEW mv;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart ALTER COLUMN f1 SET COMPRESSION pglz;
+SELECT pg_column_compression(f1) FROM cmpart;
+
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0008-Add-support-for-PRESERVE.patchtext/x-diff; charset=iso-8859-1Download
From f6cde260dc4350bb54cb48f078cc1d07eac0bb63 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 11:49:23 +0530
Subject: [PATCH 08/12] Add support for PRESERVE
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Now the compression method can be changed without forcing a table
rewrite, by including the old method in the PRESERVE list. Â For
supporting this the column will maintain the dependency with all
the supported compression methods. Â So whenever the compression
method is altered the dependency is added with the new compression
method and the dependency is removed for all the old compression
methods which are not given in the preserve list. Â If PRESERVE ALL
is given then all the dependency is maintained.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 10 +-
src/backend/catalog/pg_depend.c | 7 +
src/backend/commands/Makefile | 1 +
src/backend/commands/compressioncmds.c | 300 ++++++++++++++++++++
src/backend/commands/tablecmds.c | 126 ++++----
src/backend/executor/nodeModifyTable.c | 12 +-
src/backend/nodes/copyfuncs.c | 17 +-
src/backend/nodes/equalfuncs.c | 15 +-
src/backend/nodes/outfuncs.c | 15 +-
src/backend/parser/gram.y | 52 +++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 101 +++++++
src/bin/pg_dump/pg_dump.h | 15 +-
src/bin/psql/tab-complete.c | 7 +
src/include/commands/defrem.h | 7 +
src/include/commands/tablecmds.h | 2 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 16 +-
src/test/regress/expected/compression.out | 37 ++-
src/test/regress/expected/compression_1.out | 39 ++-
src/test/regress/expected/create_index.out | 56 ++--
src/test/regress/sql/compression.sql | 9 +
22 files changed, 739 insertions(+), 108 deletions(-)
create mode 100644 src/backend/commands/compressioncmds.c
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 0bd0c1a503..c9f443a59c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -387,7 +387,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
<varlistentry>
<term>
- <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]</literal>
</term>
<listitem>
<para>
@@ -395,6 +395,12 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
+ The <literal>PRESERVE</literal> list contains a list of compression
+ methods used in the column and determines which of them may be kept.
+ Without <literal>PRESERVE</literal> or if any of the pre-existing
+ compression methods are not preserved, the table will be rewritten. If
+ <literal>PRESERVE ALL</literal> is specified, then all of the existing
+ methods will be preserved and the table will not be rewritten.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/catalog/pg_depend.c b/src/backend/catalog/pg_depend.c
index 63da24322d..dd376484b7 100644
--- a/src/backend/catalog/pg_depend.c
+++ b/src/backend/catalog/pg_depend.c
@@ -17,6 +17,7 @@
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "catalog/pg_am.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
#include "catalog/pg_collation.h"
@@ -125,6 +126,12 @@ recordMultipleDependencies(const ObjectAddress *depender,
if (referenced->objectId == DEFAULT_COLLATION_OID)
ignore_systempin = true;
}
+ /*
+ * Record the dependency on compression access method for handling
+ * preserve.
+ */
+ if (referenced->classId == AccessMethodRelationId)
+ ignore_systempin = true;
}
else
Assert(!version);
diff --git a/src/backend/commands/Makefile b/src/backend/commands/Makefile
index e8504f0ae4..a7395ad77d 100644
--- a/src/backend/commands/Makefile
+++ b/src/backend/commands/Makefile
@@ -21,6 +21,7 @@ OBJS = \
cluster.o \
collationcmds.o \
comment.o \
+ compressioncmds.o \
constraint.o \
conversioncmds.o \
copy.o \
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
new file mode 100644
index 0000000000..fd6db24e7f
--- /dev/null
+++ b/src/backend/commands/compressioncmds.c
@@ -0,0 +1,300 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressioncmds.c
+ * Routines for SQL commands for attribute compression methods
+ *
+ * Portions Copyright (c) 1996-2017, PostgreSQL Global Development Group
+ * Portions Copyright (c) 1994, Regents of the University of California
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/commands/compressioncmds.c
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "access/heapam.h"
+#include "access/htup_details.h"
+#include "access/reloptions.h"
+#include "catalog/catalog.h"
+#include "catalog/dependency.h"
+#include "catalog/indexing.h"
+#include "catalog/pg_attribute.h"
+#include "catalog/pg_depend.h"
+#include "commands/defrem.h"
+#include "commands/tablecmds.h"
+#include "miscadmin.h"
+#include "nodes/parsenodes.h"
+#include "utils/builtins.h"
+#include "utils/fmgroids.h"
+#include "utils/lsyscache.h"
+
+/*
+ * Get list of all supported compression methods for the given attribute.
+ *
+ * We maintain dependency of the attribute on the pg_am row for the current
+ * compression AM and all the preserved compression AM. So scan pg_depend and
+ * find the column dependency on the pg_am. Collect the list of access method
+ * oids on which this attribute has a dependency.
+ */
+static List *
+lookup_attribute_compression(Oid attrelid, AttrNumber attnum, List *oldcmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+ List *cmoids = NIL;
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (depform->refclassid == AccessMethodRelationId)
+ cmoids = list_append_unique_oid(cmoids, depform->refobjid);
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+
+ return cmoids;
+}
+
+/*
+ * Remove the attribute dependency on the old compression methods
+ *
+ * Scan the pg_depend and search the attribute dependency on the pg_am. Remove
+ * dependency on previous am which is not preserved. The list of non-preserved
+ * AMs is given in cmoids.
+ */
+static void
+remove_old_dependencies(Oid attrelid, AttrNumber attnum, List *cmoids)
+{
+ LOCKMODE lock = AccessShareLock;
+ HeapTuple tup;
+ Relation rel;
+ SysScanDesc scan;
+ ScanKeyData key[3];
+
+ rel = table_open(DependRelationId, lock);
+
+ ScanKeyInit(&key[0],
+ Anum_pg_depend_classid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(RelationRelationId));
+ ScanKeyInit(&key[1],
+ Anum_pg_depend_objid,
+ BTEqualStrategyNumber, F_OIDEQ,
+ ObjectIdGetDatum(attrelid));
+ ScanKeyInit(&key[2],
+ Anum_pg_depend_objsubid,
+ BTEqualStrategyNumber, F_INT4EQ,
+ Int32GetDatum((int32) attnum));
+
+ scan = systable_beginscan(rel, DependDependerIndexId, true,
+ NULL, 3, key);
+
+ while (HeapTupleIsValid(tup = systable_getnext(scan)))
+ {
+ Form_pg_depend depform = (Form_pg_depend) GETSTRUCT(tup);
+
+ if (list_member_oid(cmoids, depform->refobjid))
+ {
+ Assert(depform->refclassid == AccessMethodRelationId);
+ CatalogTupleDelete(rel, &tup->t_self);
+ }
+ }
+
+ systable_endscan(scan);
+ table_close(rel, lock);
+}
+
+/*
+ * Check whether the given compression method oid is supported by
+ * the target attribute.
+ */
+bool
+IsCompressionSupported(Form_pg_attribute att, Oid cmoid)
+{
+ List *cmoids = NIL;
+
+ /* Check whether it is same as the current compression oid */
+ if (cmoid == att->attcompression)
+ return true;
+
+ /* Check the oid in all preserved compresion methods */
+ cmoids = lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+ if (list_member_oid(cmoids, cmoid))
+ return true;
+ else
+ return false;
+}
+
+/*
+ * In binary upgrade mode add the dependencies for all the preserved compression
+ * method.
+ */
+static void
+BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
+{
+ ListCell *cell;
+
+ foreach(cell, preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ add_column_compression_dependency(att->attrelid, att->attnum, cmoid_p);
+ }
+}
+
+/*
+ * Get the compression method oid based on the compression method name. When
+ * compression is not specified returns default attribute compression. It is
+ * possible case for CREATE TABLE and ADD COLUMN commands where COMPRESSION
+ * syntax is optional.
+ *
+ * For ALTER command, check all the supported compression methods for the
+ * attribute and if the preserve list is not passed or some of the old
+ * compression methods are not given in the preserved list then delete
+ * dependency from the old compression methods and force the table rewrite.
+ */
+Oid
+GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
+ bool *need_rewrite)
+{
+ Oid cmoid;
+ char typstorage = get_typstorage(att->atttypid);
+ ListCell *cell;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return GetDefaultToastCompression();
+
+ cmoid = get_compression_am_oid(compression->cmname, false);
+
+#ifndef HAVE_LIBLZ4
+ if (cmoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+
+ /*
+ * Determine if the column needs rewrite or not. Rewrite conditions: SET
+ * COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
+ * with full list of previous access methods.
+ */
+ if (need_rewrite != NULL)
+ {
+ List *previous_cmoids = NIL;
+
+ *need_rewrite = false;
+
+ /*
+ * In binary upgrade mode, just create a dependency on all preserved
+ * methods.
+ */
+ if (IsBinaryUpgrade)
+ {
+ BinaryUpgradeAddPreserve(att, compression->preserve);
+ return cmoid;
+ }
+
+ /* If we have preserved all then rewrite is not required */
+ if (compression->preserve_all)
+ return cmoid;
+
+ previous_cmoids =
+ lookup_attribute_compression(att->attrelid, att->attnum, NULL);
+
+ foreach(cell, compression->preserve)
+ {
+ char *cmname_p = strVal(lfirst(cell));
+ Oid cmoid_p = get_compression_am_oid(cmname_p, false);
+
+ if (!list_member_oid(previous_cmoids, cmoid_p))
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("\"%s\" compression access method cannot be preserved", cmname_p)));
+
+ /*
+ * Remove from previous list, also protect from duplicate
+ * entries in the PRESERVE list
+ */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid_p);
+ }
+
+ /* delete the current cmoid from the list */
+ previous_cmoids = list_delete_oid(previous_cmoids, cmoid);
+
+ /*
+ * If the list of previous Oids is not empty after deletions then
+ * we need to rewrite tuples in the table. Also remove the dependency
+ * on the old compression methods which are no longer preserved.
+ */
+ if (list_length(previous_cmoids) != 0)
+ {
+ remove_old_dependencies(att->attrelid, att->attnum,
+ previous_cmoids);
+ *need_rewrite = true;
+ }
+
+ /* Cleanup */
+ list_free(previous_cmoids);
+ }
+
+ return cmoid;
+}
+
+/*
+ * Construct ColumnCompression node from the compression method oid.
+ */
+ColumnCompression *
+MakeColumnCompression(Oid attcompression)
+{
+ ColumnCompression *node;
+
+ if (!OidIsValid(attcompression))
+ return NULL;
+
+ node = makeNode(ColumnCompression);
+ node->cmname = get_am_name(attcompression);
+
+ return node;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5e24af9cb3..5fba7352a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,7 +530,9 @@ static void ATExecEnableRowSecurity(Relation rel);
static void ATExecDisableRowSecurity(Relation rel);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
- const char *column, Node *newValue, LOCKMODE lockmode);
+ const char *column,
+ ColumnCompression *compression,
+ LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -562,7 +564,6 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -587,6 +588,7 @@ ObjectAddress
DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
ObjectAddress *typaddress, const char *queryString)
{
+ int i;
char relname[NAMEDATALEN];
Oid namespaceId;
Oid relationId;
@@ -865,7 +867,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
attr->attcompression =
- GetAttributeCompression(attr, colDef->compression);
+ GetAttributeCompression(attr, colDef->compression, NULL);
else
attr->attcompression = InvalidOid;
}
@@ -935,6 +937,20 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
*/
rel = relation_open(relationId, AccessExclusiveLock);
+ /*
+ * Add the dependency on the respective compression AM for the relation
+ * attributes.
+ */
+ for (i = 0; i < (RelationGetDescr(rel))->natts; i++)
+ {
+ Form_pg_attribute attr;
+
+ attr = TupleDescAttr(RelationGetDescr(rel), i);
+ if (OidIsValid(attr->attcompression))
+ add_column_compression_dependency(attr->attrelid, attr->attnum,
+ attr->attcompression);
+ }
+
/*
* Now add any newly specified column default and generation expressions
* to the new relation. These are passed to us in the form of raw
@@ -2415,16 +2431,17 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
/* Copy/check compression parameter */
if (OidIsValid(attribute->attcompression))
{
- char *compression = get_am_name(attribute->attcompression);
+ ColumnCompression *compression =
+ MakeColumnCompression(attribute->attcompression);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression, compression) != 0)
+ else if (strcmp(def->compression->cmname, compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, compression)));
+ errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
}
def->inhcount++;
@@ -2461,7 +2478,8 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(
+ attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2712,12 +2730,12 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->compression = newdef->compression;
else if (newdef->compression)
{
- if (strcmp(def->compression, newdef->compression))
+ if (strcmp(def->compression->cmname, newdef->compression->cmname))
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
errmsg("column \"%s\" has a compression method conflict",
attributeName),
- errdetail("%s versus %s", def->compression, newdef->compression)));
+ errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
}
/* Mark the column as locally defined */
@@ -4908,7 +4926,8 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
case AT_SetCompression:
- address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ address = ATExecSetCompression(tab, rel, cmd->name,
+ (ColumnCompression *) cmd->def,
lockmode);
break;
default: /* oops */
@@ -6414,7 +6433,8 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
if (rel->rd_rel->relkind == RELKIND_RELATION ||
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
- colDef->compression);
+ colDef->compression,
+ NULL);
else
attribute.attcompression = InvalidOid;
@@ -6589,6 +6609,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
*/
add_column_datatype_dependency(myrelid, newattnum, attribute.atttypid);
add_column_collation_dependency(myrelid, newattnum, attribute.attcollation);
+ add_column_compression_dependency(myrelid, newattnum, attribute.attcompression);
/*
* Propagate to children as appropriate. Unlike most other ALTER
@@ -6736,6 +6757,28 @@ add_column_collation_dependency(Oid relid, int32 attnum, Oid collid)
}
}
+/*
+ * Install a dependency for compression on its column.
+ *
+ * This is used for identifying all the supported compression methods
+ * (current and preserved) for a attribute.
+ *
+ * If dependency is already there the whole thing is skipped.
+ */
+void
+add_column_compression_dependency(Oid relid, int32 attnum, Oid cmoid)
+{
+ ObjectAddress acref,
+ attref;
+
+ Assert(relid > 0 && attnum > 0);
+
+ ObjectAddressSet(acref, AccessMethodRelationId, cmoid);
+ ObjectAddressSubSet(attref, RelationRelationId, relid, attnum);
+
+ recordMultipleDependencies(&attref, &acref, 1, DEPENDENCY_NORMAL, true);
+}
+
/*
* ALTER TABLE ALTER COLUMN DROP NOT NULL
*/
@@ -11867,7 +11910,8 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
foundDep->refobjid == attTup->attcollation) &&
!(foundDep->refclassid == RelationRelationId &&
foundDep->refobjid == RelationGetRelid(rel) &&
- foundDep->refobjsubid != 0)
+ foundDep->refobjsubid != 0) &&
+ foundDep->refclassid != AccessMethodRelationId
)
elog(ERROR, "found unexpected dependency for column: %s",
getObjectDescription(&foundObject, false));
@@ -11982,6 +12026,11 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
add_column_datatype_dependency(RelationGetRelid(rel), attnum, targettype);
add_column_collation_dependency(RelationGetRelid(rel), attnum, targetcollid);
+ /* Create dependency for new attribute compression */
+ if (OidIsValid(attTup->attcompression))
+ add_column_compression_dependency(RelationGetRelid(rel), attnum,
+ attTup->attcompression);
+
/*
* Drop any pg_statistic entry for the column, since it's now wrong type
*/
@@ -15086,24 +15135,21 @@ static ObjectAddress
ATExecSetCompression(AlteredTableInfo *tab,
Relation rel,
const char *column,
- Node *newValue,
+ ColumnCompression *compression,
LOCKMODE lockmode)
{
Relation attrel;
HeapTuple tuple;
Form_pg_attribute atttableform;
AttrNumber attnum;
- char *compression;
char typstorage;
Oid cmoid;
+ bool need_rewrite;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
ObjectAddress address;
- Assert(IsA(newValue, String));
- compression = strVal(newValue);
-
attrel = table_open(AttributeRelationId, RowExclusiveLock);
tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
@@ -15136,11 +15182,16 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
if (atttableform->attcompression != cmoid)
+ add_column_compression_dependency(atttableform->attrelid,
+ atttableform->attnum, cmoid);
+ if (need_rewrite)
tab->rewrite |= AT_REWRITE_ALTER_COMPRESSION;
+ atttableform->attcompression = cmoid;
+
atttableform->attcompression = cmoid;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
@@ -17865,42 +17916,3 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
-
-/*
- * resolve column compression specification to an OID.
- */
-static Oid
-GetAttributeCompression(Form_pg_attribute att, char *compression)
-{
- char typstorage = get_typstorage(att->atttypid);
- Oid amoid;
-
- /*
- * No compression for the plain/external storage, refer comments atop
- * attcompression parameter in pg_attribute.h
- */
- if (!IsStorageCompressible(typstorage))
- {
- if (compression == NULL)
- return InvalidOid;
-
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("column data type %s does not support compression",
- format_type_be(att->atttypid))));
- }
-
- /* fallback to default compression if it's not specified */
- if (compression == NULL)
- return GetDefaultToastCompression();
-
- amoid = get_compression_am_oid(compression, false);
-
-#ifndef HAVE_LIBLZ4
- if (amoid == LZ4_COMPRESSION_AM_OID)
- ereport(ERROR,
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
- errmsg("not built with lz4 support")));
-#endif
- return amoid;
-}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index ea82a05591..90d092671e 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -44,6 +44,7 @@
#include "access/tableam.h"
#include "access/xact.h"
#include "catalog/catalog.h"
+#include "commands/defrem.h"
#include "commands/trigger.h"
#include "executor/execPartition.h"
#include "executor/executor.h"
@@ -2068,8 +2069,8 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
/*
* Loop over all the attributes in the tuple and check if any attribute is
- * compressed and its compression method is not same as the target
- * atrribute's compression method then decompress it.
+ * compressed and its compression method is not is not supported by the
+ * target attribute then we need to decompress
*/
for (i = 0; i < natts; i++)
{
@@ -2094,12 +2095,13 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
DatumGetPointer(slot->tts_values[attnum - 1]);
/*
- * Get the compression method Oid stored in the toast header and
- * compare it with the compression method of the target.
+ * Get the compression method stored in the toast header and if the
+ * compression method is not supported by the target attribute then
+ * we need to decompress it.
*/
cmoid = toast_get_compression_oid(new_value);
if (OidIsValid(cmoid) &&
- targetTupDesc->attrs[i].attcompression != cmoid)
+ !IsCompressionSupported(&targetTupDesc->attrs[i], cmoid))
{
new_value = detoast_attr(new_value);
slot->tts_values[attnum - 1] = PointerGetDatum(new_value);
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 1338e04409..6a11f8eb60 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,7 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
- COPY_STRING_FIELD(compression);
+ COPY_NODE_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
@@ -2986,6 +2986,18 @@ _copyColumnDef(const ColumnDef *from)
return newnode;
}
+static ColumnCompression *
+_copyColumnCompression(const ColumnCompression *from)
+{
+ ColumnCompression *newnode = makeNode(ColumnCompression);
+
+ COPY_STRING_FIELD(cmname);
+ COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(preserve);
+
+ return newnode;
+}
+
static Constraint *
_copyConstraint(const Constraint *from)
{
@@ -5675,6 +5687,9 @@ copyObjectImpl(const void *from)
case T_ColumnDef:
retval = _copyColumnDef(from);
break;
+ case T_ColumnCompression:
+ retval = _copyColumnCompression(from);
+ break;
case T_Constraint:
retval = _copyConstraint(from);
break;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index f3592003da..26a9b85974 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,7 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
- COMPARE_STRING_FIELD(compression);
+ COMPARE_NODE_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
@@ -2619,6 +2619,16 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
return true;
}
+static bool
+_equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
+{
+ COMPARE_STRING_FIELD(cmname);
+ COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(preserve);
+
+ return true;
+}
+
static bool
_equalConstraint(const Constraint *a, const Constraint *b)
{
@@ -3724,6 +3734,9 @@ equal(const void *a, const void *b)
case T_ColumnDef:
retval = _equalColumnDef(a, b);
break;
+ case T_ColumnCompression:
+ retval = _equalColumnCompression(a, b);
+ break;
case T_Constraint:
retval = _equalConstraint(a, b);
break;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 0605ef3f84..b584a58ba3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,7 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
- WRITE_STRING_FIELD(compression);
+ WRITE_NODE_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
@@ -2881,6 +2881,16 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_LOCATION_FIELD(location);
}
+static void
+_outColumnCompression(StringInfo str, const ColumnCompression *node)
+{
+ WRITE_NODE_TYPE("COLUMNCOMPRESSION");
+
+ WRITE_STRING_FIELD(cmname);
+ WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(preserve);
+}
+
static void
_outTypeName(StringInfo str, const TypeName *node)
{
@@ -4258,6 +4268,9 @@ outNode(StringInfo str, const void *obj)
case T_ColumnDef:
_outColumnDef(str, obj);
break;
+ case T_ColumnCompression:
+ _outColumnCompression(str, obj);
+ break;
case T_TypeName:
_outTypeName(str, obj);
break;
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 30acfe615d..9eb2b04d58 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,7 +596,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
-%type <str> optColumnCompression
+%type <node> optColumnCompression alterColumnCompression
+%type <str> compressionClause
+%type <list> optCompressionPreserve
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
@@ -2309,12 +2311,12 @@ alter_table_cmd:
$$ = (Node *)n;
}
/* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
- | ALTER opt_column ColId SET optColumnCompression
+ | ALTER opt_column ColId SET alterColumnCompression
{
AlterTableCmd *n = makeNode(AlterTableCmd);
n->subtype = AT_SetCompression;
n->name = $3;
- n->def = (Node *) makeString($5);
+ n->def = $5;
$$ = (Node *)n;
}
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
@@ -3437,7 +3439,7 @@ columnDef: ColId Typename optColumnCompression create_generic_options ColQualLis
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
- n->compression = $3;
+ n->compression = (ColumnCompression *) $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3492,13 +3494,43 @@ columnOptions: ColId ColQualList
}
;
+optCompressionPreserve:
+ PRESERVE '(' name_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+compressionClause:
+ COMPRESSION name { $$ = pstrdup($2); }
+ ;
+
optColumnCompression:
- COMPRESSION name
- {
- $$ = $2;
- }
- | /*EMPTY*/ { $$ = NULL; }
- ;
+ compressionClause
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
+alterColumnCompression:
+ compressionClause optCompressionPreserve
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve = (List *) $2;
+ $$ = (Node *) n;
+ }
+ | compressionClause PRESERVE ALL
+ {
+ ColumnCompression *n = makeNode(ColumnCompression);
+ n->cmname = $1;
+ n->preserve_all = true;
+ n->preserve = NIL;
+ $$ = (Node *) n;
+ }
+ ;
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index e262ec5bcb..f50b50cc8e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = get_am_name(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute->attcompression);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 46044cb92a..7bf345a4ac 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -9037,6 +9037,80 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
}
PQclear(res);
}
+
+ /*
+ * Get compression info
+ */
+ if (fout->remoteVersion >= 140000 && dopt->binary_upgrade)
+ {
+ int i_amname;
+ int i_amoid;
+ int i_curattnum;
+ int start;
+
+ pg_log_info("finding compression info for table \"%s.%s\"",
+ tbinfo->dobj.namespace->dobj.name,
+ tbinfo->dobj.name);
+
+ tbinfo->attcompression = pg_malloc0(tbinfo->numatts * sizeof(AttrCompressionInfo *));
+
+ resetPQExpBuffer(q);
+ appendPQExpBuffer(q,
+ " SELECT attrelid::pg_catalog.regclass AS relname, attname,"
+ " amname, am.oid as amoid, d.objsubid AS curattnum"
+ " FROM pg_depend d"
+ " JOIN pg_attribute a ON"
+ " (classid = 'pg_class'::pg_catalog.regclass::pg_catalog.oid AND a.attrelid = d.objid"
+ " AND a.attnum = d.objsubid AND d.deptype = 'n'"
+ " AND d.refclassid = 'pg_am'::pg_catalog.regclass::pg_catalog.oid)"
+ " JOIN pg_am am ON"
+ " (d.deptype = 'n' AND d.refobjid = am.oid)"
+ " WHERE (deptype = 'n' AND d.objid = %d AND a.attcompression != am.oid);",
+ tbinfo->dobj.catId.oid);
+
+ res = ExecuteSqlQuery(fout, q->data, PGRES_TUPLES_OK);
+ ntups = PQntuples(res);
+
+ if (ntups > 0)
+ {
+ int j;
+ int k;
+
+ i_amname = PQfnumber(res, "amname");
+ i_amoid = PQfnumber(res, "amoid");
+ i_curattnum = PQfnumber(res, "curattnum");
+
+ start = 0;
+
+ for (j = 0; j < ntups; j++)
+ {
+ int attnum = atoi(PQgetvalue(res, j, i_curattnum));
+
+ if ((j == ntups - 1) || atoi(PQgetvalue(res, j + 1, i_curattnum)) != attnum)
+ {
+ AttrCompressionInfo *cminfo = pg_malloc(sizeof(AttrCompressionInfo));
+
+ cminfo->nitems = j - start + 1;
+ cminfo->items = pg_malloc(sizeof(AttrCompressionItem *) * cminfo->nitems);
+
+ for (k = start; k < start + cminfo->nitems; k++)
+ {
+ AttrCompressionItem *cmitem = pg_malloc0(sizeof(AttrCompressionItem));
+
+ cmitem->amname = pg_strdup(PQgetvalue(res, k, i_amname));
+ cmitem->amoid = atooid(PQgetvalue(res, k, i_amoid));
+
+ cminfo->items[k - start] = cmitem;
+ }
+
+ tbinfo->attcompression[attnum - 1] = cminfo;
+ start = j + 1; /* start from next */
+ }
+ }
+ }
+
+ PQclear(res);
+ }
}
destroyPQExpBuffer(q);
@@ -16343,6 +16417,33 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (tbinfo->attcompression && tbinfo->attcompression[j])
+ {
+ AttrCompressionInfo *cminfo = tbinfo->attcompression[j];
+
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
+ qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+
+ if (cminfo->nitems > 0)
+ {
+ appendPQExpBuffer(q, "\nPRESERVE (");
+ for (int i = 0; i < cminfo->nitems; i++)
+ {
+ AttrCompressionItem *item = cminfo->items[i];
+
+ if (i == 0)
+ appendPQExpBuffer(q, "%s", item->amname);
+ else
+ appendPQExpBuffer(q, ", %s", item->amname);
+ }
+ appendPQExpBuffer(q, ")");
+ }
+ appendPQExpBuffer(q, ";\n");
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 1789e18f46..a829528cd0 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,7 +327,8 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
-
+ struct _attrCompressionInfo **attcompression; /* per-attribute all
+ compression data */
/*
* Stuff computed only for dumpable tables.
*/
@@ -356,6 +357,18 @@ typedef struct _attrDefInfo
bool separate; /* true if must dump as separate item */
} AttrDefInfo;
+typedef struct _attrCompressionItem
+{
+ Oid amoid; /* attribute compression oid */
+ char *amname; /* compression access method name */
+} AttrCompressionItem;
+
+typedef struct _attrCompressionInfo
+{
+ int nitems;
+ AttrCompressionItem **items;
+} AttrCompressionInfo;
+
typedef struct _tableDataInfo
{
DumpableObject dobj;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 2aedddc1d8..05c782e8e9 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2112,6 +2112,13 @@ psql_completion(const char *text, int start, int end)
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
COMPLETE_WITH("n_distinct", "n_distinct_inherited");
+ /* ALTER TABLE ALTER [COLUMN] <foo> SET COMPRESSION */
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny) ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny))
+ COMPLETE_WITH("PRESERVE");
+ else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE") ||
+ Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "COMPRESSION", MatchAny, "PRESERVE"))
+ COMPLETE_WITH("( ", "ALL");
/* ALTER TABLE ALTER [COLUMN] <foo> SET STORAGE */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STORAGE") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "STORAGE"))
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index e5aea8a240..bd53f9bb0f 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -143,6 +143,13 @@ extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
+/* commands/compressioncmds.c */
+extern Oid GetAttributeCompression(Form_pg_attribute att,
+ ColumnCompression *compression,
+ bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+
/* support routines in commands/define.c */
extern char *defGetString(DefElem *def);
diff --git a/src/include/commands/tablecmds.h b/src/include/commands/tablecmds.h
index b3d30acc35..e6c98e65d4 100644
--- a/src/include/commands/tablecmds.h
+++ b/src/include/commands/tablecmds.h
@@ -97,5 +97,7 @@ extern void RangeVarCallbackOwnsRelation(const RangeVar *relation,
Oid relId, Oid oldRelId, void *arg);
extern bool PartConstraintImpliedByRelConstraint(Relation scanrel,
List *partConstraint);
+extern void add_column_compression_dependency(Oid relid, int32 attnum,
+ Oid cmoid);
#endif /* TABLECMDS_H */
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 20d6f96f62..24deaad253 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -481,6 +481,7 @@ typedef enum NodeTag
T_PartitionBoundSpec,
T_PartitionRangeDatum,
T_PartitionCmd,
+ T_ColumnCompression,
T_VacuumRelation,
/*
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index f9a87dee02..ce0913e18a 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -623,6 +623,20 @@ typedef struct RangeTableSample
int location; /* method name location, or -1 if unknown */
} RangeTableSample;
+/*
+ * ColumnCompression - compression parameters for some attribute
+ *
+ * This represents compression information defined using clause:
+ * .. COMPRESSION <compression method> PRESERVE <compression methods>
+ */
+typedef struct ColumnCompression
+{
+ NodeTag type;
+ char *cmname;
+ bool preserve_all;
+ List *preserve;
+} ColumnCompression;
+
/*
* ColumnDef - column definition (used in various creates)
*
@@ -646,7 +660,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
- char *compression; /* compression method for column */
+ ColumnCompression *compression; /* column compression */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 21c1b451d2..3ed33b6534 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -226,12 +226,47 @@ SELECT pg_column_compression(f1) FROM cmpart;
lz4
(2 rows)
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 64c5855bf7..36a5f8ba5e 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -205,12 +205,49 @@ SELECT pg_column_compression(f1) FROM cmpart;
ERROR: relation "cmpart" does not exist
LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
^
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 10040
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/expected/create_index.out b/src/test/regress/expected/create_index.out
index 830fdddf24..8f984510ac 100644
--- a/src/test/regress/expected/create_index.out
+++ b/src/test/regress/expected/create_index.out
@@ -2066,19 +2066,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
REINDEX INDEX CONCURRENTLY concur_reindex_ind1;
REINDEX TABLE CONCURRENTLY concur_reindex_tab;
@@ -2095,19 +2097,21 @@ WHERE classid = 'pg_class'::regclass AND
'concur_reindex_ind4'::regclass,
'concur_reindex_matview'::regclass)
ORDER BY 1, 2;
- obj | objref | deptype
-------------------------------------------+------------------------------------------------------------+---------
- index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
- index concur_reindex_ind2 | collation "default" | n
- index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
- index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind3 | table concur_reindex_tab | a
- index concur_reindex_ind4 | collation "default" | n
- index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
- index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
- materialized view concur_reindex_matview | schema public | n
- table concur_reindex_tab | schema public | n
-(10 rows)
+ obj | objref | deptype
+-------------------------------------------------------+------------------------------------------------------------+---------
+ column c2 of materialized view concur_reindex_matview | access method pglz | n
+ column c2 of table concur_reindex_tab | access method pglz | n
+ index concur_reindex_ind1 | constraint concur_reindex_ind1 on table concur_reindex_tab | i
+ index concur_reindex_ind2 | collation "default" | n
+ index concur_reindex_ind2 | column c2 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind3 | table concur_reindex_tab | a
+ index concur_reindex_ind4 | collation "default" | n
+ index concur_reindex_ind4 | column c1 of table concur_reindex_tab | a
+ index concur_reindex_ind4 | column c2 of table concur_reindex_tab | a
+ materialized view concur_reindex_matview | schema public | n
+ table concur_reindex_tab | schema public | n
+(12 rows)
-- Check that comments are preserved
CREATE TABLE testcomment (i int);
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b9daa33b74..5774b55d82 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -96,6 +96,15 @@ ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
SELECT pg_column_compression(f1) FROM cmpart;
+-- preserve old compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (lz4);
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
+SELECT pg_column_compression(f1) FROM cmdata;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0009-Create-custom-compression-methods.patchtext/x-diff; charset=us-asciiDownload
From 54fbe5935051558be3dff4989936cb163147a756 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 16:27:04 +0530
Subject: [PATCH 09/12] Create custom compression methods
Provide syntax to create custom compression methods.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 6 ++-
doc/src/sgml/ref/create_access_method.sgml | 12 +++--
doc/src/sgml/ref/create_table.sgml | 9 ++--
src/backend/access/common/detoast.c | 51 ++++++++++++++++---
src/backend/access/common/toast_internals.c | 19 +++++--
src/backend/access/compression/compress_lz4.c | 21 ++++----
.../access/compression/compress_pglz.c | 20 ++++----
src/backend/access/index/amapi.c | 51 ++++++++++++++-----
src/backend/commands/amcmds.c | 22 +++++++-
src/backend/parser/gram.y | 1 +
src/backend/utils/adt/pg_upgrade_support.c | 10 ++++
src/bin/pg_dump/pg_dump.c | 8 +++
src/bin/psql/tab-complete.c | 6 +++
src/include/access/amapi.h | 1 +
src/include/access/compressamapi.h | 45 ++++++++++++++--
src/include/access/toast_internals.h | 16 ++++++
src/include/catalog/binary_upgrade.h | 2 +
src/include/catalog/pg_proc.dat | 4 ++
src/include/postgres.h | 8 +++
src/test/regress/expected/compression.out | 40 ++++++++++++++-
src/test/regress/expected/compression_1.out | 41 ++++++++++++++-
src/test/regress/sql/compression.sql | 10 ++++
22 files changed, 344 insertions(+), 59 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c9f443a59c..49c43df1c1 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -391,8 +391,10 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</term>
<listitem>
<para>
- This sets the compression method for a column. The supported compression
- methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ This sets the compression method for a column. The Compression method
+ could be created with <xref linkend="sql-create-access-method"/> or
+ it can be set to any available compression method. The supported buit-in
+ compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
was used when building <productname>PostgreSQL</productname>.
The <literal>PRESERVE</literal> list contains a list of compression
diff --git a/doc/src/sgml/ref/create_access_method.sgml b/doc/src/sgml/ref/create_access_method.sgml
index dae43dbaed..c5ef8b738d 100644
--- a/doc/src/sgml/ref/create_access_method.sgml
+++ b/doc/src/sgml/ref/create_access_method.sgml
@@ -61,8 +61,8 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
<listitem>
<para>
This clause specifies the type of access method to define.
- Only <literal>TABLE</literal> and <literal>INDEX</literal>
- are supported at present.
+ Currently, <literal>TABLE</literal>, <literal>INDEX</literal> and
+ <literal>COMPRESSION</literal> access methods are supported.
</para>
</listitem>
</varlistentry>
@@ -77,11 +77,13 @@ CREATE ACCESS METHOD <replaceable class="parameter">name</replaceable>
declared to take a single argument of type <type>internal</type>,
and its return type depends on the type of access method;
for <literal>TABLE</literal> access methods, it must
- be <type>table_am_handler</type> and for <literal>INDEX</literal>
- access methods, it must be <type>index_am_handler</type>.
+ be <type>table_am_handler</type>, for <literal>INDEX</literal>
+ access methods, it must be <type>index_am_handler</type> and
+ for <literal>COMPRESSION</literal> access methods, it must be
+ <type>compression_am_handler</type>.
The C-level API that the handler function must implement varies
depending on the type of access method. The table access method API
- is described in <xref linkend="tableam"/> and the index access method
+ is described in <xref linkend="tableam"/>, the index access method
API is described in <xref linkend="indexam"/>.
</para>
</listitem>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 807597e648..5e13b3288c 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -1004,11 +1004,14 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
column storage types.) Setting this property for a partitioned table
has no direct effect, because such tables have no storage of their own,
but the configured value is inherited by newly-created partitions.
- The supported compression methods are <literal>pglz</literal> and
+ The supported built-in compression methods are <literal>pglz</literal> and
<literal>lz4</literal>. <literal>lz4</literal> is available only if
<literal>--with-lz4</literal> was used when building
- <productname>PostgreSQL</productname>. The default
- is <literal>pglz</literal>.
+ <productname>PostgreSQL</productname>.
+ Compression methods can be created with <xref
+ linkend="sql-create-access-method"/> or it can be set to any available
+ compression method.
+ The default is <literal>pglz</literal>.
</para>
</listitem>
</varlistentry>
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 95d5b1c12a..fd7c42c95e 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -462,10 +462,17 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*
* Returns the Oid of the compression method stored in the compressed data. If
* the varlena is not compressed then returns InvalidOid.
+ *
+ * For built-in methods, we only store the built-in compression method id in
+ * first 2-bits of the rawsize and that is directly mapped to the compression
+ * method Oid. And, for the custom compression method we store the Oid of the
+ * compression method in the custom compression header.
*/
Oid
toast_get_compression_oid(struct varlena *attr)
{
+ CompressionId cmid;
+
if (VARATT_IS_EXTERNAL_ONDISK(attr))
{
struct varatt_external toast_pointer;
@@ -485,7 +492,21 @@ toast_get_compression_oid(struct varlena *attr)
else if (!VARATT_IS_COMPRESSED(attr))
return InvalidOid;
- return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+ /*
+ * If it is custom compression id then get the Oid from the custom
+ * compression header otherwise, directly translate the built-in
+ * compression id to compression method Oid.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ if (IsCustomCompression(cmid))
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ return hdr->cmoid;
+ }
+ else
+ return CompressionIdToOid(cmid);
}
/* ----------
@@ -494,7 +515,7 @@ toast_get_compression_oid(struct varlena *attr)
* helper function for toast_decompress_datum and toast_decompress_datum_slice
*/
static inline const CompressionAmRoutine *
-toast_get_compression_handler(struct varlena *attr)
+toast_get_compression_handler(struct varlena *attr, int32 *header_size)
{
const CompressionAmRoutine *cmroutine;
CompressionId cmid;
@@ -507,10 +528,21 @@ toast_get_compression_handler(struct varlena *attr)
{
case PGLZ_COMPRESSION_ID:
cmroutine = &pglz_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
case LZ4_COMPRESSION_ID:
cmroutine = &lz4_compress_methods;
+ *header_size = TOAST_COMPRESS_HDRSZ;
break;
+ case CUSTOM_COMPRESSION_ID:
+ {
+ toast_compress_header_custom *hdr;
+
+ hdr = (toast_compress_header_custom *) attr;
+ cmroutine = GetCompressionAmRoutineByAmId(hdr->cmoid);
+ *header_size = TOAST_CUSTOM_COMPRESS_HDRSZ;
+ break;
+ }
default:
elog(ERROR, "invalid compression method id %d", cmid);
}
@@ -526,9 +558,11 @@ toast_get_compression_handler(struct varlena *attr)
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
@@ -542,16 +576,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+ int32 header_size;
+ const CompressionAmRoutine *cmroutine =
+ toast_get_compression_handler(attr, &header_size);
/*
* If the handler supports the slice decompression then decompress the
* slice otherwise decompress complete data.
*/
if (cmroutine->datum_decompress_slice)
- return cmroutine->datum_decompress_slice(attr, slicelength);
+ return cmroutine->datum_decompress_slice(attr, header_size,
+ slicelength);
else
- return cmroutine->datum_decompress(attr);
+ return cmroutine->datum_decompress(attr, header_size);
}
/* ----------
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index b04c5a5eb8..a3539065d3 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -48,6 +48,7 @@ toast_compress_datum(Datum value, Oid cmoid)
{
struct varlena *tmp = NULL;
int32 valsize;
+ bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
@@ -65,11 +66,16 @@ toast_compress_datum(Datum value, Oid cmoid)
cmroutine = &lz4_compress_methods;
break;
default:
- elog(ERROR, "Invalid compression method oid %u", cmoid);
+ isCustomCompression = true;
+ cmroutine = GetCompressionAmRoutineByAmId(cmoid);
+ break;
}
/* Call the actual compression function */
- tmp = cmroutine->datum_compress((const struct varlena *) value);
+ tmp = cmroutine->datum_compress((const struct varlena *)value,
+ isCustomCompression ?
+ TOAST_CUSTOM_COMPRESS_HDRSZ :
+ TOAST_COMPRESS_HDRSZ);
if (!tmp)
return PointerGetDatum(NULL);
@@ -88,7 +94,14 @@ toast_compress_datum(Datum value, Oid cmoid)
if (VARSIZE(tmp) < valsize - 2)
{
/* successful compression */
- TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, isCustomCompression ?
+ CUSTOM_COMPRESSION_ID :
+ CompressionOidToId(cmoid));
+
+ /* For custom compression, set the oid of the compression method */
+ if (isCustomCompression)
+ TOAST_COMPRESS_SET_CMOID(tmp, cmoid);
+
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 3079eff7eb..2a3c162836 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -27,7 +27,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value)
+lz4_cmcompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -46,10 +46,10 @@ lz4_cmcompress(const struct varlena *value)
* that will be needed for varlena overhead, and allocate that amount.
*/
max_size = LZ4_compressBound(valsize);
- tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+ tmp = (struct varlena *) palloc(max_size + header_size);
len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + VARHDRSZ_COMPRESS,
+ (char *) tmp + header_size,
valsize, max_size);
if (len <= 0)
elog(ERROR, "could not compress data with lz4");
@@ -61,7 +61,7 @@ lz4_cmcompress(const struct varlena *value)
return NULL;
}
- SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
#endif
@@ -73,7 +73,7 @@ lz4_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress(const struct varlena *value)
+lz4_cmdecompress(const struct varlena *value, int32 header_size)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -87,9 +87,9 @@ lz4_cmdecompress(const struct varlena *value)
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
/* decompress the data */
- rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ rawsize = LZ4_decompress_safe((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARSIZE(value) - header_size,
VARRAWSIZE_4B_C(value));
if (rawsize < 0)
ereport(ERROR,
@@ -109,7 +109,8 @@ lz4_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
+ int32 slicelength)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -123,9 +124,9 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
/* decompress partial data using lz4 routine */
- rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ rawsize = LZ4_decompress_safe_partial((char *) value + header_size,
VARDATA(result),
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARSIZE(value) - header_size,
slicelength,
VARRAWSIZE_4B_C(value));
if (rawsize < 0)
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 8a4bf427cf..7f6e7429fe 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -26,7 +26,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value)
+pglz_cmcompress(const struct varlena *value, int32 header_size)
{
int32 valsize,
len;
@@ -47,11 +47,11 @@ pglz_cmcompress(const struct varlena *value)
* and allocate the memory for holding the compressed data and the header.
*/
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- VARHDRSZ_COMPRESS);
+ header_size);
len = pglz_compress(VARDATA_ANY(value),
valsize,
- (char *) tmp + VARHDRSZ_COMPRESS,
+ (char *) tmp + header_size,
NULL);
if (len < 0)
{
@@ -59,7 +59,7 @@ pglz_cmcompress(const struct varlena *value)
return NULL;
}
- SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
return tmp;
}
@@ -70,15 +70,15 @@ pglz_cmcompress(const struct varlena *value)
* Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress(const struct varlena *value)
+pglz_cmdecompress(const struct varlena *value, int32 header_size)
{
struct varlena *result;
int32 rawsize;
result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
- rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
VARRAWSIZE_4B_C(value), true);
if (rawsize < 0)
@@ -97,7 +97,7 @@ pglz_cmdecompress(const struct varlena *value)
* Decompresses part of the data. Returns the decompressed varlena.
*/
static struct varlena *
-pglz_cmdecompress_slice(const struct varlena *value,
+pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
int32 slicelength)
{
struct varlena *result;
@@ -105,8 +105,8 @@ pglz_cmdecompress_slice(const struct varlena *value,
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
- rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
- VARSIZE(value) - VARHDRSZ_COMPRESS,
+ rawsize = pglz_decompress((char *) value + header_size,
+ VARSIZE(value) - header_size,
VARDATA(result),
slicelength, false);
if (rawsize < 0)
diff --git a/src/backend/access/index/amapi.c b/src/backend/access/index/amapi.c
index d30bc43514..16f8fab359 100644
--- a/src/backend/access/index/amapi.c
+++ b/src/backend/access/index/amapi.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "catalog/pg_am.h"
#include "catalog/pg_opclass.h"
@@ -46,14 +47,14 @@ GetIndexAmRoutine(Oid amhandler)
}
/*
- * GetIndexAmRoutineByAmId - look up the handler of the index access method
- * with the given OID, and get its IndexAmRoutine struct.
+ * GetAmHandlerByAmId - look up the handler of the index/compression access
+ * method with the given OID, and get its handler function.
*
- * If the given OID isn't a valid index access method, returns NULL if
- * noerror is true, else throws error.
+ * If the given OID isn't a valid index/compression access method, returns
+ * Invalid Oid if noerror is true, else throws error.
*/
-IndexAmRoutine *
-GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+regproc
+GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror)
{
HeapTuple tuple;
Form_pg_am amform;
@@ -64,24 +65,25 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (!HeapTupleIsValid(tuple))
{
if (noerror)
- return NULL;
+ return InvalidOid;
elog(ERROR, "cache lookup failed for access method %u",
amoid);
}
amform = (Form_pg_am) GETSTRUCT(tuple);
/* Check if it's an index access method as opposed to some other AM */
- if (amform->amtype != AMTYPE_INDEX)
+ if (amform->amtype != amtype)
{
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
errmsg("access method \"%s\" is not of type %s",
- NameStr(amform->amname), "INDEX")));
+ NameStr(amform->amname), (amtype == AMTYPE_INDEX ?
+ "INDEX" : "COMPRESSION"))));
}
amhandler = amform->amhandler;
@@ -92,16 +94,41 @@ GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
if (noerror)
{
ReleaseSysCache(tuple);
- return NULL;
+ return InvalidOid;
}
ereport(ERROR,
(errcode(ERRCODE_OBJECT_NOT_IN_PREREQUISITE_STATE),
- errmsg("index access method \"%s\" does not have a handler",
+ errmsg("access method \"%s\" does not have a handler",
NameStr(amform->amname))));
}
ReleaseSysCache(tuple);
+ return amhandler;
+}
+
+/*
+ * GetIndexAmRoutineByAmId - look up the handler of the index access method
+ * with the given OID, and get its IndexAmRoutine struct.
+ *
+ * If the given OID isn't a valid index access method, returns NULL if
+ * noerror is true, else throws error.
+ */
+IndexAmRoutine *
+GetIndexAmRoutineByAmId(Oid amoid, bool noerror)
+{
+ regproc amhandler;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_INDEX, noerror);
+
+ /* Complain if handler OID is invalid */
+ if (!OidIsValid(amhandler))
+ {
+ Assert(noerror);
+ return NULL;
+ }
+
/* And finally, call the handler function to get the API struct. */
return GetIndexAmRoutine(amhandler);
}
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 1682afd2a4..0ac86eee02 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -45,6 +45,9 @@ static Oid default_toast_compression_oid = InvalidOid;
static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue);
+/* Set by pg_upgrade_support functions */
+Oid binary_upgrade_next_pg_am_oid = InvalidOid;
+
/*
* CreateAccessMethod
* Registers a new access method.
@@ -93,7 +96,19 @@ CreateAccessMethod(CreateAmStmt *stmt)
memset(values, 0, sizeof(values));
memset(nulls, false, sizeof(nulls));
- amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+ if (IsBinaryUpgrade && OidIsValid(binary_upgrade_next_pg_am_oid))
+ {
+ /* amoid should be found in some cases */
+ if (binary_upgrade_next_pg_am_oid < FirstNormalObjectId &&
+ (!OidIsValid(amoid) || binary_upgrade_next_pg_am_oid != amoid))
+ elog(ERROR, "could not link to built-in attribute compression");
+
+ amoid = binary_upgrade_next_pg_am_oid;
+ binary_upgrade_next_pg_am_oid = InvalidOid;
+ }
+ else
+ amoid = GetNewOidWithIndex(rel, AmOidIndexId, Anum_pg_am_oid);
+
values[Anum_pg_am_oid - 1] = ObjectIdGetDatum(amoid);
values[Anum_pg_am_amname - 1] =
DirectFunctionCall1(namein, CStringGetDatum(stmt->amname));
@@ -237,6 +252,8 @@ get_am_type_string(char amtype)
return "INDEX";
case AMTYPE_TABLE:
return "TABLE";
+ case AMTYPE_COMPRESSION:
+ return "COMPRESSION";
default:
/* shouldn't happen */
elog(ERROR, "invalid access method type '%c'", amtype);
@@ -274,6 +291,9 @@ lookup_am_handler_func(List *handler_name, char amtype)
case AMTYPE_TABLE:
expectedType = TABLE_AM_HANDLEROID;
break;
+ case AMTYPE_COMPRESSION:
+ expectedType = COMPRESSION_AM_HANDLEROID;
+ break;
default:
elog(ERROR, "unrecognized access method type \"%c\"", amtype);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9eb2b04d58..b22ce818cd 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -5305,6 +5305,7 @@ CreateAmStmt: CREATE ACCESS METHOD name TYPE_P am_type HANDLER handler_name
am_type:
INDEX { $$ = AMTYPE_INDEX; }
| TABLE { $$ = AMTYPE_TABLE; }
+ | COMPRESSION { $$ = AMTYPE_COMPRESSION; }
;
/*****************************************************************************
diff --git a/src/backend/utils/adt/pg_upgrade_support.c b/src/backend/utils/adt/pg_upgrade_support.c
index a575c95079..f026104c01 100644
--- a/src/backend/utils/adt/pg_upgrade_support.c
+++ b/src/backend/utils/adt/pg_upgrade_support.c
@@ -128,6 +128,16 @@ binary_upgrade_set_next_pg_authid_oid(PG_FUNCTION_ARGS)
PG_RETURN_VOID();
}
+Datum
+binary_upgrade_set_next_pg_am_oid(PG_FUNCTION_ARGS)
+{
+ Oid amoid = PG_GETARG_OID(0);
+
+ CHECK_IS_BINARY_UPGRADE;
+ binary_upgrade_next_pg_am_oid = amoid;
+ PG_RETURN_VOID();
+}
+
Datum
binary_upgrade_create_empty_extension(PG_FUNCTION_ARGS)
{
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 7bf345a4ac..88fa7e1ed3 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -13136,6 +13136,11 @@ dumpAccessMethod(Archive *fout, const AccessMethodInfo *aminfo)
qamname = pg_strdup(fmtId(aminfo->dobj.name));
+ if (dopt->binary_upgrade && aminfo->amtype == AMTYPE_COMPRESSION)
+ appendPQExpBuffer(q,
+ "SELECT pg_catalog.binary_upgrade_set_next_pg_am_oid('%u'::pg_catalog.oid);\n",
+ aminfo->dobj.catId.oid);
+
appendPQExpBuffer(q, "CREATE ACCESS METHOD %s ", qamname);
switch (aminfo->amtype)
@@ -13146,6 +13151,9 @@ dumpAccessMethod(Archive *fout, const AccessMethodInfo *aminfo)
case AMTYPE_TABLE:
appendPQExpBufferStr(q, "TYPE TABLE ");
break;
+ case AMTYPE_COMPRESSION:
+ appendPQExpBufferStr(q, "TYPE COMPRESSION ");
+ break;
default:
pg_log_warning("invalid type \"%c\" of access method \"%s\"",
aminfo->amtype, qamname);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 05c782e8e9..fe4be7bda4 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -883,6 +883,12 @@ static const SchemaQuery Query_for_list_of_statistics = {
" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
" amtype=" CppAsString2(AMTYPE_TABLE)
+#define Query_for_list_of_compression_access_methods \
+" SELECT pg_catalog.quote_ident(amname) "\
+" FROM pg_catalog.pg_am "\
+" WHERE substring(pg_catalog.quote_ident(amname),1,%d)='%s' AND "\
+" amtype=" CppAsString2(AMTYPE_COMPRESSION)
+
/* the silly-looking length condition is just to eat up the current word */
#define Query_for_list_of_arguments \
"SELECT pg_catalog.oidvectortypes(proargtypes)||')' "\
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index 1513cafcf4..bc1e2437f0 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -285,6 +285,7 @@ typedef struct IndexAmRoutine
/* Functions in access/index/amapi.c */
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
+extern regproc GetAmHandlerByAmId(Oid amoid, char amtype, bool noerror);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
void InitializeAccessMethods(void);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index d75a8e9df2..bac7b47cfc 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -15,10 +15,13 @@
#include "postgres.h"
+#include "fmgr.h"
+#include "access/amapi.h"
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
#include "utils/guc.h"
+
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
@@ -27,7 +30,9 @@
typedef enum CompressionId
{
PGLZ_COMPRESSION_ID = 0,
- LZ4_COMPRESSION_ID = 1
+ LZ4_COMPRESSION_ID = 1,
+ /* one free slot for the future built-in method */
+ CUSTOM_COMPRESSION_ID = 3
} CompressionId;
/* Default compression method if not specified. */
@@ -41,13 +46,19 @@ extern bool check_default_toast_compression(char **newval, void **extra, GucSour
extern Oid GetDefaultToastCompression(void);
+#define IsCustomCompression(cmid) ((cmid) == CUSTOM_COMPRESSION_ID)
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
-typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
-typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
+ int32 toast_header_size);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
+ int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
- (const struct varlena *value, int32 slicelength);
+ (const struct varlena *value,
+ int32 toast_header_size,
+ int32 slicelength);
/*
* API struct for a compression AM.
@@ -106,4 +117,30 @@ CompressionIdToOid(CompressionId cmid)
}
}
+/*
+ * GetCompressionAmRoutineByAmId - look up the handler of the compression access
+ * method with the given OID, and get its CompressionAmRoutine struct.
+ */
+static inline CompressionAmRoutine *
+GetCompressionAmRoutineByAmId(Oid amoid)
+{
+ regproc amhandler;
+ Datum datum;
+ CompressionAmRoutine *routine;
+
+ /* Get handler function OID for the access method */
+ amhandler = GetAmHandlerByAmId(amoid, AMTYPE_COMPRESSION, false);
+ Assert(OidIsValid(amhandler));
+
+ /* And finally, call the handler function to get the API struct */
+ datum = OidFunctionCall0(amhandler);
+ routine = (CompressionAmRoutine *) DatumGetPointer(datum);
+
+ if (routine == NULL || !IsA(routine, CompressionAmRoutine))
+ elog(ERROR, "compression access method handler function %u did not return an CompressionAmRoutine struct",
+ amhandler);
+
+ return routine;
+}
+
#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index 31ff91a09c..ac28f9ed55 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -27,10 +27,23 @@ typedef struct toast_compress_header
* rawsize */
} toast_compress_header;
+/*
+ * If the compression method were used, then data also contains
+ * Oid of compression options
+ */
+typedef struct toast_compress_header_custom
+{
+ int32 vl_len_; /* varlena header (do not touch directly!) */
+ uint32 info; /* 2 bits for compression method + rawsize */
+ Oid cmoid; /* Oid from pg_am */
+} toast_compress_header_custom;
+
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
+#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
+#define TOAST_CUSTOM_COMPRESS_HDRSZ ((int32)sizeof(toast_compress_header_custom))
#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->info >> VARLENA_RAWSIZE_BITS)
#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
do { \
@@ -38,6 +51,9 @@ typedef struct toast_compress_header
((toast_compress_header *) (ptr))->info = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
} while (0)
+#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
+ (((toast_compress_header_custom *)(ptr))->cmoid = (oid))
+
extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
diff --git a/src/include/catalog/binary_upgrade.h b/src/include/catalog/binary_upgrade.h
index f6e82e7ac5..9d65bae238 100644
--- a/src/include/catalog/binary_upgrade.h
+++ b/src/include/catalog/binary_upgrade.h
@@ -26,6 +26,8 @@ extern PGDLLIMPORT Oid binary_upgrade_next_toast_pg_class_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_enum_oid;
extern PGDLLIMPORT Oid binary_upgrade_next_pg_authid_oid;
+extern PGDLLIMPORT Oid binary_upgrade_next_pg_am_oid;
+
extern PGDLLIMPORT bool binary_upgrade_record_init_privs;
#endif /* BINARY_UPGRADE_H */
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 4b3d34ae1a..6556fe847e 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -10777,6 +10777,10 @@
proname => 'binary_upgrade_set_next_pg_authid_oid', provolatile => 'v',
proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
prosrc => 'binary_upgrade_set_next_pg_authid_oid' },
+{ oid => '2137', descr => 'for use by pg_upgrade',
+ proname => 'binary_upgrade_set_next_pg_am_oid', provolatile => 'v',
+ proparallel => 'r', prorettype => 'void', proargtypes => 'oid',
+ prosrc => 'binary_upgrade_set_next_pg_am_oid' },
{ oid => '3591', descr => 'for use by pg_upgrade',
proname => 'binary_upgrade_create_empty_extension', proisstrict => 'f',
provolatile => 'v', proparallel => 'u', prorettype => 'void',
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 667927fd7c..ede3b11ef3 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -149,6 +149,14 @@ typedef union
* flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
+ struct /* Compressed-in-line format */
+ {
+ uint32 va_header;
+ uint32 va_info; /* Original data size (excludes header) and
+ * flags */
+ Oid va_cmid; /* Oid of compression method */
+ char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
+ } va_custom_compressed;
} varattrib_4b;
typedef struct
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3ed33b6534..e52e272d39 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -260,13 +260,51 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 36a5f8ba5e..d4e215c931 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -241,13 +241,52 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz2
+(3 rows)
+
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+ERROR: not built with lz4 support
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz2
+(3 rows)
+
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz2 | |
+Indexes:
+ "idx" btree (f1)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
10040
-(2 rows)
+ 10040
+(3 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 5774b55d82..9d2e72b784 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -105,6 +105,16 @@ SELECT pg_column_compression(f1) FROM cmdata;
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE ALL;
SELECT pg_column_compression(f1) FROM cmdata;
+-- create compression method
+CREATE ACCESS METHOD pglz2 TYPE COMPRESSION HANDLER pglzhandler;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION pglz2 PRESERVE ALL;
+INSERT INTO cmdata VALUES (repeat('1234567890',1004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
+SELECT pg_column_compression(f1) FROM cmdata;
+\d+ cmdata
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0010-new-compression-method-extension-for-zlib.patchtext/x-diff; charset=us-asciiDownload
From 43fd97de3bc880cff9f53c517be6f99df3e781b7 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 16 Oct 2020 19:56:14 +0530
Subject: [PATCH 10/12] new compression method extension for zlib
Dilip Kumar
---
contrib/Makefile | 1 +
contrib/cmzlib/.gitignore | 4 +
contrib/cmzlib/Makefile | 26 +++++
contrib/cmzlib/cmzlib--1.0.sql | 13 +++
contrib/cmzlib/cmzlib.c | 157 +++++++++++++++++++++++++++++
contrib/cmzlib/cmzlib.control | 5 +
contrib/cmzlib/expected/cmzlib.out | 53 ++++++++++
contrib/cmzlib/sql/cmzlib.sql | 22 ++++
8 files changed, 281 insertions(+)
create mode 100644 contrib/cmzlib/.gitignore
create mode 100644 contrib/cmzlib/Makefile
create mode 100644 contrib/cmzlib/cmzlib--1.0.sql
create mode 100644 contrib/cmzlib/cmzlib.c
create mode 100644 contrib/cmzlib/cmzlib.control
create mode 100644 contrib/cmzlib/expected/cmzlib.out
create mode 100644 contrib/cmzlib/sql/cmzlib.sql
diff --git a/contrib/Makefile b/contrib/Makefile
index f27e458482..9e452d8dd0 100644
--- a/contrib/Makefile
+++ b/contrib/Makefile
@@ -12,6 +12,7 @@ SUBDIRS = \
bloom \
btree_gin \
btree_gist \
+ cmzlib \
citext \
cube \
dblink \
diff --git a/contrib/cmzlib/.gitignore b/contrib/cmzlib/.gitignore
new file mode 100644
index 0000000000..5dcb3ff972
--- /dev/null
+++ b/contrib/cmzlib/.gitignore
@@ -0,0 +1,4 @@
+# Generated subdirectories
+/log/
+/results/
+/tmp_check/
diff --git a/contrib/cmzlib/Makefile b/contrib/cmzlib/Makefile
new file mode 100644
index 0000000000..956fbe7cc8
--- /dev/null
+++ b/contrib/cmzlib/Makefile
@@ -0,0 +1,26 @@
+# contrib/cmzlib/Makefile
+
+MODULE_big = cmzlib
+OBJS = \
+ $(WIN32RES) \
+ cmzlib.o
+
+EXTENSION = cmzlib
+DATA = cmzlib--1.0.sql
+PGFILEDESC = "zlib compression method "
+
+SHLIB_LINK += $(filter -lz, $(LIBS))
+
+REGRESS = cmzlib
+
+
+ifdef USE_PGXS
+PG_CONFIG = pg_config
+PGXS := $(shell $(PG_CONFIG) --pgxs)
+include $(PGXS)
+else
+subdir = contrib/cmzlib
+top_builddir = ../..
+include $(top_builddir)/src/Makefile.global
+include $(top_srcdir)/contrib/contrib-global.mk
+endif
diff --git a/contrib/cmzlib/cmzlib--1.0.sql b/contrib/cmzlib/cmzlib--1.0.sql
new file mode 100644
index 0000000000..41f2f95870
--- /dev/null
+++ b/contrib/cmzlib/cmzlib--1.0.sql
@@ -0,0 +1,13 @@
+/* contrib/cm_lz4/cmzlib--1.0.sql */
+
+-- complain if script is sourced in psql, rather than via CREATE EXTENSION
+\echo Use "CREATE EXTENSION cmzlib" to load this file. \quit
+
+CREATE FUNCTION zlibhandler(internal)
+RETURNS compression_am_handler
+AS 'MODULE_PATHNAME'
+LANGUAGE C;
+
+-- Compression method
+CREATE ACCESS METHOD zlib TYPE COMPRESSION HANDLER zlibhandler;
+COMMENT ON ACCESS METHOD zlib IS 'zlib compression method';
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
new file mode 100644
index 0000000000..686a7c7e0d
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.c
@@ -0,0 +1,157 @@
+/*-------------------------------------------------------------------------
+ *
+ * cmzlib.c
+ * zlib compression method
+ *
+ * Copyright (c) 2015-2018, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * contrib/cmzlib/cmzlib.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+#include <zlib.h>
+
+PG_MODULE_MAGIC;
+
+PG_FUNCTION_INFO_V1(zlibhandler);
+
+void _PG_init(void);
+
+/*
+ * Module initialize function: initialize info about zlib
+ */
+void
+_PG_init(void)
+{
+
+}
+
+#define ZLIB_MAX_DICTIONARY_LENGTH 32768
+#define ZLIB_DICTIONARY_DELIM (" ,")
+
+typedef struct
+{
+ int level;
+ Bytef dict[ZLIB_MAX_DICTIONARY_LENGTH];
+ unsigned int dictlen;
+} zlib_state;
+
+/*
+ * zlib_cmcompress - compression routine for zlib compression method
+ *
+ * Compresses source into dest using the default compression level.
+ * Returns the compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+zlib_cmcompress(const struct varlena *value, int32 header_size)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+ z_streamp zp;
+ int res;
+ zlib_state state;
+
+ state.level = Z_DEFAULT_COMPRESSION;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (deflateInit(zp, state.level) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ tmp = (struct varlena *) palloc(valsize + header_size);
+ zp->next_in = (void *) VARDATA_ANY(value);
+ zp->avail_in = valsize;
+ zp->avail_out = valsize;
+ zp->next_out = (void *) ((char *) tmp + header_size);
+
+ do
+ {
+ res = deflate(zp, Z_FINISH);
+ if (res == Z_STREAM_ERROR)
+ elog(ERROR, "could not compress data: %s", zp->msg);
+ } while (zp->avail_in != 0);
+
+ Assert(res == Z_STREAM_END);
+
+ len = valsize - zp->avail_out;
+ if (deflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression stream: %s", zp->msg);
+ pfree(zp);
+
+ if (len > 0)
+ {
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
+ }
+
+ pfree(tmp);
+ return NULL;
+}
+
+/*
+ * zlib_cmdecompress - decompression routine for zlib compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+zlib_cmdecompress(const struct varlena *value, int32 header_size)
+{
+ struct varlena *result;
+ z_streamp zp;
+ int res = Z_OK;
+
+ zp = (z_streamp) palloc(sizeof(z_stream));
+ zp->zalloc = Z_NULL;
+ zp->zfree = Z_NULL;
+ zp->opaque = Z_NULL;
+
+ if (inflateInit(zp) != Z_OK)
+ elog(ERROR, "could not initialize compression library: %s", zp->msg);
+
+ zp->next_in = (void *) ((char *) value + header_size);
+ zp->avail_in = VARSIZE(value) - header_size;
+ zp->avail_out = VARRAWSIZE_4B_C(value);
+
+ result = (struct varlena *) palloc(zp->avail_out + VARHDRSZ);
+ SET_VARSIZE(result, zp->avail_out + VARHDRSZ);
+ zp->next_out = (void *) VARDATA(result);
+
+ while (zp->avail_in > 0)
+ {
+ res = inflate(zp, 0);
+ if (!(res == Z_OK || res == Z_STREAM_END))
+ elog(ERROR, "could not uncompress data: %s", zp->msg);
+ }
+
+ if (inflateEnd(zp) != Z_OK)
+ elog(ERROR, "could not close compression library: %s", zp->msg);
+
+ pfree(zp);
+ return result;
+}
+
+const CompressionAmRoutine zlib_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = zlib_cmcompress,
+ .datum_decompress = zlib_cmdecompress,
+ .datum_decompress_slice = NULL};
+
+Datum
+zlibhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&zlib_compress_methods);
+}
diff --git a/contrib/cmzlib/cmzlib.control b/contrib/cmzlib/cmzlib.control
new file mode 100644
index 0000000000..2eb10f3a83
--- /dev/null
+++ b/contrib/cmzlib/cmzlib.control
@@ -0,0 +1,5 @@
+# cm_lz4 extension
+comment = 'cmzlib compression method'
+default_version = '1.0'
+module_pathname = '$libdir/cmzlib'
+relocatable = true
diff --git a/contrib/cmzlib/expected/cmzlib.out b/contrib/cmzlib/expected/cmzlib.out
new file mode 100644
index 0000000000..2b6fac7e0b
--- /dev/null
+++ b/contrib/cmzlib/expected/cmzlib.out
@@ -0,0 +1,53 @@
+CREATE EXTENSION cmzlib;
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+(2 rows)
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | zlib | |
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+ Table "public.zlibtest"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM zlibtest;
+ pg_column_compression
+-----------------------
+ zlib
+ zlib
+ pglz
+(3 rows)
+
+SELECT length(f1) FROM zlibtest;
+ length
+--------
+ 10040
+ 24096
+ 10040
+(3 rows)
+
+DROP TABLE zlibtest;
diff --git a/contrib/cmzlib/sql/cmzlib.sql b/contrib/cmzlib/sql/cmzlib.sql
new file mode 100644
index 0000000000..ea8d206625
--- /dev/null
+++ b/contrib/cmzlib/sql/cmzlib.sql
@@ -0,0 +1,22 @@
+CREATE EXTENSION cmzlib;
+
+-- zlib compression
+CREATE TABLE zlibtest(f1 TEXT COMPRESSION pglz);
+INSERT INTO zlibtest VALUES(repeat('1234567890',1004));
+INSERT INTO zlibtest VALUES(repeat('1234567890 one two three',1004));
+SELECT length(f1) FROM zlibtest;
+
+-- alter compression method with rewrite
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz;
+\d+ zlibtest
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION zlib;
+\d+ zlibtest
+
+-- preserve old compression method
+ALTER TABLE zlibtest ALTER COLUMN f1 SET COMPRESSION pglz PRESERVE (zlib);
+INSERT INTO zlibtest VALUES (repeat('1234567890',1004));
+\d+ zlibtest
+SELECT pg_column_compression(f1) FROM zlibtest;
+SELECT length(f1) FROM zlibtest;
+
+DROP TABLE zlibtest;
--
2.17.0
0011-Support-compression-methods-options.patchtext/x-diff; charset=us-asciiDownload
From cefc9e9428e6aecf58ccfefe31342b6bf97afa8f Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Feb 2021 16:29:55 +0530
Subject: [PATCH 11/12] Support compression methods options
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Tomas Vondra and Robert Haas
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
contrib/cmzlib/cmzlib.c | 75 +++++++++--
doc/src/sgml/ref/alter_table.sgml | 6 +-
doc/src/sgml/ref/create_table.sgml | 6 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/reloptions.c | 64 ++++++++++
src/backend/access/common/toast_internals.c | 14 ++-
src/backend/access/compression/compress_lz4.c | 75 ++++++++++-
.../access/compression/compress_pglz.c | 116 +++++++++++++++---
src/backend/access/table/toast_helper.c | 8 +-
src/backend/bootstrap/bootparse.y | 1 +
src/backend/catalog/heap.c | 15 ++-
src/backend/catalog/index.c | 43 +++++--
src/backend/catalog/toasting.c | 1 +
src/backend/commands/cluster.c | 1 +
src/backend/commands/compressioncmds.c | 79 +++++++++++-
src/backend/commands/foreigncmds.c | 44 -------
src/backend/commands/tablecmds.c | 102 ++++++++++-----
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 16 ++-
src/backend/parser/parse_utilcmd.c | 2 +-
src/bin/pg_dump/pg_dump.c | 23 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/include/access/compressamapi.h | 18 ++-
src/include/access/toast_helper.h | 2 +
src/include/access/toast_internals.h | 2 +-
src/include/catalog/heap.h | 2 +
src/include/catalog/pg_attribute.h | 3 +
src/include/commands/defrem.h | 9 +-
src/include/nodes/parsenodes.h | 1 +
src/test/regress/expected/compression.out | 52 ++++++++
src/test/regress/expected/compression_1.out | 54 ++++++++
src/test/regress/expected/misc_sanity.out | 3 +-
src/test/regress/sql/compression.sql | 18 +++
36 files changed, 727 insertions(+), 141 deletions(-)
diff --git a/contrib/cmzlib/cmzlib.c b/contrib/cmzlib/cmzlib.c
index 686a7c7e0d..d8c2865f21 100644
--- a/contrib/cmzlib/cmzlib.c
+++ b/contrib/cmzlib/cmzlib.c
@@ -14,6 +14,7 @@
#include "postgres.h"
#include "access/compressamapi.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
@@ -45,6 +46,62 @@ typedef struct
unsigned int dictlen;
} zlib_state;
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+zlib_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ {
+ int8 level = pg_atoi(defGetString(def), sizeof(int8), 0);
+
+ if (level < 0 || level > 9)
+ ereport(ERROR,
+ (errcode(ERRCODE_INVALID_PARAMETER_VALUE),
+ errmsg("unexpected value for zlib compression level: \"%s\"",
+ defGetString(def)),
+ errhint("expected value between 0 and 9")
+ ));
+ }
+ else
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for zlib: \"%s\"", def->defname)));
+ }
+}
+
+static void *
+zlib_cminitstate(List *options)
+{
+ zlib_state *state = NULL;
+
+ state = palloc0(sizeof(zlib_state));
+ state->level = Z_DEFAULT_COMPRESSION;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "level") == 0)
+ state->level = pg_atoi(defGetString(def), sizeof(int), 0);
+ }
+ }
+
+ return state;
+}
+
/*
* zlib_cmcompress - compression routine for zlib compression method
*
@@ -52,23 +109,21 @@ typedef struct
* Returns the compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-zlib_cmcompress(const struct varlena *value, int32 header_size)
+zlib_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
- int32 valsize,
- len;
+ int32 valsize,
+ len;
struct varlena *tmp = NULL;
- z_streamp zp;
- int res;
- zlib_state state;
-
- state.level = Z_DEFAULT_COMPRESSION;
+ z_streamp zp;
+ int res;
+ zlib_state *state = (zlib_state *) options;
zp = (z_streamp) palloc(sizeof(z_stream));
zp->zalloc = Z_NULL;
zp->zfree = Z_NULL;
zp->opaque = Z_NULL;
- if (deflateInit(zp, state.level) != Z_OK)
+ if (deflateInit(zp, state->level) != Z_OK)
elog(ERROR, "could not initialize compression library: %s", zp->msg);
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
@@ -146,6 +201,8 @@ zlib_cmdecompress(const struct varlena *value, int32 header_size)
const CompressionAmRoutine zlib_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = zlib_cmcheck,
+ .datum_initstate = zlib_cminitstate,
.datum_compress = zlib_cmcompress,
.datum_decompress = zlib_cmdecompress,
.datum_decompress_slice = NULL};
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 49c43df1c1..0f861e78f9 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,7 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
- ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_method_options</replaceable>) ] [ PRESERVE (<replaceable class="parameter">compression_preserve_list</replaceable>) | PRESERVE ALL ]
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -396,7 +396,9 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
it can be set to any available compression method. The supported buit-in
compression methods are <literal>pglz</literal> and <literal>lz4</literal>.
<literal>lz4</literal> is available only if <literal>--with-lz4</literal>
- was used when building <productname>PostgreSQL</productname>.
+ was used when building <productname>PostgreSQL</productname>. If the
+ compression method has options they can be specified with the <literal>WITH
+ </literal> parameter.
The <literal>PRESERVE</literal> list contains a list of compression
methods used in the column and determines which of them may be kept.
Without <literal>PRESERVE</literal> or if any of the pre-existing
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 5e13b3288c..78210c5461 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -69,7 +69,7 @@ CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXI
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
- COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_options</replaceable>) ] |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -994,7 +994,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
- <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable> [ WITH (<replaceable class="parameter">compression_am_options</replaceable>) ]</literal></term>
<listitem>
<para>
The <literal>COMPRESSION</literal> clause sets the compression method
@@ -1011,6 +1011,8 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
Compression methods can be created with <xref
linkend="sql-create-access-method"/> or it can be set to any available
compression method.
+ If the compression method has options they can be specified with the
+ <literal>WITH </literal> parameter.
The default is <literal>pglz</literal>.
</para>
</listitem>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712c71..76f824bc6f 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -38,6 +38,7 @@
#include "access/toast_internals.h"
#include "access/tupdesc.h"
#include "access/tupmacs.h"
+#include "commands/defrem.h"
#include "utils/datum.h"
#include "utils/memutils.h"
@@ -215,8 +216,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
{
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+ att->attcompression,
+ acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1d43d5d2ff..0d3307e94f 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -22,6 +22,7 @@
#include "access/htup_details.h"
#include "access/itup.h"
#include "access/toast_internals.h"
+#include "commands/defrem.h"
/*
* This enables de-toasting of index entries. Needed until VACUUM is
@@ -104,8 +105,9 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
+ List *acoption = GetAttributeCompressionOptions(att);
Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ att->attcompression, acoption);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/reloptions.c b/src/backend/access/common/reloptions.c
index c687d3ee9e..2dda6c038b 100644
--- a/src/backend/access/common/reloptions.c
+++ b/src/backend/access/common/reloptions.c
@@ -2112,3 +2112,67 @@ AlterTableGetRelOptionsLockLevel(List *defList)
return lockmode;
}
+
+/*
+ * Convert a DefElem list to the text array format that is used in
+ * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
+ * pg_foreign_table.
+ *
+ * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
+ * if the list is empty.
+ *
+ * Note: The array is usually stored to database without further
+ * processing, hence any validation should be done before this
+ * conversion.
+ */
+Datum
+optionListToArray(List *options)
+{
+ ArrayBuildState *astate = NULL;
+ ListCell *cell;
+
+ foreach(cell, options)
+ {
+ DefElem *def = lfirst(cell);
+ const char *value;
+ Size len;
+ text *t;
+
+ value = defGetString(def);
+ len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
+ t = palloc(len + 1);
+ SET_VARSIZE(t, len);
+ sprintf(VARDATA(t), "%s=%s", def->defname, value);
+
+ astate = accumArrayResult(astate, PointerGetDatum(t),
+ false, TEXTOID,
+ CurrentMemoryContext);
+ }
+
+ if (astate)
+ return makeArrayResult(astate, CurrentMemoryContext);
+
+ return PointerGetDatum(NULL);
+}
+
+/*
+ * Return human readable list of reloptions
+ */
+char *
+formatRelOptions(List *options)
+{
+ StringInfoData buf;
+ ListCell *cell;
+
+ initStringInfo(&buf);
+
+ foreach(cell, options)
+ {
+ DefElem *def = (DefElem *) lfirst(cell);
+
+ appendStringInfo(&buf, "%s%s=%s", buf.len > 0 ? ", " : "",
+ def->defname, defGetString(def));
+ }
+
+ return buf.data;
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index a3539065d3..5ed3312f39 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,10 +44,11 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value, Oid cmoid)
+toast_compress_datum(Datum value, Oid cmoid, List *cmoptions)
{
struct varlena *tmp = NULL;
int32 valsize;
+ void *options = NULL;
bool isCustomCompression = false;
const CompressionAmRoutine *cmroutine = NULL;
@@ -71,11 +72,20 @@ toast_compress_datum(Datum value, Oid cmoid)
break;
}
+ if (cmroutine->datum_initstate)
+ options = cmroutine->datum_initstate(cmoptions);
+
/* Call the actual compression function */
tmp = cmroutine->datum_compress((const struct varlena *)value,
isCustomCompression ?
TOAST_CUSTOM_COMPRESS_HDRSZ :
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_HDRSZ, options);
+ if (options != NULL)
+ {
+ pfree(options);
+ list_free(cmoptions);
+ }
+
if (!tmp)
return PointerGetDatum(NULL);
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
index 2a3c162836..7c422fc534 100644
--- a/src/backend/access/compression/compress_lz4.c
+++ b/src/backend/access/compression/compress_lz4.c
@@ -17,9 +17,73 @@
#endif
#include "access/compressamapi.h"
+#include "commands/defrem.h"
#include "fmgr.h"
#include "utils/builtins.h"
+/*
+ * Check options if specified. All validation is located here so
+ * we don't need to do it again in cminitstate function.
+ */
+static void
+lz4_cmcheck(List *options)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ /*
+ * We don't need to check the range for the acceleration parameter
+ * because the LZ4_compress_fast will automatically replace the
+ * values <=0 with LZ4_ACCELERATION_DEFAULT (currently == 1). And,
+ * Values > LZ4_ACCELERATION_MAX with LZ4_ACCELERATION_MAX
+ * (currently == 65537c).
+ */
+ if (strcmp(def->defname, "acceleration") != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for lz4: \"%s\"", def->defname)));
+ }
+#endif
+}
+
+static void *
+lz4_cminitstate(List *options)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 *acceleration = palloc(sizeof(int32));
+
+ /* initialize with the default acceleration */
+ *acceleration = 1;
+
+ if (list_length(options) > 0)
+ {
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ if (strcmp(def->defname, "acceleration") == 0)
+ *acceleration = pg_atoi(defGetString(def), sizeof(int32), 0);
+ }
+ }
+
+ return acceleration;
+#endif
+}
+
/*
* lz4_cmcompress - compression routine for lz4 compression method
*
@@ -27,7 +91,7 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-lz4_cmcompress(const struct varlena *value, int32 header_size)
+lz4_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
#ifndef HAVE_LIBLZ4
ereport(ERROR,
@@ -37,6 +101,7 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
int32 valsize;
int32 len;
int32 max_size;
+ int32 *acceleration = (int32 *) options;
struct varlena *tmp = NULL;
valsize = VARSIZE_ANY_EXHDR(value);
@@ -48,9 +113,9 @@ lz4_cmcompress(const struct varlena *value, int32 header_size)
max_size = LZ4_compressBound(valsize);
tmp = (struct varlena *) palloc(max_size + header_size);
- len = LZ4_compress_default(VARDATA_ANY(value),
- (char *) tmp + header_size,
- valsize, max_size);
+ len = LZ4_compress_fast(VARDATA_ANY(value),
+ (char *) tmp + header_size,
+ valsize, max_size, *acceleration);
if (len <= 0)
elog(ERROR, "could not compress data with lz4");
@@ -146,6 +211,8 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 header_size,
*/
const CompressionAmRoutine lz4_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = lz4_cmcheck,
+ .datum_initstate = lz4_cminitstate,
.datum_compress = lz4_cmcompress,
.datum_decompress = lz4_cmdecompress,
.datum_decompress_slice = lz4_cmdecompress_slice
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
index 7f6e7429fe..1ca912831b 100644
--- a/src/backend/access/compression/compress_pglz.c
+++ b/src/backend/access/compression/compress_pglz.c
@@ -14,11 +14,93 @@
#include "postgres.h"
#include "access/compressamapi.h"
+#include "access/toast_internals.h"
+#include "commands/defrem.h"
#include "common/pg_lzcompress.h"
#include "fmgr.h"
#include "utils/builtins.h"
+#define PGLZ_OPTIONS_COUNT 6
+
+static char *PGLZ_options[PGLZ_OPTIONS_COUNT] = {
+ "min_input_size",
+ "max_input_size",
+ "min_comp_rate",
+ "first_success_by",
+ "match_size_good",
+ "match_size_drop"
+};
+
+/*
+ * Convert value from reloptions to int32, and report if it is not correct.
+ * Also checks parameter names
+ */
+static int32
+parse_option(char *name, char *value)
+{
+ int i;
+
+ for (i = 0; i < PGLZ_OPTIONS_COUNT; i++)
+ {
+ if (strcmp(PGLZ_options[i], name) == 0)
+ return pg_atoi(value, 4, 0);
+ }
+
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_PARAMETER),
+ errmsg("unknown compression option for pglz: \"%s\"", name)));
+}
+
+/*
+ * Check PGLZ options if specified
+ */
+static void
+pglz_cmcheck(List *options)
+{
+ ListCell *lc;
+
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+
+ parse_option(def->defname, defGetString(def));
+ }
+}
+
+/*
+ * Configure PGLZ_Strategy struct for compression function
+ */
+static void *
+pglz_cminitstate(List *options)
+{
+ ListCell *lc;
+ PGLZ_Strategy *strategy = palloc(sizeof(PGLZ_Strategy));
+
+ /* initialize with default strategy values */
+ memcpy(strategy, PGLZ_strategy_default, sizeof(PGLZ_Strategy));
+ foreach(lc, options)
+ {
+ DefElem *def = (DefElem *) lfirst(lc);
+ int32 val = parse_option(def->defname, defGetString(def));
+
+ /* fill the strategy */
+ if (strcmp(def->defname, "min_input_size") == 0)
+ strategy->min_input_size = val;
+ else if (strcmp(def->defname, "max_input_size") == 0)
+ strategy->max_input_size = val;
+ else if (strcmp(def->defname, "min_comp_rate") == 0)
+ strategy->min_comp_rate = val;
+ else if (strcmp(def->defname, "first_success_by") == 0)
+ strategy->first_success_by = val;
+ else if (strcmp(def->defname, "match_size_good") == 0)
+ strategy->match_size_good = val;
+ else if (strcmp(def->defname, "match_size_drop") == 0)
+ strategy->match_size_drop = val;
+ }
+ return (void *) strategy;
+}
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -26,42 +108,41 @@
* compressed varlena, or NULL if compression fails.
*/
static struct varlena *
-pglz_cmcompress(const struct varlena *value, int32 header_size)
+pglz_cmcompress(const struct varlena *value, int32 header_size, void *options)
{
int32 valsize,
len;
- struct varlena *tmp = NULL;
+ struct varlena *tmp = NULL;
+ PGLZ_Strategy *strategy;
valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+ strategy = (PGLZ_Strategy *) options;
/*
* No point in wasting a palloc cycle if value size is outside the allowed
* range for compression.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
+ if (valsize < strategy->min_input_size ||
+ valsize > strategy->max_input_size)
return NULL;
- /*
- * Get maximum size of the compressed data that pglz compression may output
- * and allocate the memory for holding the compressed data and the header.
- */
tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
header_size);
- len = pglz_compress(VARDATA_ANY(value),
+ len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
valsize,
(char *) tmp + header_size,
- NULL);
- if (len < 0)
+ strategy);
+
+ if (len >= 0)
{
- pfree(tmp);
- return NULL;
+ SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ return tmp;
}
- SET_VARSIZE_COMPRESSED(tmp, len + header_size);
+ pfree(tmp);
- return tmp;
+ return NULL;
}
/*
@@ -125,10 +206,11 @@ pglz_cmdecompress_slice(const struct varlena *value, int32 header_size,
*/
const CompressionAmRoutine pglz_compress_methods = {
.type = T_CompressionAmRoutine,
+ .datum_check = pglz_cmcheck,
+ .datum_initstate = pglz_cminitstate,
.datum_compress = pglz_cmcompress,
.datum_decompress = pglz_cmdecompress,
- .datum_decompress_slice = pglz_cmdecompress_slice
-};
+ .datum_decompress_slice = pglz_cmdecompress_slice};
/* pglz compression handler function */
Datum
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index 53f78f9c3e..c9d44c62fa 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -15,11 +15,13 @@
#include "postgres.h"
#include "access/detoast.h"
+#include "access/reloptions.h"
#include "access/table.h"
#include "access/toast_helper.h"
#include "access/toast_internals.h"
#include "catalog/pg_type_d.h"
-
+#include "commands/defrem.h"
+#include "utils/syscache.h"
/*
* Prepare to TOAST a tuple.
@@ -55,6 +57,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
ttc->ttc_attr[i].tai_compression = att->attcompression;
+ ttc->ttc_attr[i].tai_cmoptions = GetAttributeCompressionOptions(att);
if (ttc->ttc_oldvalues != NULL)
{
@@ -230,7 +233,8 @@ toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
- new_value = toast_compress_datum(*value, attr->tai_compression);
+ new_value = toast_compress_datum(*value, attr->tai_compression,
+ attr->tai_cmoptions);
if (DatumGetPointer(new_value) != NULL)
{
diff --git a/src/backend/bootstrap/bootparse.y b/src/backend/bootstrap/bootparse.y
index 5fcd004e1b..a43e0e197c 100644
--- a/src/backend/bootstrap/bootparse.y
+++ b/src/backend/bootstrap/bootparse.y
@@ -239,6 +239,7 @@ Boot_CreateStmt:
true,
false,
InvalidOid,
+ NULL,
NULL);
elog(DEBUG4, "relation created with OID %u", id);
}
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index b53b6b50e6..18b04816f7 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -741,6 +741,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate)
{
TupleTableSlot **slot;
@@ -796,6 +797,11 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
else
slot[slotCount]->tts_isnull[Anum_pg_attribute_attoptions - 1] = true;
+ if (attcmoptions && attcmoptions[natts] != (Datum) 0)
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcmoptions - 1] = attcmoptions[natts];
+ else
+ slot[slotCount]->tts_isnull[Anum_pg_attribute_attcmoptions - 1] = true;
+
/* start out with empty permissions and empty options */
slot[slotCount]->tts_isnull[Anum_pg_attribute_attacl - 1] = true;
slot[slotCount]->tts_isnull[Anum_pg_attribute_attfdwoptions - 1] = true;
@@ -843,6 +849,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
static void
AddNewAttributeTuples(Oid new_rel_oid,
TupleDesc tupdesc,
+ Datum *acoption,
char relkind)
{
Relation rel;
@@ -861,7 +868,8 @@ AddNewAttributeTuples(Oid new_rel_oid,
/* set stats detail level to a sane default */
for (int i = 0; i < natts; i++)
tupdesc->attrs[i].attstattarget = -1;
- InsertPgAttributeTuples(rel, tupdesc, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, tupdesc, new_rel_oid,
+ NULL, acoption, indstate);
/* add dependencies on their datatypes and collations */
for (int i = 0; i < natts; i++)
@@ -893,7 +901,7 @@ AddNewAttributeTuples(Oid new_rel_oid,
td = CreateTupleDesc(lengthof(SysAtt), (FormData_pg_attribute **) &SysAtt);
- InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, indstate);
+ InsertPgAttributeTuples(rel, td, new_rel_oid, NULL, NULL, indstate);
FreeTupleDesc(td);
}
@@ -1160,6 +1168,7 @@ heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress)
{
Relation pg_class_desc;
@@ -1418,7 +1427,7 @@ heap_create_with_catalog(const char *relname,
/*
* now add tuples to pg_attribute for the attributes in our new relation.
*/
- AddNewAttributeTuples(relid, new_rel_desc->rd_att, relkind);
+ AddNewAttributeTuples(relid, new_rel_desc->rd_att, acoptions, relkind);
/*
* Make a dependency link to force the relation to be deleted if its
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 0db2a7430f..ce59075ffd 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -108,10 +108,12 @@ static TupleDesc ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId);
+ Oid *classObjectId,
+ Datum *acoptions);
static void InitializeAttributeOids(Relation indexRelation,
int numatts, Oid indexoid);
-static void AppendAttributeTuples(Relation indexRelation, Datum *attopts);
+static void AppendAttributeTuples(Relation indexRelation, Datum *attopts,
+ Datum *attcmopts);
static void UpdateIndexRelation(Oid indexoid, Oid heapoid,
Oid parentIndexId,
IndexInfo *indexInfo,
@@ -273,7 +275,8 @@ ConstructTupleDescriptor(Relation heapRelation,
List *indexColNames,
Oid accessMethodObjectId,
Oid *collationObjectId,
- Oid *classObjectId)
+ Oid *classObjectId,
+ Datum *acoptions)
{
int numatts = indexInfo->ii_NumIndexAttrs;
int numkeyatts = indexInfo->ii_NumIndexKeyAttrs;
@@ -334,6 +337,9 @@ ConstructTupleDescriptor(Relation heapRelation,
{
/* Simple index column */
const FormData_pg_attribute *from;
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ bool isNull;
Assert(atnum > 0); /* should've been caught above */
@@ -350,6 +356,23 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attstorage = from->attstorage;
to->attalign = from->attalign;
to->attcompression = from->attcompression;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(from->attrelid),
+ Int16GetDatum(from->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ from->attnum, from->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions[i] = PointerGetDatum(NULL);
+ else
+ acoptions[i] =
+ PointerGetDatum(DatumGetArrayTypePCopy(attcmoptions));
+ ReleaseSysCache(attr_tuple);
}
else
{
@@ -490,7 +513,7 @@ InitializeAttributeOids(Relation indexRelation,
* ----------------------------------------------------------------
*/
static void
-AppendAttributeTuples(Relation indexRelation, Datum *attopts)
+AppendAttributeTuples(Relation indexRelation, Datum *attopts, Datum *attcmopts)
{
Relation pg_attribute;
CatalogIndexState indstate;
@@ -508,7 +531,8 @@ AppendAttributeTuples(Relation indexRelation, Datum *attopts)
*/
indexTupDesc = RelationGetDescr(indexRelation);
- InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid, attopts, indstate);
+ InsertPgAttributeTuples(pg_attribute, indexTupDesc, InvalidOid,
+ attopts, attcmopts, indstate);
CatalogCloseIndexes(indstate);
@@ -721,6 +745,7 @@ index_create(Relation heapRelation,
bool concurrent = (flags & INDEX_CREATE_CONCURRENT) != 0;
bool partitioned = (flags & INDEX_CREATE_PARTITIONED) != 0;
char relkind;
+ Datum *acoptions;
TransactionId relfrozenxid;
MultiXactId relminmxid;
@@ -876,6 +901,8 @@ index_create(Relation heapRelation,
indexRelationName, RelationGetRelationName(heapRelation))));
}
+ acoptions = (Datum *) palloc0(sizeof(Datum) * indexInfo->ii_NumIndexAttrs);
+
/*
* construct tuple descriptor for index tuples
*/
@@ -884,7 +911,8 @@ index_create(Relation heapRelation,
indexColNames,
accessMethodObjectId,
collationObjectId,
- classObjectId);
+ classObjectId,
+ acoptions);
/*
* Allocate an OID for the index, unless we were told what to use.
@@ -975,7 +1003,8 @@ index_create(Relation heapRelation,
/*
* append ATTRIBUTE tuples for the index
*/
- AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions);
+ AppendAttributeTuples(indexRelation, indexInfo->ii_OpclassOptions,
+ acoptions);
/* ----------------
* update pg_index
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index a549481557..348b7d3a37 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -260,6 +260,7 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
true,
true,
InvalidOid,
+ NULL,
NULL);
Assert(toast_relid != InvalidOid);
diff --git a/src/backend/commands/cluster.c b/src/backend/commands/cluster.c
index 096a06f7b3..0ad946d802 100644
--- a/src/backend/commands/cluster.c
+++ b/src/backend/commands/cluster.c
@@ -699,6 +699,7 @@ make_new_heap(Oid OIDOldHeap, Oid NewTableSpace, char relpersistence,
true,
true,
OIDOldHeap,
+ NULL,
NULL);
Assert(OIDNewHeap != InvalidOid);
diff --git a/src/backend/commands/compressioncmds.c b/src/backend/commands/compressioncmds.c
index fd6db24e7f..160d64ad32 100644
--- a/src/backend/commands/compressioncmds.c
+++ b/src/backend/commands/compressioncmds.c
@@ -29,6 +29,7 @@
#include "utils/builtins.h"
#include "utils/fmgroids.h"
#include "utils/lsyscache.h"
+#include "utils/syscache.h"
/*
* Get list of all supported compression methods for the given attribute.
@@ -181,7 +182,7 @@ BinaryUpgradeAddPreserve(Form_pg_attribute att, List *preserve)
*/
Oid
GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
- bool *need_rewrite)
+ Datum *acoptions, bool *need_rewrite)
{
Oid cmoid;
char typstorage = get_typstorage(att->atttypid);
@@ -215,6 +216,20 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
errmsg("not built with lz4 support")));
#endif
+ /* if compression options are given then check them */
+ if (compression->options)
+ {
+ CompressionAmRoutine *routine = GetCompressionAmRoutineByAmId(cmoid);
+
+ /* we need routine only to call cmcheck function */
+ if (routine->datum_check != NULL)
+ routine->datum_check(compression->options);
+
+ *acoptions = optionListToArray(compression->options);
+ }
+ else
+ *acoptions = PointerGetDatum(NULL);
+
/*
* Determine if the column needs rewrite or not. Rewrite conditions: SET
* COMPRESSION without PRESERVE - SET COMPRESSION with PRESERVE but not
@@ -286,15 +301,71 @@ GetAttributeCompression(Form_pg_attribute att, ColumnCompression *compression,
* Construct ColumnCompression node from the compression method oid.
*/
ColumnCompression *
-MakeColumnCompression(Oid attcompression)
+MakeColumnCompression(Form_pg_attribute att)
{
ColumnCompression *node;
- if (!OidIsValid(attcompression))
+ if (!OidIsValid(att->attcompression))
return NULL;
node = makeNode(ColumnCompression);
- node->cmname = get_am_name(attcompression);
+ node->cmname = get_am_name(att->attcompression);
+ node->options = GetAttributeCompressionOptions(att);
return node;
}
+
+/*
+ * Fetch atttribute's compression options
+ */
+List *
+GetAttributeCompressionOptions(Form_pg_attribute att)
+{
+ HeapTuple attr_tuple;
+ Datum attcmoptions;
+ List *acoptions;
+ bool isNull;
+
+ attr_tuple = SearchSysCache2(ATTNUM,
+ ObjectIdGetDatum(att->attrelid),
+ Int16GetDatum(att->attnum));
+ if (!HeapTupleIsValid(attr_tuple))
+ elog(ERROR, "cache lookup failed for attribute %d of relation %u",
+ att->attnum, att->attrelid);
+
+ attcmoptions = SysCacheGetAttr(ATTNUM, attr_tuple,
+ Anum_pg_attribute_attcmoptions,
+ &isNull);
+ if (isNull)
+ acoptions = NULL;
+ else
+ acoptions = untransformRelOptions(attcmoptions);
+
+ ReleaseSysCache(attr_tuple);
+
+ return acoptions;
+}
+
+/*
+ * Compare compression options for two columns.
+ */
+void
+CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName)
+{
+ if (strcmp(c1->cmname, c2->cmname))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", c1->cmname, c2->cmname)));
+
+ if (!equal(c1->options, c2->options))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression options conflict",
+ attributeName),
+ errdetail("(%s) versus (%s)",
+ formatRelOptions(c1->options),
+ formatRelOptions(c2->options))));
+}
diff --git a/src/backend/commands/foreigncmds.c b/src/backend/commands/foreigncmds.c
index eb7103fd3b..ae9ae2c51b 100644
--- a/src/backend/commands/foreigncmds.c
+++ b/src/backend/commands/foreigncmds.c
@@ -49,50 +49,6 @@ typedef struct
/* Internal functions */
static void import_error_callback(void *arg);
-
-/*
- * Convert a DefElem list to the text array format that is used in
- * pg_foreign_data_wrapper, pg_foreign_server, pg_user_mapping, and
- * pg_foreign_table.
- *
- * Returns the array in the form of a Datum, or PointerGetDatum(NULL)
- * if the list is empty.
- *
- * Note: The array is usually stored to database without further
- * processing, hence any validation should be done before this
- * conversion.
- */
-static Datum
-optionListToArray(List *options)
-{
- ArrayBuildState *astate = NULL;
- ListCell *cell;
-
- foreach(cell, options)
- {
- DefElem *def = lfirst(cell);
- const char *value;
- Size len;
- text *t;
-
- value = defGetString(def);
- len = VARHDRSZ + strlen(def->defname) + 1 + strlen(value);
- t = palloc(len + 1);
- SET_VARSIZE(t, len);
- sprintf(VARDATA(t), "%s=%s", def->defname, value);
-
- astate = accumArrayResult(astate, PointerGetDatum(t),
- false, TEXTOID,
- CurrentMemoryContext);
- }
-
- if (astate)
- return makeArrayResult(astate, CurrentMemoryContext);
-
- return PointerGetDatum(NULL);
-}
-
-
/*
* Transform a list of DefElem into text array format. This is substantially
* the same thing as optionListToArray(), except we recognize SET/ADD/DROP
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 5fba7352a3..b72da302f9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -609,6 +609,7 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
LOCKMODE parentLockmode;
const char *accessMethod = NULL;
Oid accessMethodId = InvalidOid;
+ Datum *acoptions;
/*
* Truncate relname to appropriate length (probably a waste of time, as
@@ -813,6 +814,8 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
cookedDefaults = NIL;
attnum = 0;
+ acoptions = (Datum *) palloc0(sizeof(Datum) * descriptor->natts);
+
foreach(listptr, stmt->tableElts)
{
ColumnDef *colDef = lfirst(listptr);
@@ -866,8 +869,9 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (relkind == RELKIND_RELATION ||
relkind == RELKIND_PARTITIONED_TABLE ||
relkind == RELKIND_MATVIEW)
- attr->attcompression =
- GetAttributeCompression(attr, colDef->compression, NULL);
+ attr->attcompression = GetAttributeCompression(attr,
+ colDef->compression,
+ &acoptions[attnum - 1], NULL);
else
attr->attcompression = InvalidOid;
}
@@ -921,8 +925,11 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
allowSystemTableMods,
false,
InvalidOid,
+ acoptions,
typaddress);
+ pfree(acoptions);
+
/*
* We must bump the command counter to make the newly-created relation
* tuple visible for opening.
@@ -2432,16 +2439,14 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (OidIsValid(attribute->attcompression))
{
ColumnCompression *compression =
- MakeColumnCompression(attribute->attcompression);
+ MakeColumnCompression(attribute);
if (!def->compression)
def->compression = compression;
- else if (strcmp(def->compression->cmname, compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, compression->cmname)));
+ else
+ CheckCompressionMismatch(def->compression,
+ compression,
+ attributeName);
}
def->inhcount++;
@@ -2478,8 +2483,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
- def->compression = MakeColumnCompression(
- attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2729,14 +2733,9 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
if (!def->compression)
def->compression = newdef->compression;
else if (newdef->compression)
- {
- if (strcmp(def->compression->cmname, newdef->compression->cmname))
- ereport(ERROR,
- (errcode(ERRCODE_DATATYPE_MISMATCH),
- errmsg("column \"%s\" has a compression method conflict",
- attributeName),
- errdetail("%s versus %s", def->compression->cmname, newdef->compression->cmname)));
- }
+ CheckCompressionMismatch(def->compression,
+ newdef->compression,
+ attributeName);
/* Mark the column as locally defined */
def->is_local = true;
@@ -6271,6 +6270,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
AclResult aclresult;
ObjectAddress address;
TupleDesc tupdesc;
+ Datum acoptions = PointerGetDatum(NULL);
FormData_pg_attribute *aattr[] = {&attribute};
/* At top level, permission check was done in ATPrepCmd, else do it */
@@ -6434,6 +6434,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
attribute.attcompression = GetAttributeCompression(&attribute,
colDef->compression,
+ &acoptions,
NULL);
else
attribute.attcompression = InvalidOid;
@@ -6444,7 +6445,7 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
tupdesc = CreateTupleDesc(lengthof(aattr), (FormData_pg_attribute **) &aattr);
- InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, NULL);
+ InsertPgAttributeTuples(attrdesc, tupdesc, myrelid, NULL, &acoptions, NULL);
table_close(attrdesc, RowExclusiveLock);
@@ -7840,13 +7841,20 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
* the respective input values are valid.
*/
static void
-ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
- Oid newcompression, char newstorage, LOCKMODE lockmode)
+ApplyChangesToIndexes(Relation rel, Relation attrel, AttrNumber attnum,
+ Oid newcompression, char newstorage, Datum acoptions,
+ LOCKMODE lockmode)
{
HeapTuple tuple;
ListCell *lc;
Form_pg_attribute attrtuple;
+ /*
+ * Compression option can only be valid if we are updating the compression
+ * method.
+ */
+ Assert(DatumGetPointer(acoptions) == NULL || OidIsValid(newcompression));
+
foreach(lc, RelationGetIndexList(rel))
{
Oid indexoid = lfirst_oid(lc);
@@ -7882,7 +7890,29 @@ ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
if (newstorage != '\0')
attrtuple->attstorage = newstorage;
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+ if (DatumGetPointer(acoptions) != NULL)
+ {
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ HeapTuple newtuple;
+
+ /* Initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+
+ heap_freetuple(newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -7973,7 +8003,7 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* matching behavior of index.c ConstructTupleDescriptor()).
*/
ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
- lockmode);
+ PointerGetDatum(NULL), lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15145,9 +15175,11 @@ ATExecSetCompression(AlteredTableInfo *tab,
char typstorage;
Oid cmoid;
bool need_rewrite;
+ HeapTuple newtuple;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
+ Datum acoptions;
ObjectAddress address;
attrel = table_open(AttributeRelationId, RowExclusiveLock);
@@ -15182,7 +15214,8 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* Get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression, &need_rewrite);
+ cmoid = GetAttributeCompression(atttableform, compression, &acoptions,
+ &need_rewrite);
if (atttableform->attcompression != cmoid)
add_column_compression_dependency(atttableform->attrelid,
@@ -15192,8 +15225,17 @@ ATExecSetCompression(AlteredTableInfo *tab,
atttableform->attcompression = cmoid;
- atttableform->attcompression = cmoid;
- CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+ /* update an existing entry */
+ if (acoptions)
+ {
+ values[Anum_pg_attribute_attcmoptions - 1] = acoptions;
+ replace[Anum_pg_attribute_attcmoptions - 1] = true;
+ newtuple = heap_modify_tuple(tuple, RelationGetDescr(attrel),
+ values, nulls, replace);
+ CatalogTupleUpdate(attrel, &newtuple->t_self, newtuple);
+ }
+ else
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
RelationGetRelid(rel),
@@ -15201,8 +15243,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
- /* apply changes to the index column as well */
- ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ /* Apply the change to indexes as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', acoptions,
+ lockmode);
+
table_close(attrel, RowExclusiveLock);
/* make changes visible */
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 6a11f8eb60..991340e2ad 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2993,6 +2993,7 @@ _copyColumnCompression(const ColumnCompression *from)
COPY_STRING_FIELD(cmname);
COPY_SCALAR_FIELD(preserve_all);
+ COPY_NODE_FIELD(options);
COPY_NODE_FIELD(preserve);
return newnode;
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index 26a9b85974..57a2975da1 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2624,6 +2624,7 @@ _equalColumnCompression(const ColumnCompression *a, const ColumnCompression *b)
{
COMPARE_STRING_FIELD(cmname);
COMPARE_SCALAR_FIELD(preserve_all);
+ COMPARE_NODE_FIELD(options);
COMPARE_NODE_FIELD(preserve);
return true;
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index b584a58ba3..bc5c64df1e 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2888,6 +2888,7 @@ _outColumnCompression(StringInfo str, const ColumnCompression *node)
WRITE_STRING_FIELD(cmname);
WRITE_BOOL_FIELD(preserve_all);
+ WRITE_NODE_FIELD(options);
WRITE_NODE_FIELD(preserve);
}
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index b22ce818cd..00fe3b7a5f 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -415,6 +415,7 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
relation_expr_list dostmt_opt_list
transform_element_list transform_type_list
TriggerTransitions TriggerReferencing
+ optCompressionParameters
vacuum_relation_list opt_vacuum_relation_list
drop_option_list
@@ -3503,11 +3504,17 @@ compressionClause:
COMPRESSION name { $$ = pstrdup($2); }
;
+optCompressionParameters:
+ WITH '(' generic_option_list ')' { $$ = $3; }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
optColumnCompression:
- compressionClause
+ compressionClause optCompressionParameters
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
+ n->options = (List *) $2;
n->preserve = NIL;
$$ = (Node *) n;
}
@@ -3515,14 +3522,15 @@ optColumnCompression:
;
alterColumnCompression:
- compressionClause optCompressionPreserve
+ compressionClause optCompressionParameters optCompressionPreserve
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
- n->preserve = (List *) $2;
+ n->options = (List *) $2;
+ n->preserve = (List *) $3;
$$ = (Node *) n;
}
- | compressionClause PRESERVE ALL
+ | compressionClause optCompressionParameters PRESERVE ALL
{
ColumnCompression *n = makeNode(ColumnCompression);
n->cmname = $1;
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index f50b50cc8e..24e192175e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1086,7 +1086,7 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
/* Likewise, copy compression if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
&& OidIsValid(attribute->attcompression))
- def->compression = MakeColumnCompression(attribute->attcompression);
+ def->compression = MakeColumnCompression(attribute);
else
def->compression = NULL;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 88fa7e1ed3..f9517549f9 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8709,10 +8709,17 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
if (createWithCompression)
appendPQExpBuffer(q,
- "am.amname AS attcmname,\n");
+ "am.amname AS attcmname,\n"
+ "pg_catalog.array_to_string(ARRAY("
+ "SELECT pg_catalog.quote_ident(option_name) || "
+ "' ' || pg_catalog.quote_literal(option_value) "
+ "FROM pg_catalog.pg_options_to_table(a.attcmoptions) "
+ "ORDER BY option_name"
+ "), E',\n ') AS attcmoptions,\n");
else
appendPQExpBuffer(q,
- "NULL AS attcmname,\n");
+ "NULL AS attcmname,\n"
+ "NULL AS attcmoptions,\n");
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
@@ -8765,6 +8772,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8794,6 +8802,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
+ tbinfo->attcmoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmoptions")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15959,7 +15968,8 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
continue;
has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+ ((strcmp(tbinfo->attcmnames[j], "pglz") != 0) ||
+ nonemptyReloptions(tbinfo->attcmoptions[j])));
/* Format properly if not first attr */
if (actual_atts == 0)
@@ -16005,6 +16015,10 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
+
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, " WITH (%s)",
+ tbinfo->attcmoptions[j]);
}
if (print_default)
@@ -16436,6 +16450,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s",
qualrelname, fmtId(tbinfo->attnames[j]), tbinfo->attcmnames[j]);
+ if (nonemptyReloptions(tbinfo->attcmoptions[j]))
+ appendPQExpBuffer(q, "\nWITH (%s) ", tbinfo->attcmoptions[j]);
+
if (cminfo->nitems > 0)
{
appendPQExpBuffer(q, "\nPRESERVE (");
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index a829528cd0..da47f3173f 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -327,6 +327,7 @@ typedef struct _tableInfo
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
char **attcmnames; /* per-attribute current compression method */
+ char **attcmoptions; /* per-attribute compression options */
struct _attrCompressionInfo **attcompression; /* per-attribute all
compression data */
/*
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index bac7b47cfc..7f3346b2a6 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -51,8 +51,11 @@ extern Oid GetDefaultToastCompression(void);
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
+typedef void (*cmcheck_function) (List *options);
+typedef void *(*cminitstate_function) (List *options);
typedef struct varlena *(*cmcompress_function) (const struct varlena *value,
- int32 toast_header_size);
+ int32 toast_header_size,
+ void *options);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value,
int32 toast_header_size);
typedef struct varlena *(*cmdecompress_slice_function)
@@ -63,14 +66,25 @@ typedef struct varlena *(*cmdecompress_slice_function)
/*
* API struct for a compression AM.
*
+ * 'cmcheck' - called when attribute is linking with compression method.
+ * This function should check compability of compression method with
+ * the attribute and its options.
+ *
+ * 'cminitstate' - called when CompressionAmOptions instance is created.
+ * Should return pointer to a memory in a caller memory context, or NULL.
+ * Could be used to pass some internal state between compression function
+ * calls, like internal structure for parsed compression options.
+ *
* 'datum_compress' - varlena compression function.
* 'datum_decompress' - varlena decompression function.
* 'datum_decompress_slice' - varlena slice decompression functions.
*/
typedef struct CompressionAmRoutine
{
- NodeTag type;
+ NodeTag type;
+ cmcheck_function datum_check; /* can be NULL */
+ cminitstate_function datum_initstate; /* can be NULL */
cmcompress_function datum_compress;
cmdecompress_function datum_decompress;
cmdecompress_slice_function datum_decompress_slice;
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index dca0bc37f3..fe8de405ac 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -15,6 +15,7 @@
#define TOAST_HELPER_H
#include "utils/rel.h"
+#include "nodes/pg_list.h"
/*
* Information about one column of a tuple being toasted.
@@ -33,6 +34,7 @@ typedef struct
int32 tai_size;
uint8 tai_colflags;
Oid tai_compression;
+ List *tai_cmoptions;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index ac28f9ed55..864de31739 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -54,7 +54,7 @@ typedef struct toast_compress_header_custom
#define TOAST_COMPRESS_SET_CMOID(ptr, oid) \
(((toast_compress_header_custom *)(ptr))->cmoid = (oid))
-extern Datum toast_compress_datum(Datum value, Oid cmoid);
+extern Datum toast_compress_datum(Datum value, Oid cmoid, List *cmoptions);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/heap.h b/src/include/catalog/heap.h
index 6ce480b49c..5ecb6f3653 100644
--- a/src/include/catalog/heap.h
+++ b/src/include/catalog/heap.h
@@ -81,6 +81,7 @@ extern Oid heap_create_with_catalog(const char *relname,
bool allow_system_table_mods,
bool is_internal,
Oid relrewrite,
+ Datum *acoptions,
ObjectAddress *typaddress);
extern void heap_drop_with_catalog(Oid relid);
@@ -97,6 +98,7 @@ extern void InsertPgAttributeTuples(Relation pg_attribute_rel,
TupleDesc tupdesc,
Oid new_rel_oid,
Datum *attoptions,
+ Datum *attcmoptions,
CatalogIndexState indstate);
extern void InsertPgClassTuple(Relation pg_class_desc,
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 797e78b17c..7c74bc314e 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -178,6 +178,9 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* Column-level FDW options */
text attfdwoptions[1] BKI_DEFAULT(_null_);
+ /* current compression options */
+ text attcmoptions[1] BKI_DEFAULT(_null_);
+
/*
* Missing value for added columns. This is a one element array which lets
* us store a value of the attribute type here.
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index bd53f9bb0f..8271e97dea 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -134,6 +134,8 @@ extern Datum transformGenericOptions(Oid catalogId,
Datum oldOptions,
List *options,
Oid fdwvalidator);
+extern Datum optionListToArray(List *options);
+extern char *formatRelOptions(List *options);
/* commands/amcmds.c */
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
@@ -146,9 +148,12 @@ extern char *get_am_name(Oid amOid);
/* commands/compressioncmds.c */
extern Oid GetAttributeCompression(Form_pg_attribute att,
ColumnCompression *compression,
- bool *need_rewrite);
-extern ColumnCompression *MakeColumnCompression(Oid atttcompression);
+ Datum *acoptions, bool *need_rewrite);
+extern ColumnCompression *MakeColumnCompression(Form_pg_attribute att);
extern bool IsCompressionSupported(Form_pg_attribute att, Oid cmoid);
+extern List *GetAttributeCompressionOptions(Form_pg_attribute att);
+extern void CheckCompressionMismatch(ColumnCompression *c1, ColumnCompression *c2,
+ const char *attributeName);
/* support routines in commands/define.c */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ce0913e18a..c3c878b80e 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -634,6 +634,7 @@ typedef struct ColumnCompression
NodeTag type;
char *cmname;
bool preserve_all;
+ List *options;
List *preserve;
} ColumnCompression;
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index e52e272d39..4af72bdbdc 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -297,6 +297,58 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+-------------------
+ {acceleration=50}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=200}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index d4e215c931..211fbfc0bb 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -279,6 +279,60 @@ SELECT pg_column_compression(f1) FROM cmdata;
Indexes:
"idx" btree (f1)
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+ERROR: not built with lz4 support
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+ERROR: "lz4" compression access method cannot be preserved
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+ attcmoptions
+----------------------
+ {min_input_size=100}
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+ pglz
+(3 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/misc_sanity.out b/src/test/regress/expected/misc_sanity.out
index 9ebe28a78d..4e3b0c13c1 100644
--- a/src/test/regress/expected/misc_sanity.out
+++ b/src/test/regress/expected/misc_sanity.out
@@ -99,6 +99,7 @@ ORDER BY 1, 2;
relname | attname | atttypid
-------------------------+---------------+--------------
pg_attribute | attacl | aclitem[]
+ pg_attribute | attcmoptions | text[]
pg_attribute | attfdwoptions | text[]
pg_attribute | attmissingval | anyarray
pg_attribute | attoptions | text[]
@@ -109,7 +110,7 @@ ORDER BY 1, 2;
pg_index | indpred | pg_node_tree
pg_largeobject | data | bytea
pg_largeobject_metadata | lomacl | aclitem[]
-(11 rows)
+(12 rows)
-- system catalogs without primary keys
--
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 9d2e72b784..17320c9062 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -115,6 +115,24 @@ ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4 PRESERVE (pglz2);
SELECT pg_column_compression(f1) FROM cmdata;
\d+ cmdata
+-- compression options
+CREATE TABLE cmdata3(f1 TEXT COMPRESSION pglz WITH (min_input_size '100'));
+CREATE INDEX idx1 ON cmdata3(f1);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1000));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION lz4 WITH (acceleration '50');
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1004));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+
+ALTER TABLE cmdata3 ALTER COLUMN f1 SET COMPRESSION pglz WITH (min_input_size '200') PRESERVE (lz4);
+INSERT INTO cmdata3 VALUES(repeat('1234567890',1008));
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'cmdata3'::regclass AND attname='f1';
+SELECT attcmoptions FROM pg_attribute WHERE attrelid = 'idx1'::regclass AND attname='f1';
+SELECT pg_column_compression(f1) FROM cmdata3;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
Re: [HACKERS] Custom compression methods
On Fri, Feb 19, 2021 at 2:43 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
I had an off list discussion with Robert and based on his suggestion
and a poc patch, I have come up with an updated version for handling
the composite type. Basically, the problem was that ExecEvalRow we
are first forming the tuple and then we are calling
HeapTupleHeaderGetDatum and then we again need to deform to find any
compressed data so that can cause huge performance penalty in all
unrelated paths which don't even contain any compressed data. So
Robert's idea was to check for the compressed/external data even
before forming the tuple. I have implemented that and I can see we
are not seeing any performance penalty.
Test setup:
----------------
create table t1 (f1 int, f2 text, f3 text, f4 text, f5 text, f6
text,f7 text, f8 text, f9 text);
create table t2 (f1 int, f2 text, f3 text, f4 text, f5 text, f6
text,f7 text, f8 text, f9 text);
create table t3(x t1);
pgbench custom script for all test:
------------------------------------------------
\set x random(1, 10000)
select row(f1,f2,f3,f4,f5,f6,f7,f8,f9)::t1 from t2 where f1=:x;
test1:
Objective: Just select on data and form row, data contain no
compressed/external (should not create regression on unrelated paths)
data: insert into t2 select i, repeat('f1',
10),repeat('f2',10),repeat('f3', 10),repeat('f4', 10),repeat('f5',
10),repeat('f6',10),repeat('f7', 10),repeat('f8', 10) from
generate_series(1,10000) as i;
Result(TPS): Head: 1509.79 Patch: 1509.67
test2: data contains 1 compressed filed no external data
data: insert into t2 select i, repeat('f2',
10),repeat('f3',10000),repeat('f3', 10),repeat('f5', 10),repeat('f6',
4000),repeat('f7',10),repeat('f8', 10),repeat('f9', 10) from
generate_series(1,10000) as i;
Result(TPS): Head: 1088.08 Patch: 1071.48
test4: data contains 1 compressed/1 external field
(alter table t2 alter COLUMN f2 set storage external;)
data: (insert into t2 select i, repeat('f2',
10000),repeat('f3',10000),repeat('f3', 10),repeat('f5',
10),repeat('f6', 4000),repeat('f7',10),repeat('f8', 10),repeat('f9',
10) from generate_series(1,10000) as i;)
Result(TPS): Head: 1459.28 Patch: 1459.37
test5: where head need not decompress but patch needs to:
data: insert into t2 select i, repeat('f2',
10),repeat('f3',6000),repeat('f34', 5000),repeat('f5',
10),repeat('f6', 4000),repeat('f7',10),repeat('f8', 10),repeat('f9',
10) from generate_series(1,10000) as I;
--pgbench script
\set x random(1, 10000)
insert into t3 select row(f1,f2,f3,f4,f5,f6,f7,f8,f9)::t1 from t2 where f1=:x;
Result(TPS): Head: 562.36 Patch: 469.91
Summary: It seems like in most of the unrelated cases we are not
creating any regression with the attached patch. There is only some
performance loss when there is only the compressed data in such cases
with the patch we have to decompress whereas in head we don't. But, I
think it is not a overall loss because eventually if we have to fetch
the data multiple time then with patch we just have to decompress once
as whole row is compressed whereas on head we have to decompress field
by field, so I don't think this can be considered as a regression.
I also had to put the handling in the extended record so that it can
decompress any compressed data in the extended record. I think I need
to put some more effort into cleaning up this code. I have put a very
localized fix in ER_get_flat_size, basically this will ignore the
ER_FLAG_HAVE_EXTERNAL flag and it will always process the record. I
think the handling might not be perfect but I posted it to get the
feedback on the idea.
Other changes:
- I have fixed other pending comments from Robert. I will reply to
individual comments in a separate mail.
- Merge HIDE_COMPRESSAM with 0001.
Pending work:
- Cleanup 0001, especially for extended records.
- Rebased other patches.
- Review default compression method guc from Justin
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v25-0001-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v25-0001-Disallow-compressed-data-inside-container-types.patchDownload
From e6aaafe34f9fe4cbecbba2403a3d6248a12ca88d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 19 Feb 2021 17:57:05 +0530
Subject: [PATCH v25 1/3] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 15 ++++++-
src/backend/utils/adt/expandedrecord.c | 76 ++++++++++++----------------------
3 files changed, 43 insertions(+), 52 deletions(-)
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..5684a6d 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !(VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(op->d.row.elemvalues[i])) ||
+ VARATT_IS_COMPRESSED((struct varlena *) DatumGetPointer(op->d.row.elemvalues[i]))))
+ continue;
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = PointerGetDatum(tuple->t_data);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..66031a5 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
--
1.8.3.1
v25-0003-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v25-0003-default-to-with-lz4.patchDownload
From f7bd76ab6361ca50edae545297c30ec0709bdbe5 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v25 3/3] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b7..62180c5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v25-0002-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v25-0002-Built-in-compression-method.patchDownload
From babdf8c3320fd8527f5dae4b041cfe0e09adfb6a Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v25 2/3] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 95 ++++++++----
src/backend/access/common/indextuple.c | 4 +-
src/backend/access/common/toast_internals.c | 63 ++++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 142 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 2 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 ++
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/executor/nodeModifyTable.c | 129 ++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 8 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 ++++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 29 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 ++
src/include/access/compressamapi.h | 99 ++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 19 +--
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 187 +++++++++++++++++++++++
src/test/regress/expected/psql.out | 68 +++++----
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
66 files changed, 1776 insertions(+), 126 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..66dcb1b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..946770d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,77 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
+
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ return cmroutine;
+}
- return result;
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +542,9 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1d43d5d 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -16,6 +16,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/heaptoast.h"
#include "access/htup_details.h"
@@ -103,7 +104,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..bd25d8c
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..11405e9
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,142 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..b53b6b5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4ab0b8..0db2a74 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -24,6 +24,7 @@
#include <unistd.h>
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/heapam.h"
#include "access/multixact.h"
#include "access/reloptions.h"
@@ -348,6 +349,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b2457a6..811dbc3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..be8a8fe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,120 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+
+ }
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2360,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..e262ec5 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..46044cb 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..1789e18 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b835b0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,21 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2036,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2117,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a818ee5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..9755e8e 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..554b643 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,12 @@ show_context_hook(const char *newval)
}
static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..4e4d5e1
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..a061ea2 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,22 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710..4b3d34a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 943931f..182b77a 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..8e40d93
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..1e0864d
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,187 @@
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..14f3fa3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..07fd3f6 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..7970f6e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM false
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..46cac11 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Thu, Feb 11, 2021 at 1:37 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Feb 10, 2021 at 9:52 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
[ new patches ]
I think that in both varattrib_4b and toast_internals.h it would be
better to pick a less generic field name. In toast_internals.h it's
just info; in postgres.h it's va_info. But:[rhaas pgsql]$ git grep info | wc -l
24552There are no references in the current source tree to va_info, so at
least that one is greppable, but it's still not very descriptive. I
suggest info -> tcinfo and va_info -> va_tcinfo, where "tc" stands for
"TOAST compression". Looking through 24552 references to info to find
the ones that pertain to this feature might take longer than searching
the somewhat shorter list of references to tcinfo, which prepatch is
just:[rhaas pgsql]$ git grep tcinfo | wc -l
0
Done as suggested
I don't see why we should allow for datum_decompress to be optional,
as toast_decompress_datum_slice does. Likely every serious compression
method will support that anyway. If not, the compression AM can deal
with the problem, rather than having the core code do it. That will
save some tiny amount of performance, too.
Done
src/backend/access/compression/Makefile is missing a copyright header.
Fixed
It's really sad that lz4_cmdecompress_slice allocates
VARRAWSIZE_4B_C(value) + VARHDRSZ rather than slicelength + VARHDRSZ
as pglz_cmdecompress_slice() does. Is that a mistake, or is that
necessary for some reason? If it's a mistake, let's fix it. If it's
necessary, let's add a comment about why, probably starting with
"Unfortunately, ....".
In older versions of the lz4 there was a problem that the decompressed
data size could be bigger than the slicelength which is resolved now
so we can allocate slicelength + VARHDRSZ, I have fixed it.
Please refer the latest patch at
/messages/by-id/CAFiTN-u2pyXDDDwZXJ-fVUwbLhJSe9TbrVR6rfW_rhdyL1A5bg@mail.gmail.com
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Feb 11, 2021 at 3:26 AM Robert Haas <robertmhaas@gmail.com> wrote:
In CompareCompressionMethodAndDecompress, I think this is still
playing a bit fast and loose with the rules around slots. I think we
can do better. Suppose that at the point where we discover that we
need to decompress at least one attribute, we create the new slot
right then, and also memcpy tts_values and tts_isnull. Then, for that
attribute and any future attributes that need decompression, we reset
tts_values in the *new* slot, leaving the old one untouched. Then,
after finishing all the attributes, the if (decompressed_any) block,
you just have a lot less stuff to do. The advantage of this is that
you haven't tainted the old slot; it's still got whatever contents it
had before, and is in a clean state, which seems better to me.
Fixed
It's unclear to me whether this function actually needs to
ExecMaterializeSlot(newslot). It definitely does need to
ExecStoreVirtualTuple(newslot) and I think it's a very good idea, if
not absolutely mandatory, for it not to modify anything about the old
slot. But what's the argument that the new slot needs to be
materialized at this point? It may be needed, if the old slot would've
had to be materialized at this point. But it's something to think
about.
I think if the original slot was materialized then materialing the new
slot make more sense to me so done that way.
The CREATE TABLE documentation says that COMPRESSION is a kind of
column constraint, but that's wrong. For example, you can't write
CREATE TABLE a (b int4 CONSTRAINT thunk COMPRESSION lz4), for example,
contrary to what the syntax summary implies. When you fix this so that
the documentation matches the grammar change, you may also need to
move the longer description further up in create_table.sgml so the
order matches.
Fixed
The use of VARHDRSZ_COMPRESS in toast_get_compression_oid() appears to
be incorrect. VARHDRSZ_COMPRESS is offsetof(varattrib_4b,
va_compressed.va_data). But what gets externalized in the case of a
compressed datum is just VARDATA(dval), which excludes the length
word, unlike VARHDRSZ_COMPRESS, which does not. This has no
consequences since we're only going to fetch 1 chunk either way, but I
think we should make it correct.
Fixed
TOAST_COMPRESS_SET_SIZE_AND_METHOD() could Assert something about cm_method.
While replying to the comments, I realised that I have missed it. I
will fix it in the next version.
Small delta patch with a few other suggested changes attached.
Merged
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Feb 19, 2021 at 11:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I had an off list discussion with Robert and based on his suggestion
and a poc patch, I have come up with an updated version for handling
the composite type. Basically, the problem was that ExecEvalRow we
are first forming the tuple and then we are calling
HeapTupleHeaderGetDatum and then we again need to deform to find any
compressed data so that can cause huge performance penalty in all
unrelated paths which don't even contain any compressed data. So
Robert's idea was to check for the compressed/external data even
before forming the tuple. I have implemented that and I can see we
are not seeing any performance penalty.
I think that these performance tests aren't really exercising the
expanded-record stuff, just the ExecEvalRow changes. We need to test
that test case, and I tend to suspect there's going to be a measurable
regression.
I spent some time looking at how datums get into the expanded record
system. There seem to be four possible paths:
expanded_record_set_tuple(), make_expanded_record_from_datum(),
expanded_record_set_field_internal(), and
expanded_record_set_fields(). The first two of these inject an entire
tuple, while the latter two work on a field-by-field basis. For that
reason, the latter two are not really problematic. I'm not quite sure
what the right thing to do is here, but if we wanted to check whether
a Datum that we're absorbing is non-pglz-compressed in those places,
it would be easy to do. Also, as far as I can see,
make_expanded_record_from_datum() is completely unused. So the problem
case is where expanded_record_set_tuple() is getting called, and
specifically where it's being called with expand_external = true. Any
place that it's being called with expand_external = false, there's
apparently no problem with the result tuple containing external
datums, so probably non-pglz compressed data is OK there too.
All of the places that can potentially pass expand_external = true are
actually passing !estate->atomic, where estate is a PLpgSQL_execstate.
In other words, I think the case where this happens is when we're in a
context where the computed value could need to survive across a COMMIT
or ROLLBACK, like there may be a procedure running (maybe more than
one, each invoking the next via CALL) but there are no queries in
progress. We have to expand TOAST references because committing a
transaction that deleted data means you might not be able to resolve
the old TOAST pointer any more: even if you use a snapshot that can
see everything, VACUUM could nuke the deleted rows - or some of them -
at any time. To avoid trouble we have to un-externalize before any
COMMIT or ROLLBACK occurs. That can suck for performance because we
might be fetching a big value that we don't end up using for anything
- say if the variable isn't used again - but it beats failing.
The argument that we need to force decompression in such cases is
considerably more tenuous. It revolves around the possibility that the
compression AM itself has been dropped. As long as we have only
built-in compression methods, which are undroppable, it seems like we
could potentially just decide to do nothing at all about this issue.
If the only reason for expanding TOAST pointers inside the
expanded-record stuff is to avoid the possibility of one being
invalidated by a transaction commit, and if compression methods can't
be invalidated by a transaction commit, well then we don't really have
a problem. That's not a great solution in terms of our chances of
getting this whole patch series committed, but it might at least be
enough to unblock the first few patches, and we could document the
rest of the issue for later research.
What makes me a bit uncomfortable about that approach is that it
presupposes that everything that uses expanded records has some other
defense against those tuples getting written to disk without first
expanding any external datums. And it isn't obvious that this is the
case, or at least not to me. For example, PLpgsql's
coerce_function_result_tuple()'s code for
VARATT_IS_EXTERNAL_EXPANDED() has three cases. The first case passes
the tuple through SPI_returntuple() which calls
heap_copy_tuple_as_datum() which calls toast_flatten_tuple_to_datum()
if required, but the second case calls EOH_flatten_into() and does NOT
pass the result through SPI_returntuple(). And ER_flatten_info() has
no defense against this case that I can see: sure, it skips the fast
path if ER_FLAG_HAVE_EXTERNAL is set, but that doesn't actually do
anything to resolve TOAST pointers. Maybe there's no bug there for
some reason, but I don't know what that reason might be. We seem to
have no test cases either in the main test suite or in the plpgsql
test suite where ER_flatten_info gets called with
ER_FLAG_HAVE_EXTERNAL is set, which seems a little unfortunate. If
there is such a bug here it's independent of this patch, I suppose,
but it would still be nice to understand what's going on here better
than I do.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Feb 19, 2021 at 4:21 PM Robert Haas <robertmhaas@gmail.com> wrote:
What makes me a bit uncomfortable about that approach is that it
presupposes that everything that uses expanded records has some other
defense against those tuples getting written to disk without first
expanding any external datums. And it isn't obvious that this is the
case, or at least not to me. For example, PLpgsql's
coerce_function_result_tuple()'s code for
VARATT_IS_EXTERNAL_EXPANDED() has three cases. The first case passes
the tuple through SPI_returntuple() which calls
heap_copy_tuple_as_datum() which calls toast_flatten_tuple_to_datum()
if required, but the second case calls EOH_flatten_into() and does NOT
pass the result through SPI_returntuple(). And ER_flatten_info() has
no defense against this case that I can see: sure, it skips the fast
path if ER_FLAG_HAVE_EXTERNAL is set, but that doesn't actually do
anything to resolve TOAST pointers. Maybe there's no bug there for
some reason, but I don't know what that reason might be. We seem to
have no test cases either in the main test suite or in the plpgsql
test suite where ER_flatten_info gets called with
ER_FLAG_HAVE_EXTERNAL is set, which seems a little unfortunate. If
there is such a bug here it's independent of this patch, I suppose,
but it would still be nice to understand what's going on here better
than I do.
Andres just pointed out to me the error of my thinking here:
ER_flatten_into can *never* encounter a case with both
ER_FLAG_FVALUE_VALID and ER_FLAG_HAVE_EXTERNAL, because
ER_get_flat_size has to get called first, and will de-toast external
values as it goes. So there actually is justification for
coerce_function_result_tuple() to skip the call to SPI_returntuple().
Given that, one might wonder why the test in ER_flatten_into() even
cares about ER_FLAG_HAVE_EXTERNAL in the first place... I suppose it's
just a harmless oversight.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Feb 20, 2021 at 2:51 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Feb 19, 2021 at 11:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I think that these performance tests aren't really exercising the
expanded-record stuff, just the ExecEvalRow changes. We need to test
that test case, and I tend to suspect there's going to be a measurable
regression.
I will do testing around this area.
I spent some time looking at how datums get into the expanded record
system. There seem to be four possible paths:
expanded_record_set_tuple(), make_expanded_record_from_datum(),
expanded_record_set_field_internal(), and
expanded_record_set_fields(). The first two of these inject an entire
tuple, while the latter two work on a field-by-field basis. For that
reason, the latter two are not really problematic. I'm not quite sure
what the right thing to do is here, but if we wanted to check whether
a Datum that we're absorbing is non-pglz-compressed in those places,
it would be easy to do. Also, as far as I can see,
make_expanded_record_from_datum() is completely unused. So the problem
case is where expanded_record_set_tuple() is getting called, and
specifically where it's being called with expand_external = true. Any
place that it's being called with expand_external = false, there's
apparently no problem with the result tuple containing external
datums, so probably non-pglz compressed data is OK there too.
All of the places that can potentially pass expand_external = true are
actually passing !estate->atomic, where estate is a PLpgSQL_execstate.
In other words, I think the case where this happens is when we're in a
context where the computed value could need to survive across a COMMIT
or ROLLBACK, like there may be a procedure running (maybe more than
one, each invoking the next via CALL) but there are no queries in
progress. We have to expand TOAST references because committing a
transaction that deleted data means you might not be able to resolve
the old TOAST pointer any more: even if you use a snapshot that can
see everything, VACUUM could nuke the deleted rows - or some of them -
at any time. To avoid trouble we have to un-externalize before any
COMMIT or ROLLBACK occurs. That can suck for performance because we
might be fetching a big value that we don't end up using for anything
- say if the variable isn't used again - but it beats failing.
I agree with most of this, but I don't think the only reason to
un-externalize is just for COMMIT or ROLLBACK. I mean using the
trigger function we might insert a RECORD type to another table which
has the ROWTYPE as the table on which we are doing the operation. See
below example
CREATE TABLE t1(a int, b varchar compression lz4);
INSERT INTO t1 select 1, repeat('a', 3000);
CREATE TABLE t2 (x t1 compression pglz);
CREATE OR REPLACE FUNCTION log_last_name_changes()
RETURNS TRIGGER
LANGUAGE PLPGSQL
AS
$$
BEGIN
INSERT INTO t2 select OLD;
RETURN NEW;
END;
$$;
CREATE TRIGGER last_name_changes
BEFORE UPDATE
ON t1
FOR EACH ROW
EXECUTE PROCEDURE log_last_name_changes();
UPDATE t1 SET a=2;
SELECT pg_column_compression((t2.x).b) FROM t2;
pg_column_compression
-----------------------
lz4
(1 row)
So basically, in this case, we are not un-externalizing because of
ROLLBACK or COMMIT, instead, we are doing that because we want to
insert it into the new table. So this is without my patch and without
my patch (v25_0001_Disallow_compressed_data_inside_container_types,
basically without the changes in expandedrecord.c). Here is the call
stack when exactly this tuple gets flattened.
#0 expanded_record_set_field_internal (erh=0x2bbcfb0, fnumber=2,
newValue=45863112, isnull=false, expand_external=true,
check_constraints=false) at expandedrecord.c:1225
#1 0x00000000009a1899 in ER_get_flat_size (eohptr=0x2bbcfb0) at
expandedrecord.c:713
#2 0x00000000009a0954 in EOH_get_flat_size (eohptr=0x2bbcfb0) at
expandeddatum.c:77
#3 0x000000000048f61c in heap_compute_data_size (tupleDesc=0x2bd0168,
values=0x2bd02c8, isnull=0x2bd02d0) at heaptuple.c:155
#4 0x00000000004916b3 in heap_form_tuple (tupleDescriptor=0x2bd0168,
values=0x2bd02c8, isnull=0x2bd02d0) at heaptuple.c:1045
#5 0x00000000007296eb in tts_virtual_copy_heap_tuple (slot=0x2bd0280)
at execTuples.c:272
#6 0x0000000000728d0b in ExecCopySlotHeapTuple (slot=0x2bd0280) at
../../../src/include/executor/tuptable.h:456
#7 0x000000000072a5d8 in tts_buffer_heap_copyslot (dstslot=0x2bd0820,
srcslot=0x2bd0280) at execTuples.c:767
#8 0x0000000000758840 in ExecCopySlot (dstslot=0x2bd0820,
srcslot=0x2bd0280) at ../../../src/include/executor/tuptable.h:480
#9 0x000000000075c263 in ExecModifyTable (pstate=0x2bcfa08) at
nodeModifyTable.c:2264
The argument that we need to force decompression in such cases is
considerably more tenuous. It revolves around the possibility that the
compression AM itself has been dropped. As long as we have only
built-in compression methods, which are undroppable, it seems like we
could potentially just decide to do nothing at all about this issue.
If the only reason for expanding TOAST pointers inside the
expanded-record stuff is to avoid the possibility of one being
invalidated by a transaction commit, and if compression methods can't
be invalidated by a transaction commit, well then we don't really have
a problem.
Even after the above case, we might say it is still not a problem for
this patch because even though t2 doesn't have a direct relationship
with lz4 but it has an indirect relationship with lz4 via t1. So I
think this particular case which I showed might not be a problem even
for the custom compression method. However, I agree that the
decompressing to survive COMMIT/ROLLBACK might be a problem for custom
compression methods but not for the built-in method. So I agree with
the conclusion that even if we don't make any changes to the
"expandedrecord.c", it won't be a problem for the built-in methods.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Feb 20, 2021 at 11:04 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Even after the above case, we might say it is still not a problem for
this patch because even though t2 doesn't have a direct relationship
with lz4 but it has an indirect relationship with lz4 via t1. So I
think this particular case which I showed might not be a problem even
for the custom compression method. However, I agree that the
decompressing to survive COMMIT/ROLLBACK might be a problem for custom
compression methods but not for the built-in method. So I agree with
the conclusion that even if we don't make any changes to the
"expandedrecord.c", it won't be a problem for the built-in methods.
I think I was wrong here, consider below scenario (Robert had sent
this test to me(offlist) for showing some other example, which I have
modified a bit to prove another point)
create table foo (a int, b text);
create table foo1 (a int, b text compression lz4);
insert into foo1 select 1, repeat('a', 3000);
create or replace function make_foo() returns foo as $$declare x foo;
begin
x.a = 1;
select b into x.b from foo1;
return x;
end$$ language plpgsql;
create table bar (f foo);
insert into bar select make_foo();
SELECT pg_column_compression((bar.f).b) FROM bar;
pg_column_compression
-----------------------
lz4
(1 row)
So basically, now table bar doesn't have any relation with foo1 and it
is still the row with compression method lz4. This issue is resolved
with my changes in
(v25_0001_Disallow_compressed_data_inside_container_types, in
expandedrecord.c). So the point is that for the built-in method also
we need changes related to expandedrecord at least the changes I made
where tuple are actually formed for inserting into the target.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Feb 19, 2021 at 9:42 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Justin reported an issue offlist, basically, compression_1.out was not
updated so cfbot was failing on windows while running without-lz4, so
I have fixed that. Along with that I have also fixed 2 minor pending
issues 1) removing of compressamapi.h from unwanted places, reported
by Justin and 2) Assert in TOAST_COMPRESS_SET_SIZE_AND_METHOD macro,
suggested by Robert.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v26-0001-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v26-0001-Disallow-compressed-data-inside-container-types.patchDownload
From e6aaafe34f9fe4cbecbba2403a3d6248a12ca88d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 19 Feb 2021 17:57:05 +0530
Subject: [PATCH v26 1/3] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 15 ++++++-
src/backend/utils/adt/expandedrecord.c | 76 ++++++++++++----------------------
3 files changed, 43 insertions(+), 52 deletions(-)
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..5684a6d 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !(VARATT_IS_EXTERNAL((struct varlena *) DatumGetPointer(op->d.row.elemvalues[i])) ||
+ VARATT_IS_COMPRESSED((struct varlena *) DatumGetPointer(op->d.row.elemvalues[i]))))
+ continue;
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = PointerGetDatum(tuple->t_data);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..66031a5 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
--
1.8.3.1
v26-0003-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v26-0003-default-to-with-lz4.patchDownload
From c2cd1ff5a17f33bb2b714aa3eb7e2dd3db315ba8 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v26 3/3] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b7..62180c5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v26-0002-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v26-0002-Built-in-compression-method.patchDownload
From 63ca68d973696d3ca4d88dd2ebb1dd3e5797e31e Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v26 2/3] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 95 ++++++++----
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 63 ++++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 142 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 ++
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/executor/nodeModifyTable.c | 129 ++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 7 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 ++++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 29 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 ++
src/include/access/compressamapi.h | 99 ++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 +++++++++++++++++++++++
src/test/regress/expected/psql.out | 68 +++++----
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
66 files changed, 1775 insertions(+), 126 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..66dcb1b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..946770d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,77 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
+
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ return cmroutine;
+}
- return result;
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +542,9 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..bd25d8c
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..11405e9
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,142 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..cfd1b38 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -789,6 +789,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1716,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4ab0b8..b583063 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b2457a6..811dbc3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..be8a8fe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,120 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+
+ }
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2360,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..7e3c26a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1082,6 +1082,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..46044cb 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..1789e18 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b835b0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,21 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2036,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2117,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a818ee5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..9755e8e 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..554b643 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,12 @@ show_context_hook(const char *newval)
}
static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..4e4d5e1
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..a11015d 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710..4b3d34a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 943931f..182b77a 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..8e40d93
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..d842bdb
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..14f3fa3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..07fd3f6 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..7970f6e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM false
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..46cac11 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Sat, Feb 20, 2021 at 11:04 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Feb 20, 2021 at 2:51 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Feb 19, 2021 at 11:12 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I think that these performance tests aren't really exercising the
expanded-record stuff, just the ExecEvalRow changes. We need to test
that test case, and I tend to suspect there's going to be a measurable
regression.I will do testing around this area.
I have done testing for the expanded-record. Basically I have noticed
there is no performance regression when there are no
compressed/external fields. But there is a huge regression when there
are compressed data.
Test setup:
----------------
create table foo (a int, b text, c text);
create table bar (f foo);
create or replace function make_foo() returns foo as $$declare x foo;
begin
x.a = 1;
select b,c into x.b, x.c from foo;
return x;
end$$ language plpgsql;
create or replace function test() returns void AS
$$
begin
for i in 1..100000 loop
insert into bar select make_foo();
end loop;
end;
$$ language 'plpgsql';
Testcase: select test(); (every time truncate bar before executing this query)
Case1: No compress/no external
insert into foo select 1, repeat('1234567890', 10), repeat('1234567890', 10);
execution time for "select test()"
Head: 2536.420 ms
patch: 2688.565 ms
Case2: Only compress
insert into foo select 1, repeat('1234567890', 500), repeat('1234567890', 10);
execution time for "select test()"
head: 2545.944 ms
Patch: 9375.524 ms
Case2: compress + external
Alter table foo alter column c set storage external;
insert into foo select 1, repeat('1234567890', 500), repeat('1234567890', 500);
execution time for "select test()"
Head: 10265.052 ms
Patch: 15469.902 ms
Summary:
In this particular path we are only processing the already deformed
tuple that is the reason we are not seeing regression with the
non-compressed data. But with compressed data we have to give extra
cost for the decompression and that is why there is regression. But
IMHO with this we are getting one benefit is that now we will not have
individual compressed data inside composite type so next time when we
will have to select from those composite type then we will have to do
less decompression with patch compared to the head so we might gain
there. I will try to come up with the complete test case where we
will do selection after inserting into the composite type and compare
the performance.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sun, Feb 21, 2021 at 5:33 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Based on offlist discussion with Robert, I have done further analysis
of the composite type data. So the Idea is that I have analyzed all
the callers of
HeapTupleGetDatum and HeapTupleHeaderGetDatum and divide them into two
category 1) Callers which are forming the tuple from values that can
not have compressed/external data.
2) Callers which can have external/compressed data
So for type 1) instead of calling HeapTupleGetDatum or
HeapTupleHeaderGetDatum we can call some function HeapTupleGetRawDatum
which is similar to PoniterGetDatum. And for type 2) we will detoast
any varlena even before forming the tuple so that we don't have to pay
the penalty for checking the compressed attributes after forming the
tuple. After this change now we have no caller for HeapTupleGetDatum
but I have kept it because it is an exposed routine.
Here is the analysis for the callers for HeapTupleGetDatum and
HeapTupleHeaderGetDatum
1. functions which can build tuple from compressed/external filed
(Detoasted before forming the tuple)
ExecEvalRow(), ExecEvalConvertRowtype(), ExecEvalConvertRowtype(),
populate_record()
exec_eval_datum()->make_tuple_from_row() before forming tuple
populate_record()
2. functions (no compressed/external filed possible analysis given
function wise):
dblink_get_pkey() : INT and Name tuple built from fixed length data types
hstore_populate_record(),hstore_each() : Getting values from hstore no
ondisk varlena possible
pg_old_snapshot_time_mapping(): Building tuple from in memory old snapshot data
pg_buffercache_pages(): Buffer cache info
pg_stat_statements_info(): No disk value, just stat_statement info
pgp_armor_headers(): Building in memory string in
pgp_extract_armor_headers and operating on those values
pgstattuple_approx_internal(): no varlena
ssl_extension_info(): ssl extension info, no any ondisk data
pg_last_committed_xact(): fixed length field
pg_xact_commit_timestamp_origin(): No varlena field
pg_get_multixact_members(): multixact info no actual tuple data
pg_prepared_xact(): prepared xact info
pg_walfile_name_offset(): walfile name/offset
pg_get_object_address(), pg_identify_object(),
pg_identify_object_as_address(): Only object information, form from in
memory strings.
pg_sequence_parameters(): No varlena field
pg_stat_get_wal_receiver(): walreceiver statistics
pg_stats_ext_mcvlist_items(): In memory array or fixed length fields.
tt_process_call(),prs_process_call: parser tokens
aclexplode(): fixed type and cstring from in memory string.
pg_timezone_abbrevs(): no varlena
pg_stat_file(): no varlena
pg_lock_status(): lock stats
pg_get_keywords(), pg_get_catalog_foreign_keys(): in memory strings.
pg_partition_tree(): no varlena
pg_stat_get_wal(): no varlena
tsvector_unnest(): in memory array
show_all_settings(): guc values from in memory struct
plperl_hash_to_datum(): value fetched from perl hash
pltcl_func_handler(): tuple from cstrings
test_predtest(): no varlena
pg_visibility*(): Only data from visibility map so no varlena
pg_stat_get_wal(), pg_stat_get_archiver(): Building tuple from in-memory data
pgstatindex_impl(), pgstatginindex_internal(), pgstathashindex():
fixed/in-memory data
replication slot func in slotfunc.c: replication slot info
controls file info functions in pg_conttroldata.c : Control file data
page inspect function in contrib/pageinspect(brinfunc.c, btreefunc.c,
ginfunc.c, gistfunc.c, hashfunc.c, heapfunc.c): Only meta page or
header info.
record_in and record_recv, are forming tuple from input cstring
Next I will be working on reviewing GUC for default compression method
by Justin and post the next patch series.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v27-0001-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v27-0001-Disallow-compressed-data-inside-container-types.patchDownload
From ee86b55a13d719bca50c69f684425a77fc97dc4a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 19 Feb 2021 17:57:05 +0530
Subject: [PATCH v27 1/3] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 +-
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +-
contrib/pageinspect/ginfuncs.c | 6 +-
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 +--
contrib/pageinspect/heapfuncs.c | 6 +-
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 2 +-
contrib/pg_visibility/pg_visibility.c | 10 ++--
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +-
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/access/transam/commit_ts.c | 4 +-
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +-
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 28 ++++++++-
src/backend/executor/execTuples.c | 4 --
src/backend/replication/slotfuncs.c | 8 +--
src/backend/replication/walreceiver.c | 2 +-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 +-
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/expandedrecord.c | 76 +++++++++----------------
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/jsonfuncs.c | 7 ++-
src/backend/utils/adt/lockfuncs.c | 4 +-
src/backend/utils/adt/misc.c | 4 +-
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/adt/tsvector_op.c | 4 +-
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 +--
src/include/fmgr.h | 2 +-
src/include/funcapi.h | 4 ++
src/pl/plperl/plperl.c | 4 +-
src/pl/plpgsql/src/pl_exec.c | 4 +-
src/pl/tcl/pltcl.c | 4 +-
src/test/modules/test_predtest/test_predtest.c | 2 +-
50 files changed, 141 insertions(+), 136 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..6c3b571 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(rettuple));
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..b3147ba 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index 8bb180b..f7b220d 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -243,7 +243,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -416,7 +416,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -737,7 +737,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..4039c9d 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 9abcee3..893a15b 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 9e9ee8a..da72901 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -329,7 +329,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..da390a0 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,7 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..4819625 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,7 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -156,7 +156,7 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -197,7 +197,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +243,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +303,7 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index b1ce0d7..c2ad847 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -358,7 +358,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -567,7 +567,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -723,7 +723,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..54d835d 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 70d2257..f7afc24 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -786,7 +786,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..982130a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..32782c7 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,23 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
+ continue;
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3096,23 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3170,7 +3192,7 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
/* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..f115464 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 057f410..bc05981 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -202,7 +202,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -685,7 +685,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -906,7 +906,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 9ec7123..52946bc 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1424,5 +1424,5 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..66031a5 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..40d248e 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/*
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index f194ff9..ad07fd4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2979,7 +2979,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = PointerGetDatum(tuple);
JsObjectFree(&jso);
}
@@ -3398,6 +3398,9 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1)
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3776,7 +3779,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(PointerGetDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..8437de0 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,7 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -2259,7 +2259,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 00018ab..7a770fb 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9797,7 +9797,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..318dee2 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..47f66e7 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -372,7 +372,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_TEXT_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
-#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER(x) return PointerGetDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..b5a825a 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -207,6 +207,8 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*
* Macro declarations:
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain compressed/external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -219,8 +221,10 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*/
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) PointerGetDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
+#define HeapTupleGetDatum(tuple) PointerGetDatum((tuple)->t_data)
extern TupleDesc RelationNameGetTupleDesc(const char *relname);
extern TupleDesc TypeGetTupleDesc(Oid typeoid, List *colaliases);
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..0519253 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,8 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..51a023c 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,5 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
--
1.8.3.1
v27-0003-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v27-0003-default-to-with-lz4.patchDownload
From 8ad78d9d095c903d4e1159a24113a578fb5d7915 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v27 3/3] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b7..62180c5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v27-0002-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v27-0002-Built-in-compression-method.patchDownload
From 0fb3278da3c62d5f7a5db278d38569fb91779ea4 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v27 2/3] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 95 ++++++++----
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 63 ++++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 142 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 ++
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/executor/nodeModifyTable.c | 129 ++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 7 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 ++++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 29 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 ++
src/include/access/compressamapi.h | 99 ++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 +++++++++++++++++++++++
src/test/regress/expected/psql.out | 68 +++++----
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
66 files changed, 1775 insertions(+), 126 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..66dcb1b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..946770d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,77 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
+
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ return cmroutine;
+}
- return result;
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +542,9 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..bd25d8c
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..11405e9
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,142 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..cfd1b38 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -789,6 +789,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1716,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4ab0b8..b583063 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b2457a6..811dbc3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..be8a8fe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,120 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+
+ }
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2360,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..7e3c26a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1082,6 +1082,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..46044cb 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..1789e18 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b835b0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,21 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2036,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2117,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a818ee5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..9755e8e 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..554b643 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,12 @@ show_context_hook(const char *newval)
}
static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..4e4d5e1
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..a11015d 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710..4b3d34a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 943931f..182b77a 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..8e40d93
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..d842bdb
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..14f3fa3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..07fd3f6 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..7970f6e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM false
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..46cac11 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Fri, Feb 26, 2021 at 8:10 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Feb 21, 2021 at 5:33 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Based on offlist discussion with Robert, I have done further analysis
of the composite type data. So the Idea is that I have analyzed all
the callers of
HeapTupleGetDatum and HeapTupleHeaderGetDatum and divide them into two
category 1) Callers which are forming the tuple from values that can
not have compressed/external data.
2) Callers which can have external/compressed data
I just realized that there is one more function
"heap_copy_tuple_as_datum" which is flattening the tuple based on the
HeapTupleHasExternal check, so I think I will have to analyze the
caller of this function as well and need to do a similar analysis,
although there are just a few callers for this. And, I think the fix
in ExecEvalConvertRowtype is wrong, we will have to do something for
the compressed type here as well. I am not sure what is the best way
to fix it because we are directly getting the input tuple so we can
not put an optimization of dettoasting before forming the tuple. We
might detoast in execute_attr_map_tuple, when the source and target
row types are different because we are anyway deforming and processing
each filed in that function but the problem is execute_attr_map_tuple
is used at multiple places but for that, we can make another version
of this function which actually detoast along with conversion and use
that in ExecEvalConvertRowtype. But if there is no tuple conversion
needed then we directly use heap_copy_tuple_as_datum and in that case,
there is no deforming at all so maybe, in this case, we can not do
anything but I think ExecEvalConvertRowtype should not be the very
common path.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sat, Feb 27, 2021 at 7:44 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I just realized that there is one more function
"heap_copy_tuple_as_datum" which is flattening the tuple based on the
HeapTupleHasExternal check, so I think I will have to analyze the
caller of this function as well and need to do a similar analysis,
although there are just a few callers for this. And, I think the fix
in ExecEvalConvertRowtype is wrong, we will have to do something for
the compressed type here as well. I am not sure what is the best way
to fix it because we are directly getting the input tuple so we can
not put an optimization of dettoasting before forming the tuple. We
might detoast in execute_attr_map_tuple, when the source and target
row types are different because we are anyway deforming and processing
each filed in that function but the problem is execute_attr_map_tuple
is used at multiple places but for that, we can make another version
of this function which actually detoast along with conversion and use
that in ExecEvalConvertRowtype. But if there is no tuple conversion
needed then we directly use heap_copy_tuple_as_datum and in that case,
there is no deforming at all so maybe, in this case, we can not do
anything but I think ExecEvalConvertRowtype should not be the very
common path.
I have done further analysis for this, basically,
ExecEvalConvertRowtype can never have the compressed/external data
because it is converting from one composite type to another composite
type and while forming the composite type only we ensure that there
can not be any compressed/external data. Refer below comments in
ExecEvalConvertRowtype
/*
* The tuple is physically compatible as-is, but we need to insert the
* destination rowtype OID in its composite-datum header field, so we
* have to copy it anyway. heap_copy_tuple_as_datum() is convenient
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
* datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
* is overkill here, but its check for external fields is cheap.
*/
*op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
For heap_copy_tuple_as_datum, I have removed the external tuple check
and instead I have passed a parameter whether we need to flatten or
not. So the callers who are sure that they can not have any
compressed/external field should only pass false so that it will
completely skip the flattening path for those callers. But after
doing that in some of the callers especially
ExecFetchSlotHeapTupleDatum and
SPI_returntuple we will have to process the complete tuple when the
function's return type is tuple. I am not sure how to optimize this
because this is directly getting the tuple from the function. I am
not too much worried about the other callers like
PLyMapping_ToComposite, PLySequence_ToComposite and
PLyGenericObject_ToComposite because in these function we are forming
tuple from value before calling heap_copy_tuple_as_datum so if we
think these are performance critical paths then we have a way to
detoast even before forming the tuple.
Another function which I think can be problematic is
"expanded_record_set_tuple", because if we don't handle the compressed
types in this function then we can not go with the assumption that the
composite will never have compressed data. I am not completely sure
how much of a problem that can be? Maybe if we don't do anything here
then we might need to do something in ExecEvalConvertRowtype because
therein we assume that composite type can not contain compressed data
as well.
I have also reviewed the patch for the default compression method GUC
and made some changes.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v28-0004-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v28-0004-default-to-with-lz4.patchDownload
From afe0d59978f6dc4abe8d9a0cdddd306b89fd915a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v28 4/4] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b7..62180c5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v28-0001-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v28-0001-Disallow-compressed-data-inside-container-types.patchDownload
From 2be22e94da993f82781f028e573319ed6529e5ff Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 19 Feb 2021 17:57:05 +0530
Subject: [PATCH v28 1/4] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 +-
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +-
contrib/pageinspect/ginfuncs.c | 6 +-
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 +--
contrib/pageinspect/heapfuncs.c | 6 +-
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 2 +-
contrib/pg_visibility/pg_visibility.c | 10 ++--
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +-
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 14 +++--
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/access/transam/commit_ts.c | 4 +-
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +-
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 40 ++++++++++---
src/backend/executor/execTuples.c | 6 +-
src/backend/executor/spi.c | 3 +-
src/backend/replication/slotfuncs.c | 8 +--
src/backend/replication/walreceiver.c | 2 +-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 +-
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/expandedrecord.c | 76 +++++++++----------------
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/jsonfuncs.c | 7 ++-
src/backend/utils/adt/lockfuncs.c | 4 +-
src/backend/utils/adt/misc.c | 4 +-
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/adt/tsvector_op.c | 4 +-
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 +--
src/include/access/htup_details.h | 3 +-
src/include/fmgr.h | 2 +-
src/include/funcapi.h | 3 +
src/pl/plperl/plperl.c | 4 +-
src/pl/plpgsql/src/pl_exec.c | 4 +-
src/pl/plpython/plpy_typeio.c | 6 +-
src/pl/tcl/pltcl.c | 4 +-
src/test/modules/test_predtest/test_predtest.c | 2 +-
54 files changed, 165 insertions(+), 151 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..6c3b571 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(rettuple));
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..b3147ba 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index 8bb180b..f7b220d 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -243,7 +243,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -416,7 +416,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -737,7 +737,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..4039c9d 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 9abcee3..893a15b 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 9e9ee8a..da72901 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -329,7 +329,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..da390a0 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,7 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..4819625 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,7 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -156,7 +156,7 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -197,7 +197,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +243,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +303,7 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index b1ce0d7..c2ad847 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -358,7 +358,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -567,7 +567,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -723,7 +723,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 24a27e3..74507c3 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -977,19 +977,23 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
/* ----------------
* heap_copy_tuple_as_datum
*
- * copy a tuple as a composite-type Datum
+ * copy a tuple as a composite-type Datum, if detoast is passed true then
+ * the caller expect us to flatten any external/compressed data, this
+ * should only be passed false if the caller has already faltten all
+ * external/compressed data.
* ----------------
*/
Datum
-heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
+heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc, bool detoast)
{
HeapTupleHeader td;
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * If detoast is passed true then there could be some external/compressed
+ * data so we have to flatten those field to meet the conventions for
+ * composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
+ if (detoast)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..54d835d 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 70d2257..f7afc24 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -786,7 +786,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..982130a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..01c8f06 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,23 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
+ continue;
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3096,23 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3169,8 +3191,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to flatten
+ * any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3208,9 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any.
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc, false);
}
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..70caaf7 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1702,7 +1702,7 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
tupdesc = slot->tts_tupleDescriptor;
/* Convert to Datum form */
- ret = heap_copy_tuple_as_datum(tup, tupdesc);
+ ret = heap_copy_tuple_as_datum(tup, tupdesc, true);
if (shouldFree)
pfree(tup);
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 00aa78e..8155678 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -991,7 +991,8 @@ SPI_returntuple(HeapTuple tuple, TupleDesc tupdesc)
oldcxt = MemoryContextSwitchTo(_SPI_current->savedcxt);
- dtup = DatumGetHeapTupleHeader(heap_copy_tuple_as_datum(tuple, tupdesc));
+ dtup = DatumGetHeapTupleHeader(heap_copy_tuple_as_datum(tuple, tupdesc,
+ true));
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 057f410..bc05981 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -202,7 +202,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -685,7 +685,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -906,7 +906,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 9ec7123..52946bc 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1424,5 +1424,5 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..66031a5 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..40d248e 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/*
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index f194ff9..ad07fd4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2979,7 +2979,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = PointerGetDatum(tuple);
JsObjectFree(&jso);
}
@@ -3398,6 +3398,9 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1)
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3776,7 +3779,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(PointerGetDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..8437de0 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,7 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -2259,7 +2259,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 00018ab..7a770fb 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9797,7 +9797,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..318dee2 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..af59a84 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -789,7 +789,8 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
int attnum, bool *isnull);
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
-extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc,
+ bool detoast);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..47f66e7 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -372,7 +372,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_TEXT_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
-#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER(x) return PointerGetDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..efff83e 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -207,6 +207,8 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*
* Macro declarations:
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain compressed/external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -219,6 +221,7 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*/
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) PointerGetDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..0519253 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,8 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 5e807b1..fe67fa9 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -1414,7 +1414,7 @@ PLyMapping_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *mapping)
}
tuple = heap_form_tuple(desc, values, nulls);
- result = heap_copy_tuple_as_datum(tuple, desc);
+ result = heap_copy_tuple_as_datum(tuple, desc, true);
heap_freetuple(tuple);
pfree(values);
@@ -1491,7 +1491,7 @@ PLySequence_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *sequence)
}
tuple = heap_form_tuple(desc, values, nulls);
- result = heap_copy_tuple_as_datum(tuple, desc);
+ result = heap_copy_tuple_as_datum(tuple, desc, true);
heap_freetuple(tuple);
pfree(values);
@@ -1568,7 +1568,7 @@ PLyGenericObject_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *object
}
tuple = heap_form_tuple(desc, values, nulls);
- result = heap_copy_tuple_as_datum(tuple, desc);
+ result = heap_copy_tuple_as_datum(tuple, desc, true);
heap_freetuple(tuple);
pfree(values);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..51a023c 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,5 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
--
1.8.3.1
v28-0003-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v28-0003-Add-default_toast_compression-GUC.patchDownload
From 6e9d3a0b45f8f1a4dc125268d8177e1cea63a546 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH v28 3/4] Add default_toast_compression GUC
Justin Pryzby with modification from Dilip Kumar
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 3 +-
src/backend/commands/amcmds.c | 138 +++++++++++++++++++++++++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/init/postinit.c | 4 +
src/backend/utils/misc/guc.c | 12 +++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/pg_dump/pg_dump.c | 7 +-
src/include/access/amapi.h | 2 +
src/include/access/compressamapi.h | 12 ++-
10 files changed, 173 insertions(+), 12 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab..7afaea0 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidOid;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451ea..ec3376c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 3ad4a61..54edeff 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -13,8 +13,11 @@
*/
#include "postgres.h"
+#include "access/amapi.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
@@ -27,13 +30,20 @@
#include "parser/parse_func.h"
#include "utils/builtins.h"
#include "utils/lsyscache.h"
+#include "utils/guc.h"
+#include "utils/inval.h"
#include "utils/rel.h"
#include "utils/syscache.h"
-
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* Invalid means need to lookup the text value in the catalog */
+static Oid default_toast_compression_oid = InvalidOid;
+
+static void AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue);
/*
* CreateAccessMethod
@@ -277,3 +287,129 @@ lookup_am_handler_func(List *handler_name, char amtype)
return handlerOid;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/*
+ * assign_default_toast_compression: GUC assign_hook for default_toast_compression
+ */
+void
+assign_default_toast_compression(const char *newval, void *extra)
+{
+ /*
+ * Invalidate setting, forcing it to be looked up as needed.
+ * This avoids trying to do database access during GUC initialization,
+ * or outside a transaction.
+ */
+ default_toast_compression = NULL;
+}
+
+
+/*
+ * GetDefaultToastCompression -- get the OID of the current toast compression
+ *
+ * This exists to hide and optimize the use of the default_toast_compression
+ * GUC variable.
+ */
+Oid
+GetDefaultToastCompression(void)
+{
+ /* use pglz as default in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ return PGLZ_COMPRESSION_AM_OID;
+
+ Assert(!IsBootstrapProcessingMode());
+
+ /*
+ * If cached value isn't valid, look up the current default value, caching
+ * the result.
+ */
+ if (!OidIsValid(default_toast_compression_oid))
+ default_toast_compression_oid =
+ get_am_type_oid(default_toast_compression, AMTYPE_COMPRESSION,
+ false);
+
+ return default_toast_compression_oid;
+}
+
+/*
+ * InitializeAccessMethods: initialize module during InitPostgres.
+ *
+ * This is called after we are up enough to be able to do catalog lookups.
+ */
+void
+InitializeAccessMethods(void)
+{
+ if (IsBootstrapProcessingMode())
+ return;
+
+ /*
+ * In normal mode, arrange for a callback on any syscache invalidation
+ * of pg_am rows.
+ */
+ CacheRegisterSyscacheCallback(AMOID,
+ AccessMethodCallback,
+ (Datum) 0);
+ /* Force cached default access method to be recomputed on next use */
+ default_toast_compression_oid = InvalidOid;
+}
+
+/*
+ * AccessMethodCallback
+ * Syscache inval callback function
+ */
+static void
+AccessMethodCallback(Datum arg, int cacheid, uint32 hashvalue)
+{
+ /* Force look up of compression oid on next use */
+ default_toast_compression_oid = false;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 811dbc3..7adeaed 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidOid;
@@ -17762,7 +17762,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return GetDefaultToastCompression();
amoid = get_compression_am_oid(compression, false);
diff --git a/src/backend/utils/init/postinit.c b/src/backend/utils/init/postinit.c
index e5965bc..6a02bbd 100644
--- a/src/backend/utils/init/postinit.c
+++ b/src/backend/utils/init/postinit.c
@@ -25,6 +25,7 @@
#include "access/session.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/amapi.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -1060,6 +1061,9 @@ InitPostgres(const char *in_dbname, Oid dboid, const char *username,
/* set default namespace search path */
InitializeSearchPath();
+ /* set callback for changes to pg_am */
+ InitializeAccessMethods();
+
/* initialize client encoding */
InitializeClientEncoding();
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 7a770fb..5cdf305 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3916,6 +3917,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, assign_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 46044cb..f100e36 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -15851,7 +15851,6 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
- bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15876,9 +15875,6 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
- has_non_default_compression = (tbinfo->attcmnames[j] &&
- (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
-
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15918,8 +15914,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
* Attribute compression
*/
if (!dopt->no_compression_methods &&
- tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
- (has_non_default_compression || dopt->binary_upgrade))
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
{
appendPQExpBuffer(q, " COMPRESSION %s",
tbinfo->attcmnames[j]);
diff --git a/src/include/access/amapi.h b/src/include/access/amapi.h
index d357ebb..1513caf 100644
--- a/src/include/access/amapi.h
+++ b/src/include/access/amapi.h
@@ -287,4 +287,6 @@ typedef struct IndexAmRoutine
extern IndexAmRoutine *GetIndexAmRoutine(Oid amhandler);
extern IndexAmRoutine *GetIndexAmRoutineByAmId(Oid amoid, bool noerror);
+void InitializeAccessMethods(void);
+
#endif /* AMAPI_H */
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 4e4d5e1..c7e681f 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +30,17 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* Use default compression method if it is not specified. */
+/* Default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+
+/* GUC */
+extern char *default_toast_compression;
+extern void assign_default_toast_compression(const char *newval, void *extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+extern Oid GetDefaultToastCompression(void);
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
--
1.8.3.1
v28-0002-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v28-0002-Built-in-compression-method.patchDownload
From bb3206d924a1c3629663accd886b6175dc764490 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v28 2/4] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 95 ++++++++----
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 63 ++++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 164 ++++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 142 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 ++
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/executor/nodeModifyTable.c | 129 ++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 7 +
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 36 ++++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 29 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 ++
src/include/access/compressamapi.h | 99 ++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 +++++++++++++++++++++++
src/test/regress/expected/psql.out | 68 +++++----
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
66 files changed, 1775 insertions(+), 126 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..66dcb1b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..946770d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,77 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
+
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ return cmroutine;
+}
- return result;
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +542,9 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..bd25d8c
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,164 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..11405e9
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,142 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..cfd1b38 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -789,6 +789,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1716,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4ab0b8..b583063 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b2457a6..811dbc3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..be8a8fe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,120 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+
+ }
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2360,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..7e3c26a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1082,6 +1082,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..46044cb 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15832,6 +15851,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
{
bool print_default;
bool print_notnull;
+ bool has_non_default_compression;
/*
* Default value --- suppress if to be printed separately.
@@ -15856,6 +15876,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
!dopt->binary_upgrade)
continue;
+ has_non_default_compression = (tbinfo->attcmnames[j] &&
+ (strcmp(tbinfo->attcmnames[j], "pglz") != 0));
+
/* Format properly if not first attr */
if (actual_atts == 0)
appendPQExpBufferStr(q, " (");
@@ -15891,6 +15914,17 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]) &&
+ (has_non_default_compression || dopt->binary_upgrade))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..1789e18 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b835b0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,21 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2036,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2117,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a818ee5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..9755e8e 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..554b643 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,12 @@ show_context_hook(const char *newval)
}
static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..4e4d5e1
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..a11015d 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710..4b3d34a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 943931f..182b77a 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..8e40d93
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..d842bdb
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM false
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..14f3fa3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..07fd3f6 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..7970f6e
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM false
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..46cac11 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
Subject: [PATCH v28 3/4] Add default_toast_compression GUC
This part isn't working. My first patch worked somewhat better: due to doing
strcmp() with the default GUC, it avoided using the cached AM OID. (But it
would've failed with more than 2 AMs, since the cache wasn't invalidated, since
I couldn't tell when it was needed).
Your patch does this:
|postgres=# SET default_toast_compression=lz4 ;
|postgres=# CREATE TABLE t(a text);
|postgres=# \d+ t
| a | text | | | | extended | pglz | |
assign_default_toast_compression() should set
default_toast_compression_oid=InvalidOid, rather than
default_toast_compression=NULL.
In my original patch, that was commented, since I was confused, not realizing
that the GUC machinery itself assigns to the string value. We should assign to
the cached Oid, instead.
Reading my own patch, I see that in AccessMethodCallback() should also say
InvalidOid.
| default_toast_compression_oid = false;
The false assignment was copied from namespace.c: baseSearchPathValid.
--
Justin
Re: [HACKERS] Custom compression methods
On my PC, this new change is causing a test failure:
SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
- substr
-----------------------------------------------------
- 01234567890123456789012345678901234567890123456789
-(1 row)
-
+ERROR: compressed lz4 data is corrupt@@ -119,15 +119,15 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
int32 rawsize;
struct varlena *result;
- /* allocate memory for holding the uncompressed data */
- result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);- /* decompress partial data using lz4 routine */
+ /* decompress the data */
rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
VARDATA(result),
VARSIZE(value) - VARHDRSZ_COMPRESS,
slicelength,
- VARRAWSIZE_4B_C(value));
+ slicelength);Also, in the tests, you have this at both the top and bottom of the file:
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
Whereas the patch I sent had at the end:
+\set HIDE_COMPRESSAM on
("on" is the default when run under pg_regress)
Re: [HACKERS] Custom compression methods
On Sun, Feb 28, 2021 at 9:48 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On my PC, this new change is causing a test failure:
SELECT SUBSTR(f1, 2000, 50) FROM cmdata1; - substr ----------------------------------------------------- - 01234567890123456789012345678901234567890123456789 -(1 row) - +ERROR: compressed lz4 data is corrupt
The older version of lz4 had this problem that while decompressing
partial if we don't give the buffer size up to full data length it was
failing[1]https://docs.unrealengine.com/en-US/API/Runtime/Core/Compression/LZ4_decompress_safe_partial/index.html but it is solved in version 1.9.
@@ -119,15 +119,15 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
int32 rawsize;
struct varlena *result;- /* allocate memory for holding the uncompressed data */ - result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ); + /* allocate memory for the uncompressed data */ + result = (struct varlena *) palloc(slicelength + VARHDRSZ);- /* decompress partial data using lz4 routine */ + /* decompress the data */ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS, VARDATA(result), VARSIZE(value) - VARHDRSZ_COMPRESS, slicelength, - VARRAWSIZE_4B_C(value)); + slicelength);
This is done for the latest version as now we don't need to allocate
the buffer of full size, it is enough the allocate just equal to the
slicelength.
Also, in the tests, you have this at both the top and bottom of the file:
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM false
src/test/regress/sql/compression.sql:\set HIDE_COMPRESSAM falseWhereas the patch I sent had at the end:
+\set HIDE_COMPRESSAM on
("on" is the default when run under pg_regress)
I will fix this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 01, 2021 at 10:32:23AM +0530, Dilip Kumar wrote:
On Sun, Feb 28, 2021 at 9:48 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On my PC, this new change is causing a test failure:
SELECT SUBSTR(f1, 2000, 50) FROM cmdata1; - substr ----------------------------------------------------- - 01234567890123456789012345678901234567890123456789 -(1 row) - +ERROR: compressed lz4 data is corruptThe older version of lz4 had this problem that while decompressing
partial if we don't give the buffer size up to full data length it was
failing[1] but it is solved in version 1.9.
Thanks. It seems like that explains it.
I think if that's a problem with recent versions, then you'll have to
conditionally disable slicing.
https://packages.debian.org/liblz4-dev
Slicing isn't generally usable if it sometimes makes people's data inaccessible
and gives errors about corruption.
I guess you could make it a compile time test on these constants (I don't know
the necessary version, though)
#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
#define LZ4_VERSION_MINOR 7 /* for new (non-breaking) interface capabilities */
#define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)
If the version is too low, either make it #error, or disable slicing.
The OS usual library version infrastructure will make sure the runtime version
is at least the MAJOR+MINOR of the compile time version.
--
Justin
Re: [HACKERS] Custom compression methods
On Sat, Feb 27, 2021 at 9:35 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Subject: [PATCH v28 3/4] Add default_toast_compression GUC
This part isn't working. My first patch worked somewhat better: due to doing
strcmp() with the default GUC, it avoided using the cached AM OID. (But it
would've failed with more than 2 AMs, since the cache wasn't invalidated, since
I couldn't tell when it was needed).Your patch does this:
|postgres=# SET default_toast_compression=lz4 ;
|postgres=# CREATE TABLE t(a text);
|postgres=# \d+ t
| a | text | | | | extended | pglz | |assign_default_toast_compression() should set
default_toast_compression_oid=InvalidOid, rather than
default_toast_compression=NULL.
I will fix this.
In my original patch, that was commented, since I was confused, not realizing
that the GUC machinery itself assigns to the string value. We should assign to
the cached Oid, instead.
Reading my own patch, I see that in AccessMethodCallback() should also say
InvalidOid.
| default_toast_compression_oid = false;
The false assignment was copied from namespace.c: baseSearchPathValid.
I will fix this.
So as of now, we can make this patch such that it is enough to work
with the built-in method and later we can add another enhancement
patch that can work with the custom compression methods.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 1, 2021 at 11:06 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks. It seems like that explains it.
I think if that's a problem with recent versions, then you'll have to
conditionally disable slicing.
https://packages.debian.org/liblz4-devSlicing isn't generally usable if it sometimes makes people's data inaccessible
and gives errors about corruption.I guess you could make it a compile time test on these constants (I don't know
the necessary version, though)#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
#define LZ4_VERSION_MINOR 7 /* for new (non-breaking) interface capabilities */
#define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)If the version is too low, either make it #error, or disable slicing.
The OS usual library version infrastructure will make sure the runtime version
is at least the MAJOR+MINOR of the compile time version.
I think we can check the version and if it too low i.e. below1.8.3 (
in this release the slicing issue was fixed) then we can call the full
decompression routine from the slicing function.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 1, 2021 at 5:36 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Mar 1, 2021 at 11:06 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks. It seems like that explains it.
I think if that's a problem with recent versions, then you'll have to
conditionally disable slicing.
https://packages.debian.org/liblz4-devSlicing isn't generally usable if it sometimes makes people's data inaccessible
and gives errors about corruption.I guess you could make it a compile time test on these constants (I don't know
the necessary version, though)#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
#define LZ4_VERSION_MINOR 7 /* for new (non-breaking) interface capabilities */
#define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)If the version is too low, either make it #error, or disable slicing.
The OS usual library version infrastructure will make sure the runtime version
is at least the MAJOR+MINOR of the compile time version.I think we can check the version and if it too low i.e. below1.8.3 (
in this release the slicing issue was fixed) then we can call the full
decompression routine from the slicing function.
I have done that in the attached patch. Along with that, I have also
fixed the other issues raised by Justin related to the compression
method GUC patch and also removed the stuff from the GUC patch which
is not required for the built-in methods.
Now, I think the only pending thing is related to the expandedrecord,
basically, currently, we have detoasted the compressed filed only in
expanded_record_set_field_internal function. I am still not
completely sure that for the built-in types do we need to do something
for expanded_record_set_tuple and expanded_record_set_field or not, I
mean in these functions do we only expand the external to survive the
COMMIT/ROLLBACK or do we also expand it send it to some target table
like we do in expanded_record_set_field_internal.
As Robert mentioned upthread fixing in expanded_record_set_field might
not be very problematic as the tuple is already deformed but we have a
problem in expanded_record_set_tuple as we might need to deform the
tuple even though there are no compressed/external data.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v29-0001-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v29-0001-Disallow-compressed-data-inside-container-types.patchDownload
From 2be22e94da993f82781f028e573319ed6529e5ff Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 19 Feb 2021 17:57:05 +0530
Subject: [PATCH v29 1/4] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the comprassed field but while contructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we onlle decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 +-
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +-
contrib/pageinspect/ginfuncs.c | 6 +-
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 +--
contrib/pageinspect/heapfuncs.c | 6 +-
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 2 +-
contrib/pg_visibility/pg_visibility.c | 10 ++--
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +-
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 14 +++--
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/access/transam/commit_ts.c | 4 +-
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +-
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 40 ++++++++++---
src/backend/executor/execTuples.c | 6 +-
src/backend/executor/spi.c | 3 +-
src/backend/replication/slotfuncs.c | 8 +--
src/backend/replication/walreceiver.c | 2 +-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 +-
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/expandedrecord.c | 76 +++++++++----------------
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/jsonfuncs.c | 7 ++-
src/backend/utils/adt/lockfuncs.c | 4 +-
src/backend/utils/adt/misc.c | 4 +-
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 4 +-
src/backend/utils/adt/tsvector_op.c | 4 +-
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 +--
src/include/access/htup_details.h | 3 +-
src/include/fmgr.h | 2 +-
src/include/funcapi.h | 3 +
src/pl/plperl/plperl.c | 4 +-
src/pl/plpgsql/src/pl_exec.c | 4 +-
src/pl/plpython/plpy_typeio.c | 6 +-
src/pl/tcl/pltcl.c | 4 +-
src/test/modules/test_predtest/test_predtest.c | 2 +-
54 files changed, 165 insertions(+), 151 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..6c3b571 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(rettuple));
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..b3147ba 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index 8bb180b..f7b220d 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -243,7 +243,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -416,7 +416,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -737,7 +737,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..4039c9d 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index 9abcee3..893a15b 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 9e9ee8a..da72901 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -329,7 +329,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..da390a0 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,7 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..4819625 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,7 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -156,7 +156,7 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -197,7 +197,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +243,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +303,7 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index b1ce0d7..c2ad847 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -358,7 +358,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -567,7 +567,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -723,7 +723,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 24a27e3..74507c3 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -977,19 +977,23 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
/* ----------------
* heap_copy_tuple_as_datum
*
- * copy a tuple as a composite-type Datum
+ * copy a tuple as a composite-type Datum, if detoast is passed true then
+ * the caller expect us to flatten any external/compressed data, this
+ * should only be passed false if the caller has already faltten all
+ * external/compressed data.
* ----------------
*/
Datum
-heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
+heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc, bool detoast)
{
HeapTupleHeader td;
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * If detoast is passed true then there could be some external/compressed
+ * data so we have to flatten those field to meet the conventions for
+ * composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
+ if (detoast)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..54d835d 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 70d2257..f7afc24 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -786,7 +786,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..982130a 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..01c8f06 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,23 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
+ continue;
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3096,23 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3169,8 +3191,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to flatten
+ * any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3208,9 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any.
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc, false);
}
}
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..70caaf7 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -1702,7 +1702,7 @@ ExecFetchSlotHeapTupleDatum(TupleTableSlot *slot)
tupdesc = slot->tts_tupleDescriptor;
/* Convert to Datum form */
- ret = heap_copy_tuple_as_datum(tup, tupdesc);
+ ret = heap_copy_tuple_as_datum(tup, tupdesc, true);
if (shouldFree)
pfree(tup);
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/executor/spi.c b/src/backend/executor/spi.c
index 00aa78e..8155678 100644
--- a/src/backend/executor/spi.c
+++ b/src/backend/executor/spi.c
@@ -991,7 +991,8 @@ SPI_returntuple(HeapTuple tuple, TupleDesc tupdesc)
oldcxt = MemoryContextSwitchTo(_SPI_current->savedcxt);
- dtup = DatumGetHeapTupleHeader(heap_copy_tuple_as_datum(tuple, tupdesc));
+ dtup = DatumGetHeapTupleHeader(heap_copy_tuple_as_datum(tuple, tupdesc,
+ true));
MemoryContextSwitchTo(oldcxt);
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 057f410..bc05981 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -202,7 +202,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -685,7 +685,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -906,7 +906,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 9ec7123..52946bc 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1424,5 +1424,5 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..66031a5 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..40d248e 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(tuple));
}
/*
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index f194ff9..ad07fd4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2979,7 +2979,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = PointerGetDatum(tuple);
JsObjectFree(&jso);
}
@@ -3398,6 +3398,9 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1)
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3776,7 +3779,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(PointerGetDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..8437de0 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,7 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/*
@@ -2259,7 +2259,7 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 00018ab..7a770fb 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9797,7 +9797,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..318dee2 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(htup));
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..af59a84 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -789,7 +789,8 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
int attnum, bool *isnull);
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
-extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc,
+ bool detoast);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..47f66e7 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -372,7 +372,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_TEXT_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
-#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER(x) return PointerGetDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..efff83e 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -207,6 +207,8 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*
* Macro declarations:
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain compressed/external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -219,6 +221,7 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*/
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) PointerGetDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..0519253 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,8 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
diff --git a/src/pl/plpython/plpy_typeio.c b/src/pl/plpython/plpy_typeio.c
index 5e807b1..fe67fa9 100644
--- a/src/pl/plpython/plpy_typeio.c
+++ b/src/pl/plpython/plpy_typeio.c
@@ -1414,7 +1414,7 @@ PLyMapping_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *mapping)
}
tuple = heap_form_tuple(desc, values, nulls);
- result = heap_copy_tuple_as_datum(tuple, desc);
+ result = heap_copy_tuple_as_datum(tuple, desc, true);
heap_freetuple(tuple);
pfree(values);
@@ -1491,7 +1491,7 @@ PLySequence_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *sequence)
}
tuple = heap_form_tuple(desc, values, nulls);
- result = heap_copy_tuple_as_datum(tuple, desc);
+ result = heap_copy_tuple_as_datum(tuple, desc, true);
heap_freetuple(tuple);
pfree(values);
@@ -1568,7 +1568,7 @@ PLyGenericObject_ToComposite(PLyObToDatum *arg, TupleDesc desc, PyObject *object
}
tuple = heap_form_tuple(desc, values, nulls);
- result = heap_copy_tuple_as_datum(tuple, desc);
+ result = heap_copy_tuple_as_datum(tuple, desc, true);
heap_freetuple(tuple);
pfree(values);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..51a023c 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,5 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_DATUM(HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, nulls)));
}
--
1.8.3.1
v29-0004-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v29-0004-default-to-with-lz4.patchDownload
From 76e178a3d7c25206cb5e5ed5bffe950f7a6af889 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v29 4/4] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 63940b7..62180c5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v29-0003-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v29-0003-Add-default_toast_compression-GUC.patchDownload
From d3ca1d90a1626c2fd0e004d0af0de33e62804d21 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH v29 3/4] Add default_toast_compression GUC
Justin Pryzby with modification from Dilip Kumar
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 3 +-
src/backend/commands/amcmds.c | 99 ++++++++++++++++++++++++++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/misc/guc.c | 12 ++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/compressamapi.h | 12 +++-
7 files changed, 127 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab..7afaea0 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidOid;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451ea..ec3376c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 3ad4a61..cc9ea8e 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -13,8 +13,10 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
@@ -30,10 +32,13 @@
#include "utils/rel.h"
#include "utils/syscache.h"
-
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* Invalid means need to lookup the text value in the catalog */
+static Oid default_toast_compression_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -277,3 +282,95 @@ lookup_am_handler_func(List *handler_name, char amtype)
return handlerOid;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/*
+ * assign_default_toast_compression: GUC assign_hook for default_toast_compression
+ */
+void
+assign_default_toast_compression(const char *newval, void *extra)
+{
+ /*
+ * Invalidate setting, forcing it to be looked up as needed.
+ * This avoids trying to do database access during GUC initialization,
+ * or outside a transaction.
+ */
+ default_toast_compression_oid = InvalidOid;
+}
+
+
+/*
+ * GetDefaultToastCompression -- get the OID of the current toast compression
+ *
+ * This exists to hide and optimize the use of the default_toast_compression
+ * GUC variable.
+ */
+Oid
+GetDefaultToastCompression(void)
+{
+ /* use pglz as default in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ return PGLZ_COMPRESSION_AM_OID;
+
+ Assert(!IsBootstrapProcessingMode());
+
+ /*
+ * If cached value isn't valid, look up the current default value, caching
+ * the result.
+ */
+ if (!OidIsValid(default_toast_compression_oid))
+ default_toast_compression_oid =
+ get_compression_am_oid(default_toast_compression, false);
+
+ return default_toast_compression_oid;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 811dbc3..7adeaed 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11925,7 +11925,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidOid;
@@ -17762,7 +17762,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return GetDefaultToastCompression();
amoid = get_compression_am_oid(compression, false);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 7a770fb..5cdf305 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3916,6 +3917,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, assign_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 4e4d5e1..c7e681f 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +30,17 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* Use default compression method if it is not specified. */
+/* Default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+
+/* GUC */
+extern char *default_toast_compression;
+extern void assign_default_toast_compression(const char *newval, void *extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+extern Oid GetDefaultToastCompression(void);
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
--
1.8.3.1
v29-0002-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v29-0002-Built-in-compression-method.patchDownload
From bd8c229d0f4c078b1116bed07cce0e6734f5343c Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v29 2/4] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 95 ++++++++----
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 63 ++++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 166 ++++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 142 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 ++
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/executor/nodeModifyTable.c | 129 ++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 7 +
src/backend/utils/adt/expandedrecord.c | 6 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 31 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 29 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 ++
src/include/access/compressamapi.h | 99 ++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 +++++++++++++++++++++++
src/test/regress/expected/psql.out | 68 +++++----
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
67 files changed, 1775 insertions(+), 129 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 07da84d..63940b7 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..66dcb1b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..946770d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,77 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
+
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ return cmroutine;
+}
- return result;
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +542,9 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..e40c440
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..11405e9
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,142 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..cfd1b38 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -789,6 +789,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1716,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index b4ab0b8..b583063 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index eff9535..3ad4a61 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b2457a6..811dbc3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -559,7 +560,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -853,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2397,6 +2410,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2431,6 +2459,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2676,6 +2705,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6341,6 +6383,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11860,6 +11914,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17665,3 +17735,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..be8a8fe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,120 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+
+ }
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2360,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index 65bbc18..1338e04 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2966,6 +2966,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index f5dcedf..0605ef3 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2863,6 +2863,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index dd72a9f..52d92df 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15285,6 +15297,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15805,6 +15818,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..7e3c26a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1082,6 +1082,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 66031a5..3cbc256 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -697,9 +697,9 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
{
/*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
expanded_record_set_field_internal(erh, i + 1,
erh->dvalues[i], false,
true,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..f100e36 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15910,16 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..1789e18 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b835b0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,21 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2036,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2117,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a818ee5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..9755e8e 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..554b643 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,12 @@ show_context_hook(const char *newval)
}
static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..4e4d5e1
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..a11015d 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 1487710..4b3d34a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7095,6 +7104,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7265,6 +7278,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index 943931f..182b77a 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index 40ae489..20d6f96 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -512,6 +512,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 55cab4d..53d378b 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..ac427ed
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..dffd29e
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM true
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..14f3fa3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index 12bb67e..e9c0fc9 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..07fd3f6 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 59b416f..4d5577b 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -199,6 +199,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..08dd1b9
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 2aa062b..c64b369 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index bab4f3a..46cac11 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Mon, Mar 1, 2021 at 8:53 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Now, I think the only pending thing is related to the expandedrecord,
basically, currently, we have detoasted the compressed filed only in
expanded_record_set_field_internal function. I am still not
completely sure that for the built-in types do we need to do something
for expanded_record_set_tuple and expanded_record_set_field or not, I
mean in these functions do we only expand the external to survive the
COMMIT/ROLLBACK or do we also expand it send it to some target table
like we do in expanded_record_set_field_internal.
I have done further analysis of the compressed field in the
expandedrecord. My observation is that only in
expanded_record_set_field_internal we unconditionally pass true and
only when it is called from ER_get_flat_size. In all the other
functions (expanded_record_set_tuple and expanded_record_set_fields)
we only pass expand_external to true if estate->atomic is not set.
And, the estate->atomic is set to false only if we are executing the
anonymous block from a transaction block (there might be another way
to have estate->atomic as false). But the point is that the
flattening in these two functions are conditional which means we can
not use these expanded records to form some kind of row, otherwise, we
can not have the conditional flattening based on the way how the PL
block is being executed, so I think this proves Robert's point that we
are expanding this only for surviving the commit/rollback inside the
PL block. That means for the built-in types, decompression in
expanded_record_set_field_internal should be sufficient and that was
already done in my latest version of patch v29-0001.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
Does this patch need to do something about ExtractReplicaIdentity()?
If there are compressed fields in the tuple being built, can we rely
on the decompression engine being available at the time we need to do
something with the tuple?
More generally, I think it would be good to divide up 0001 into at
least 3 parts:
- The first part would invent HeapTupleGetRawDatum() and
HeapTupleHeaderGetRawDatum() and use them in place of the existing
functions everywhere that it's safe to do so. The current patch just
switches to using PointerGetDatum() but I think we should instead add
something like static inline Datum
HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) {
Assert(!HeapTupleHeaderHasExternal(tup)); return
PointerGetDatum(tuple); } This provides some type safety while being
just as fast as a direct use of PointerGetDatum() in optimized code. I
think that the Assert will have to be ripped out if we proceed with
the other patches, but if we can have it at this stage, so much the
better. I think this patch should also invent
PG_RETURN_HEAPTUPLEHEADER_RAW and likewise use that where appropriate
- including, I think, in place of cases that are now using
PG_RETURN_DATUM(HeapTupleGetDatum(...)). All of these changes make
sense from an efficiency standpoint apart from any possible
definitional changes.
- The second part would optimize code that the first part cannot
safely convert to use the "raw" versions. For example, the changes to
ExecEvalRow() can go in this part. You can view this part as getting
rid of calls to HeapTupleGetDatum(), HeapTupleHeaderGetDatum(), and/or
PG_RETURN_HEAPTUPLE() that couldn't be changed categorically, but can
be changed if we make some other code changes. These changes, too, can
potentially be justified on performance grounds independently of
anything else.
- Then we could maybe have some more patches that make other kinds of
preparatory changes. I'm not too sure exactly what should go in here,
or whether it should be 1 patch or several or maybe 0. But if there's
preparatory stuff that's potentially separately committable and not
the same as the stuff above, then it should go into patches here.
- The last patch would actually change the rule for composite datums.
One advantage of this is that if we have to revert the last patch for
some reason we are not ripping the entire thing, churning the code
base and widely-used APIs for everyone. Another advantage is that
getting those first two patches committed or even just applied locally
on a branch would, at least IMHO, make it a lot simpler to see what
potential problem spots remain - and by "problem" I mean mostly from a
performance point of view.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Mar 4, 2021 at 2:49 AM Robert Haas <robertmhaas@gmail.com> wrote:
Hi,
Does this patch need to do something about ExtractReplicaIdentity()?
If there are compressed fields in the tuple being built, can we rely
on the decompression engine being available at the time we need to do
something with the tuple?
We log the replica identity tuple in the WAL so that later walsender
can stream this to the subscriber, and before sending to the
subscriber anyway we have detoast all the data. Said that I think the
problem you are worried about is not only with 'replica identity
tuple' but it is with any tuple. I mean we copy the compressed field
as it is in WAL and suppose we copy some fields which are compressed
with lz4 and then we restart the server with another binary that is
compiled without lz4. Now, the problem is the walsender can not
decompress those data.
More generally, I think it would be good to divide up 0001 into at
least 3 parts:- The first part would invent HeapTupleGetRawDatum() and
HeapTupleHeaderGetRawDatum() and use them in place of the existing
functions everywhere that it's safe to do so. The current patch just
switches to using PointerGetDatum() but I think we should instead add
something like static inline Datum
HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) {
Assert(!HeapTupleHeaderHasExternal(tup)); return
PointerGetDatum(tuple); } This provides some type safety while being
just as fast as a direct use of PointerGetDatum() in optimized code. I
think that the Assert will have to be ripped out if we proceed with
the other patches, but if we can have it at this stage, so much the
better. I think this patch should also invent
PG_RETURN_HEAPTUPLEHEADER_RAW and likewise use that where appropriate
- including, I think, in place of cases that are now using
PG_RETURN_DATUM(HeapTupleGetDatum(...)). All of these changes make
sense from an efficiency standpoint apart from any possible
definitional changes.- The second part would optimize code that the first part cannot
safely convert to use the "raw" versions. For example, the changes to
ExecEvalRow() can go in this part. You can view this part as getting
rid of calls to HeapTupleGetDatum(), HeapTupleHeaderGetDatum(), and/or
PG_RETURN_HEAPTUPLE() that couldn't be changed categorically, but can
be changed if we make some other code changes. These changes, too, can
potentially be justified on performance grounds independently of
anything else.- Then we could maybe have some more patches that make other kinds of
preparatory changes. I'm not too sure exactly what should go in here,
or whether it should be 1 patch or several or maybe 0. But if there's
preparatory stuff that's potentially separately committable and not
the same as the stuff above, then it should go into patches here.- The last patch would actually change the rule for composite datums.
One advantage of this is that if we have to revert the last patch for
some reason we are not ripping the entire thing, churning the code
base and widely-used APIs for everyone. Another advantage is that
getting those first two patches committed or even just applied locally
on a branch would, at least IMHO, make it a lot simpler to see what
potential problem spots remain - and by "problem" I mean mostly from a
performance point of view.
Okay, I will work on this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Mar 4, 2021 at 4:03 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Mar 4, 2021 at 2:49 AM Robert Haas <robertmhaas@gmail.com> wrote:
More generally, I think it would be good to divide up 0001 into at
least 3 parts:Okay, I will work on this.
As per the suggestion I have divided 0001 in 3 patches.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v30-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchtext/x-patch; charset=US-ASCII; name=v30-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchDownload
From f125b3ccf67fbc0a83a6c6b374872e999960d3f2 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH v30 1/6] Get datum from tuple which doesn't contain external
data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 17 +++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 1 +
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 125 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..c508312 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..c36c283 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,21 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 80d2d20..67385e3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..20949a5 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to flatten
+ * any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index e5f8a06..f973aea 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1420,5 +1420,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..5936c39 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2259,7 +2260,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..4f1384a 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..59eb42f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9798,7 +9798,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..8771ccd 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..8ba7ae2 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v30-0003-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v30-0003-Disallow-compressed-data-inside-container-types.patchDownload
From d1c6d5bdbb89617fdf49006d12b4572b183778e1 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 16:33:10 +0530
Subject: [PATCH v30 3/6] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the compressed field but while constructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we only decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
Dilip Kumar based on idea from Robert Haas
---
src/backend/access/common/heaptuple.c | 9 ++--
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 6 +--
src/backend/executor/execTuples.c | 4 --
src/backend/utils/adt/expandedrecord.c | 76 ++++++++++++----------------------
src/backend/utils/adt/jsonfuncs.c | 3 +-
src/include/funcapi.h | 4 +-
src/pl/plpgsql/src/pl_exec.c | 3 +-
8 files changed, 38 insertions(+), 71 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c36c283..eb9f016 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -984,15 +984,12 @@ Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * We have to inline any external/compressed data to meet the conventions
+ * for composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
+ return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
- else
- return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
}
/* ----------------
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index c3754ac..71e6f41 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2845,8 +2845,7 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
- if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
continue;
op->d.row.elemvalues[i] =
@@ -3103,8 +3102,7 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
- if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
continue;
op->d.fieldstore.values[i] = PointerGetDatum(
PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..f115464 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..66031a5 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index c3d464f..821aa8f 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3388,8 +3388,7 @@ populate_record(TupleDesc tupdesc,
&field,
&nulls[i]);
- if (!nulls[i] && att->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ if (!nulls[i] && att->attlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 8ba7ae2..c869012 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -208,10 +208,10 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Macro declarations/inline functions:
* HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
* HeapTupleHeaderGetDatum but the input tuple should not contain
- * external varlena
+ * external/compressed varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
* HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
- * but the input tuple should not contain external varlena
+ * but the input tuple should not contain external/compressed varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index fd07376..0519253 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -7300,8 +7300,7 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
- if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v30-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v30-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From 568640c064dce3be36c1fde941230b8afffe9662 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v30 2/6] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/executor/execExprInterp.c | 29 +++++++++++++++++++++++++++--
src/backend/utils/adt/jsonfuncs.c | 8 ++++++--
src/pl/plpgsql/src/pl_exec.c | 5 ++++-
3 files changed, 37 insertions(+), 5 deletions(-)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 20949a5..c3754ac 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ continue;
+
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3098,24 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..c3d464f 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,10 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3769,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..fd07376 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,9 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v30-0005-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v30-0005-Add-default_toast_compression-GUC.patchDownload
From 5bcff0eb0ead1684188a0968ca870354e70dc8d5 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 29 Jan 2021 21:37:46 -0600
Subject: [PATCH v30 5/6] Add default_toast_compression GUC
Justin Pryzby with modification from Dilip Kumar
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 3 +-
src/backend/commands/amcmds.c | 99 ++++++++++++++++++++++++++-
src/backend/commands/tablecmds.c | 4 +-
src/backend/utils/misc/guc.c | 12 ++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/compressamapi.h | 12 +++-
7 files changed, 127 insertions(+), 6 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index ca26fab..7afaea0 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionOid;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidOid;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 9b451ea..ec3376c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -732,8 +732,9 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidOid;
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index afda884..35a09fa 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -13,8 +13,10 @@
*/
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/table.h"
+#include "access/xact.h"
#include "catalog/catalog.h"
#include "catalog/dependency.h"
#include "catalog/indexing.h"
@@ -30,10 +32,13 @@
#include "utils/rel.h"
#include "utils/syscache.h"
-
static Oid lookup_am_handler_func(List *handler_name, char amtype);
static const char *get_am_type_string(char amtype);
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* Invalid means need to lookup the text value in the catalog */
+static Oid default_toast_compression_oid = InvalidOid;
/*
* CreateAccessMethod
@@ -277,3 +282,95 @@ lookup_am_handler_func(List *handler_name, char amtype)
return handlerOid;
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!OidIsValid(get_compression_am_oid(*newval, true)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression access method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression access method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
+/*
+ * assign_default_toast_compression: GUC assign_hook for default_toast_compression
+ */
+void
+assign_default_toast_compression(const char *newval, void *extra)
+{
+ /*
+ * Invalidate setting, forcing it to be looked up as needed.
+ * This avoids trying to do database access during GUC initialization,
+ * or outside a transaction.
+ */
+ default_toast_compression_oid = InvalidOid;
+}
+
+
+/*
+ * GetDefaultToastCompression -- get the OID of the current toast compression
+ *
+ * This exists to hide and optimize the use of the default_toast_compression
+ * GUC variable.
+ */
+Oid
+GetDefaultToastCompression(void)
+{
+ /* use pglz as default in bootstrap mode */
+ if (IsBootstrapProcessingMode())
+ return PGLZ_COMPRESSION_AM_OID;
+
+ Assert(!IsBootstrapProcessingMode());
+
+ /*
+ * If cached value isn't valid, look up the current default value, caching
+ * the result.
+ */
+ if (!OidIsValid(default_toast_compression_oid))
+ default_toast_compression_oid =
+ get_compression_am_oid(default_toast_compression, false);
+
+ return default_toast_compression_oid;
+}
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index f0f6a88..69e3184 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11924,7 +11924,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidOid;
else if (!OidIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionOid;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidOid;
@@ -17738,7 +17738,7 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionOid;
+ return GetDefaultToastCompression();
amoid = get_compression_am_oid(compression, false);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 59eb42f..87353b4 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressamapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3917,6 +3918,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, assign_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
index 4e4d5e1..c7e681f 100644
--- a/src/include/access/compressamapi.h
+++ b/src/include/access/compressamapi.h
@@ -17,6 +17,7 @@
#include "catalog/pg_am_d.h"
#include "nodes/nodes.h"
+#include "utils/guc.h"
/*
* Built-in compression method-id. The toast compression header will store
@@ -29,8 +30,17 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* Use default compression method if it is not specified. */
+/* Default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+
+/* GUC */
+extern char *default_toast_compression;
+extern void assign_default_toast_compression(const char *newval, void *extra);
+extern bool check_default_toast_compression(char **newval, void **extra, GucSource source);
+
+extern Oid GetDefaultToastCompression(void);
+
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
/* compression handler routines */
--
1.8.3.1
v30-0004-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v30-0004-Built-in-compression-method.patchDownload
From e9488c981658cc4e613e15cf733a53b2d6ca632e Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Fri, 5 Feb 2021 18:21:47 +0530
Subject: [PATCH v30 4/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 95 ++++++++----
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 63 ++++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 166 ++++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 142 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 7 +-
src/backend/catalog/heap.c | 3 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 5 +
src/backend/commands/amcmds.c | 10 ++
src/backend/commands/createas.c | 14 ++
src/backend/commands/matview.c | 14 ++
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/executor/nodeModifyTable.c | 129 ++++++++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 7 +
src/backend/utils/adt/expandedrecord.c | 6 +-
src/backend/utils/adt/pseudotypes.c | 1 +
src/backend/utils/adt/varlena.c | 42 ++++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 31 +++-
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 29 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 9 ++
src/include/access/compressamapi.h | 99 ++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_am.dat | 6 +
src/include/catalog/pg_am.h | 1 +
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 20 +++
src/include/catalog/pg_type.dat | 5 +
src/include/commands/defrem.h | 1 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/nodes.h | 1 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 +++++++++++++++++++++++
src/test/regress/expected/psql.out | 68 +++++----
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
67 files changed, 1775 insertions(+), 129 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressamapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..da7cee0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..66dcb1b 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_COMPRESSAM</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression access method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..946770d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,28 +458,77 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
- * toast_decompress_datum -
+ * toast_get_compression_oid -
*
- * Decompress a compressed version of a varlena datum
+ * Returns the Oid of the compression method stored in the compressed data. If
+ * the varlena is not compressed then returns InvalidOid.
*/
-static struct varlena *
-toast_decompress_datum(struct varlena *attr)
+Oid
+toast_get_compression_oid(struct varlena *attr)
{
- struct varlena *result;
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidOid;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidOid;
+
+ return CompressionIdToOid(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
+ * toast_get_compression_handler - get the compression handler routines
+ *
+ * helper function for toast_decompress_datum and toast_decompress_datum_slice
+ */
+static inline const CompressionAmRoutine *
+toast_get_compression_handler(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine;
+ CompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
+ cmid = TOAST_COMPRESS_METHOD(attr);
+
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_ID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ return cmroutine;
+}
- return result;
+/* ----------
+ * toast_decompress_datum -
+ *
+ * Decompress a compressed version of a varlena datum
+ */
+static struct varlena *
+toast_decompress_datum(struct varlena *attr)
+{
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
+
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +542,9 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionAmRoutine *cmroutine = toast_get_compression_handler(attr);
- Assert(VARATT_IS_COMPRESSED(attr));
-
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..b04c5a5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,51 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, Oid cmoid)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionAmRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(OidIsValid(cmoid));
+
+ /* Get the handler routines for the compression method */
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ cmroutine = &pglz_compress_methods;
+ break;
+ case LZ4_COMPRESSION_AM_OID:
+ cmroutine = &lz4_compress_methods;
+ break;
+ default:
+ elog(ERROR, "Invalid compression method oid %u", cmoid);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* Call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, CompressionOidToId(cmoid));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..ca26fab 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressamapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionOid;
+ else
+ att->attcompression = InvalidOid;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..e40c440
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,166 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressamapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the lz4 compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine lz4_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/* lz4 compression handler function */
+Datum
+lz4handler(PG_FUNCTION_ARGS)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ PG_RETURN_POINTER(&lz4_compress_methods);
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..11405e9
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,142 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressamapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/* ------------------------------------------------------------------------
+ * Definition of the pglz compression access method.
+ * ------------------------------------------------------------------------
+ */
+const CompressionAmRoutine pglz_compress_methods = {
+ .type = T_CompressionAmRoutine,
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/* pglz compression handler function */
+Datum
+pglzhandler(PG_FUNCTION_ARGS)
+{
+ PG_RETURN_POINTER(&pglz_compress_methods);
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..9b451ea 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionOid;
+ else
+ attrtypes[attnum]->attcompression = InvalidOid;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..f5bb37b 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -187,7 +187,9 @@ my $GenbkiNextOid = $FirstGenbkiObjectId;
my $C_COLLATION_OID =
Catalog::FindDefinedSymbolFromData($catalog_data{pg_collation},
'C_COLLATION_OID');
-
+my $PGLZ_COMPRESSION_AM_OID =
+ Catalog::FindDefinedSymbolFromData($catalog_data{pg_am},
+ 'PGLZ_COMPRESSION_AM_OID');
# Fill in pg_class.relnatts by looking at the referenced catalog's schema.
# This is ugly but there's no better place; Catalog::AddDefaultValues
@@ -906,6 +908,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? $PGLZ_COMPRESSION_AM_OID : 0;
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..cfd1b38 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -789,6 +789,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1716,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidOid;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..a549481 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -220,6 +220,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidOid;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidOid;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/amcmds.c b/src/backend/commands/amcmds.c
index 188109e..afda884 100644
--- a/src/backend/commands/amcmds.c
+++ b/src/backend/commands/amcmds.c
@@ -176,6 +176,16 @@ get_table_am_oid(const char *amname, bool missing_ok)
}
/*
+ * get_compression_am_oid - given an access method name, look up its OID
+ * and verify it corresponds to an compression AM.
+ */
+Oid
+get_compression_am_oid(const char *amname, bool missing_ok)
+{
+ return get_am_type_oid(amname, AMTYPE_COMPRESSION, missing_ok);
+}
+
+/*
* get_am_oid - given an access method name, look up its OID.
* The type is not checked.
*/
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..1d17dc0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,15 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +629,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..713fc3f 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,15 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +531,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..f0f6a88 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressamapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -558,7 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static Oid GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -852,6 +853,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidOid;
}
/*
@@ -2396,6 +2409,21 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (OidIsValid(attribute->attcompression))
+ {
+ char *compression = get_am_name(attribute->attcompression);
+
+ if (!def->compression)
+ def->compression = compression;
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2458,7 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ def->compression = get_am_name(attribute->attcompression);
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2704,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (!def->compression)
+ def->compression = newdef->compression;
+ else if (newdef->compression)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6382,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store its Oid in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidOid;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11913,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidOid;
+ else if (!OidIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionOid;
+ }
+ else
+ attTup->attcompression = InvalidOid;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17711,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to an OID.
+ */
+static Oid
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ Oid amoid;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidOid;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionOid;
+
+ amoid = get_compression_am_oid(compression, false);
+
+#ifndef HAVE_LIBLZ4
+ if (amoid == LZ4_COMPRESSION_AM_OID)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return amoid;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..be8a8fe 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,6 +37,8 @@
#include "postgres.h"
+#include "access/compressamapi.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
@@ -2036,6 +2038,120 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not supported in the target attribute then
+ * decompress the value. If any of the value need to decompress then we
+ * need to store that into the new slot.
+ *
+ * The slot will hold the input slot but if any of the value need to be
+ * decompressed then the new slot will be stored into this and the old
+ * slot will be dropped.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ Oid cmoid;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method Oid stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmoid = toast_get_compression_oid(new_value);
+ if (OidIsValid(cmoid) &&
+ targetTupDesc->attrs[i].attcompression != cmoid)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc, slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+ /*
+ * Copy the value/null array to the new slot and materialize it,
+ * before clearing the tuple from the old slot.
+ */
+ memcpy(newslot->tts_values, slot->tts_values, natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull, natts * sizeof(bool));
+
+ }
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2360,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..7e3c26a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -1082,6 +1082,13 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && OidIsValid(attribute->attcompression))
+ def->compression = get_am_name(attribute->attcompression);
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index 66031a5..3cbc256 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -697,9 +697,9 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
{
/*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
expanded_record_set_field_internal(erh, i + 1,
erh->dvalues[i], false,
true,
diff --git a/src/backend/utils/adt/pseudotypes.c b/src/backend/utils/adt/pseudotypes.c
index c2f910d..fe133c7 100644
--- a/src/backend/utils/adt/pseudotypes.c
+++ b/src/backend/utils/adt/pseudotypes.c
@@ -383,6 +383,7 @@ PSEUDOTYPE_DUMMY_IO_FUNCS(language_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(fdw_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(table_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(index_am_handler);
+PSEUDOTYPE_DUMMY_IO_FUNCS(compression_am_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(tsm_handler);
PSEUDOTYPE_DUMMY_IO_FUNCS(internal);
PSEUDOTYPE_DUMMY_IO_FUNCS(anyelement);
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a35abe5 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,9 +17,11 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressamapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
+#include "commands/defrem.h"
#include "common/hashfn.h"
#include "common/hex.h"
#include "common/int.h"
@@ -5300,6 +5302,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ Oid cmoid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ cmoid =
+ toast_get_compression_oid((struct varlena *) DatumGetPointer(value));
+
+ if (!OidIsValid(cmoid))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(get_am_name(cmoid)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..f100e36 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "am.amname AS attcmname,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8720,7 +8732,12 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
/* need left join here to not fail on dropped columns ... */
appendPQExpBuffer(q,
"FROM pg_catalog.pg_attribute a LEFT JOIN pg_catalog.pg_type t "
- "ON a.atttypid = t.oid\n"
+ "ON a.atttypid = t.oid\n");
+
+ if (createWithCompression)
+ appendPQExpBuffer(q, "LEFT JOIN pg_catalog.pg_am am "
+ "ON a.attcompression = am.oid\n");
+ appendPQExpBuffer(q,
"WHERE a.attrelid = '%u'::pg_catalog.oid "
"AND a.attnum > 0::pg_catalog.int2\n"
"ORDER BY a.attnum",
@@ -8747,6 +8764,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcmnames = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8793,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcmnames[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attcmname")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15910,16 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcmnames[j] && strlen(tbinfo->attcmnames[j]))
+ {
+ appendPQExpBuffer(q, " COMPRESSION %s",
+ tbinfo->attcmnames[j]);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..1789e18 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char **attcmnames; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..97791f8 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text\E\D*,\n
+ \s+\Qcol3 text\E\D*,\n
+ \s+\Qcol4 text\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5)\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b835b0c 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -170,10 +170,12 @@ describeAccessMethods(const char *pattern, bool verbose)
" CASE amtype"
" WHEN 'i' THEN '%s'"
" WHEN 't' THEN '%s'"
+ " WHEN 'c' THEN '%s'"
" END AS \"%s\"",
gettext_noop("Name"),
gettext_noop("Index"),
gettext_noop("Table"),
+ gettext_noop("Compression"),
gettext_noop("Type"));
if (verbose)
@@ -1459,7 +1461,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1477,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1895,21 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compressam &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n CASE WHEN attcompression = 0 THEN NULL ELSE "
+ " (SELECT am.amname "
+ " FROM pg_catalog.pg_am am "
+ " WHERE am.oid = a.attcompression) "
+ " END AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2036,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2117,11 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression */
+ if (attcompression_col >= 0)
+ printTableAddCell(&cont, PQgetvalue(res, i, attcompression_col),
+ false, false);
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a818ee5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_COMPRESSAM\n"
+ " if set, compression access methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..9755e8e 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compressam;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..554b643 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,12 @@ show_context_hook(const char *newval)
}
static bool
+hide_compressam_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_COMPRESSAM", &pset.hide_compressam);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1233,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_COMPRESSAM",
+ bool_substitute_hook,
+ hide_compressam_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressamapi.h b/src/include/access/compressamapi.h
new file mode 100644
index 0000000..4e4d5e1
--- /dev/null
+++ b/src/include/access/compressamapi.h
@@ -0,0 +1,99 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressamapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressamapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAMAPI_H
+#define COMPRESSAMAPI_H
+
+#include "postgres.h"
+
+#include "catalog/pg_am_d.h"
+#include "nodes/nodes.h"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression method oid.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* Use default compression method if it is not specified. */
+#define DefaultCompressionOid PGLZ_COMPRESSION_AM_OID
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression AM.
+ *
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionAmRoutine
+{
+ NodeTag type;
+
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionAmRoutine;
+
+extern const CompressionAmRoutine pglz_compress_methods;
+extern const CompressionAmRoutine lz4_compress_methods;
+
+/*
+ * CompressionOidToId - Convert compression Oid to built-in compression id.
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline CompressionId
+CompressionOidToId(Oid cmoid)
+{
+ switch (cmoid)
+ {
+ case PGLZ_COMPRESSION_AM_OID:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION_AM_OID:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method oid %u", cmoid);
+ }
+}
+
+/*
+ * CompressionIdToOid - Convert built-in compression id to Oid
+ *
+ * For more details refer comment atop CompressionId in compressamapi.h
+ */
+static inline Oid
+CompressionIdToOid(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION_AM_OID;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION_AM_OID;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+#endif /* COMPRESSAMAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..6cdc375 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method oid from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_oid(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..dca0bc3 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ Oid tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..a11015d 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressamapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, Oid cmoid);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_am.dat b/src/include/catalog/pg_am.dat
index 6082f0e..6056844 100644
--- a/src/include/catalog/pg_am.dat
+++ b/src/include/catalog/pg_am.dat
@@ -33,5 +33,11 @@
{ oid => '3580', oid_symbol => 'BRIN_AM_OID',
descr => 'block range index (BRIN) access method',
amname => 'brin', amhandler => 'brinhandler', amtype => 'i' },
+{ oid => '4572', oid_symbol => 'PGLZ_COMPRESSION_AM_OID',
+ descr => 'pglz compression access method',
+ amname => 'pglz', amhandler => 'pglzhandler', amtype => 'c' },
+{ oid => '4573', oid_symbol => 'LZ4_COMPRESSION_AM_OID',
+ descr => 'lz4 compression access method',
+ amname => 'lz4', amhandler => 'lz4handler', amtype => 'c' },
]
diff --git a/src/include/catalog/pg_am.h b/src/include/catalog/pg_am.h
index ced86fa..65079f3 100644
--- a/src/include/catalog/pg_am.h
+++ b/src/include/catalog/pg_am.h
@@ -59,6 +59,7 @@ DECLARE_UNIQUE_INDEX_PKEY(pg_am_oid_index, 2652, on pg_am using btree(oid oid_op
*/
#define AMTYPE_INDEX 'i' /* index access method */
#define AMTYPE_TABLE 't' /* table access method */
+#define AMTYPE_COMPRESSION 'c' /* compression access method */
#endif /* EXPOSE_TO_CLIENT_CODE */
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..797e78b 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * OID of compression AM. Must be InvalidOid if and only if typstorage is
+ * 'plain' or 'external'.
+ */
+ Oid attcompression BKI_DEFAULT(0);
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(Oid))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..5d42d3a 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -941,6 +941,15 @@
prorettype => 'void', proargtypes => 'regclass int8',
prosrc => 'brin_desummarize_range' },
+{ oid => '6015', descr => 'pglz compression access method handler',
+ proname => 'pglzhandler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'pglzhandler' },
+{ oid => '6016', descr => 'lz4 compression access method handler',
+ proname => 'lz4handler', provolatile => 'v',
+ prorettype => 'compression_am_handler', proargtypes => 'internal',
+ prosrc => 'lz4handler' },
+
{ oid => '338', descr => 'validate an operator class',
proname => 'amvalidate', provolatile => 'v', prorettype => 'bool',
proargtypes => 'oid', prosrc => 'amvalidate' },
@@ -7098,6 +7107,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
@@ -7268,6 +7281,13 @@
{ oid => '268', descr => 'I/O',
proname => 'table_am_handler_out', prorettype => 'cstring',
proargtypes => 'table_am_handler', prosrc => 'table_am_handler_out' },
+{ oid => '560', descr => 'I/O',
+ proname => 'compression_am_handler_in', proisstrict => 'f',
+ prorettype => 'compression_am_handler', proargtypes => 'cstring',
+ prosrc => 'compression_am_handler_in' },
+{ oid => '561', descr => 'I/O',
+ proname => 'compression_am_handler_out', prorettype => 'cstring',
+ proargtypes => 'compression_am_handler', prosrc => 'compression_am_handler_out' },
{ oid => '5086', descr => 'I/O',
proname => 'anycompatible_in', prorettype => 'anycompatible',
proargtypes => 'cstring', prosrc => 'anycompatible_in' },
diff --git a/src/include/catalog/pg_type.dat b/src/include/catalog/pg_type.dat
index 8959c2f..306aabf 100644
--- a/src/include/catalog/pg_type.dat
+++ b/src/include/catalog/pg_type.dat
@@ -626,6 +626,11 @@
typcategory => 'P', typinput => 'index_am_handler_in',
typoutput => 'index_am_handler_out', typreceive => '-', typsend => '-',
typalign => 'i' },
+ { oid => '5559',
+ typname => 'compression_am_handler', typlen => '4', typbyval => 't', typtype => 'p',
+ typcategory => 'P', typinput => 'compression_am_handler_in',
+ typoutput => 'compression_am_handler_out', typreceive => '-', typsend => '-',
+ typalign => 'i' },
{ oid => '3310',
descr => 'pseudo-type for the result of a tablesample method function',
typname => 'tsm_handler', typlen => '4', typbyval => 't', typtype => 'p',
diff --git a/src/include/commands/defrem.h b/src/include/commands/defrem.h
index 1a79540..e5aea8a 100644
--- a/src/include/commands/defrem.h
+++ b/src/include/commands/defrem.h
@@ -139,6 +139,7 @@ extern Datum transformGenericOptions(Oid catalogId,
extern ObjectAddress CreateAccessMethod(CreateAmStmt *stmt);
extern Oid get_index_am_oid(const char *amname, bool missing_ok);
extern Oid get_table_am_oid(const char *amname, bool missing_ok);
+extern Oid get_compression_am_oid(const char *amname, bool missing_ok);
extern Oid get_am_oid(const char *amname, bool missing_ok);
extern char *get_am_name(Oid amOid);
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e31ad62..e388336 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/nodes.h b/src/include/nodes/nodes.h
index e22df89..a748504 100644
--- a/src/include/nodes/nodes.h
+++ b/src/include/nodes/nodes.h
@@ -515,6 +515,7 @@ typedef enum NodeTag
T_IndexAmRoutine, /* in access/amapi.h */
T_TableAmRoutine, /* in access/tableam.h */
T_TsmRoutine, /* in access/tsmapi.h */
+ T_CompressionAmRoutine, /* in access/compressamapi.h */
T_ForeignKeyCacheInfo, /* in utils/rel.h */
T_CallContext, /* in nodes/parsenodes.h */
T_SupportRequestSimplify, /* in nodes/supportnodes.h */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..488413a 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..ac427ed
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_COMPRESSAM true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..dffd29e
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_COMPRESSAM true
diff --git a/src/test/regress/expected/psql.out b/src/test/regress/expected/psql.out
index 7204fdb..14f3fa3 100644
--- a/src/test/regress/expected/psql.out
+++ b/src/test/regress/expected/psql.out
@@ -4898,8 +4898,8 @@ Indexes:
-- check printing info about access methods
\dA
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4907,13 +4907,15 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA *
List of access methods
- Name | Type
---------+-------
+ Name | Type
+--------+-------------
brin | Index
btree | Index
gin | Index
@@ -4921,8 +4923,10 @@ List of access methods
hash | Index
heap | Table
heap2 | Table
+ lz4 | Compression
+ pglz | Compression
spgist | Index
-(8 rows)
+(10 rows)
\dA h*
List of access methods
@@ -4947,32 +4951,36 @@ List of access methods
\dA: extra argument "bar" ignored
\dA+
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ *
- List of access methods
- Name | Type | Handler | Description
---------+-------+----------------------+----------------------------------------
- brin | Index | brinhandler | block range index (BRIN) access method
- btree | Index | bthandler | b-tree index access method
- gin | Index | ginhandler | GIN index access method
- gist | Index | gisthandler | GiST index access method
- hash | Index | hashhandler | hash index access method
- heap | Table | heap_tableam_handler | heap table access method
- heap2 | Table | heap_tableam_handler |
- spgist | Index | spghandler | SP-GiST index access method
-(8 rows)
+ List of access methods
+ Name | Type | Handler | Description
+--------+-------------+----------------------+----------------------------------------
+ brin | Index | brinhandler | block range index (BRIN) access method
+ btree | Index | bthandler | b-tree index access method
+ gin | Index | ginhandler | GIN index access method
+ gist | Index | gisthandler | GiST index access method
+ hash | Index | hashhandler | hash index access method
+ heap | Table | heap_tableam_handler | heap table access method
+ heap2 | Table | heap_tableam_handler |
+ lz4 | Compression | lz4handler | lz4 compression access method
+ pglz | Compression | pglzhandler | pglz compression access method
+ spgist | Index | spghandler | SP-GiST index access method
+(10 rows)
\dA+ h*
List of access methods
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..07fd3f6 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_COMPRESSAM=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..08dd1b9
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_COMPRESSAM false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_COMPRESSAM true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..daf163c 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..f0066d5 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionAmRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v30-0006-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v30-0006-default-to-with-lz4.patchDownload
From 8d987490107d8ec09585b55a1d22c25d9267f747 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v30 6/6] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index da7cee0..78f5065 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Thu, Mar 4, 2021 at 4:03 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Mar 4, 2021 at 2:49 AM Robert Haas <robertmhaas@gmail.com> wrote:
Hi,
Does this patch need to do something about ExtractReplicaIdentity()?
If there are compressed fields in the tuple being built, can we rely
on the decompression engine being available at the time we need to do
something with the tuple?We log the replica identity tuple in the WAL so that later walsender
can stream this to the subscriber, and before sending to the
subscriber anyway we have detoast all the data. Said that I think the
problem you are worried about is not only with 'replica identity
tuple' but it is with any tuple. I mean we copy the compressed field
as it is in WAL and suppose we copy some fields which are compressed
with lz4 and then we restart the server with another binary that is
compiled without lz4. Now, the problem is the walsender can not
decompress those
Based on the off list discussion with Robert, there are a couple of
problems which might be very difficult to handle when we support the
custom compression method using the access methods, the major problems
are 1) compressed data inside composite type 2) Access method might
get dropped before walsender decode the compressed data. I think the
first problem we still have is some solution although it may impact
performance in some cases i.e. extended record. But the problem of
the compressed data inside the WAL is a bigger problem. So as of now
we are planning to go ahead only with the built-in methods and if we
are only continuing with the built-in method so it doesn't make sense
to continue the access method infrastructure. I have rewrote the
patches without using the access method for compression. Changes in
the patches
- Removed complete dependency on the access method for compression
- While moving the tuple from one table to another table with
different compression method, no need to compare the compression
method and decompress.
- Alter table set compression, will not rewrite the old data, so only
the new tuple will be compressed with the new compression method.
- No preserve.
I feel the built-in method patch now looks cleaner and smaller than it
was before.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v31-0001-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v31-0001-Built-in-compression-method.patchDownload
From 571a1ba13a54f9c4170f0a5583e7f875dc4b48c1 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v31 1/4] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 71 ++++++---
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 54 +++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 155 +++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 137 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 116 +++++++++++++-
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 9 ++
src/backend/utils/adt/varlena.c | 41 +++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 ++
src/include/access/compressapi.h | 151 +++++++++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 189 +++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
53 files changed, 1553 insertions(+), 87 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..da7cee0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..256d179 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in the columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..6d6f26f 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column's
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..a3d259c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,6 +458,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+Oid
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
* toast_decompress_datum -
*
* Decompress a compressed version of a varlena datum
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..69dd949 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..d9f018c 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..2d16bf0
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,155 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* Definition of the lz4 compression method */
+const CompressionRoutine lz4_compress_methods = {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..9bd5370
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,137 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* Definition of the pglz compression method */
+const CompressionRoutine pglz_compress_methods = {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..3263460 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..558bc17 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..9383afd 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/heapam.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..3cd1cf6 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -558,7 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -852,6 +853,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2459,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2709,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression))
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6387,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11918,22 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * InvalidOid for the plain/external storage otherwise default
+ * compression id.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17716,42 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for the plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+#ifndef HAVE_LIBLZ4
+ if (cmethod == LZ4_COMPRESSION)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("not built with lz4 support")));
+#endif
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..e254806 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..a4249e8 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,6 +17,7 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -5300,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or the uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..7332f93 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_compression_methods;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..f886276 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-compression-methods", no_argument, &dopt.no_compression_methods, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-compression-methods do not dump compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_compression_methods &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..d956b03 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a43a8f5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressapi.h b/src/include/access/compressapi.h
new file mode 100644
index 0000000..2c59257
--- /dev/null
+++ b/src/include/access/compressapi.h
@@ -0,0 +1,151 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAPI_H
+#define COMPRESSAPI_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern const CompressionRoutine pglz_compress_methods;
+extern const CompressionRoutine lz4_compress_methods;
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in compressapi.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in compressapi.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+static inline char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(pglz_compress_methods.cmname, compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(lz4_compress_methods.cmname, compression) == 0)
+ return LZ4_COMPRESSION;
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+static inline const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return &pglz_compress_methods;
+ case LZ4_COMPRESSION:
+ return &lz4_compress_methods;
+ default:
+ elog(ERROR, "invalid compression method %u", method);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+#endif /* COMPRESSAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..d958af0 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..15dc5e6 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..faa5456 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7098,6 +7098,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..488413a 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3b546da
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..bb58dfb
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,189 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: not built with lz4 support
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: not built with lz4 support
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..ee46ba0
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..daf163c 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..0fc7017 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v31-0002-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v31-0002-Add-default_toast_compression-GUC.patchDownload
From a2581db904b38be3a2f87af25a3c7ad310bd6ff0 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sat, 6 Mar 2021 16:05:12 +0530
Subject: [PATCH v31 2/4] Add default_toast_compression GUC
Justin Pryzby with modification from Dilip Kumar
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 10 +++--
src/backend/utils/misc/guc.c | 64 +++++++++++++++++++++++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/compressapi.h | 18 +++++++-
6 files changed, 89 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index d9f018c..94e50f9 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 3263460..4142e8c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3cd1cf6..1164837 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -121,6 +121,8 @@ typedef struct OnCommitItem
static List *on_commits = NIL;
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
/*
* State information for ALTER TABLE
@@ -11929,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17743,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
#ifndef HAVE_LIBLZ4
if (cmethod == LZ4_COMPRESSION)
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..5d0b7ec 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -227,6 +228,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static bool check_default_toast_compression(char **newval, void **extra, GucSource source);
/* Private functions in guc-file.l that need to be called from guc.c */
static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -3917,6 +3919,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
@@ -12229,4 +12242,55 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
#include "guc-file.c"
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/compressapi.h b/src/include/access/compressapi.h
index 2c59257..809738e 100644
--- a/src/include/access/compressapi.h
+++ b/src/include/access/compressapi.h
@@ -36,12 +36,15 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
+
+extern char *default_toast_compression;
+
/* compression handler routines */
typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
@@ -148,4 +151,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
+
#endif /* COMPRESSAPI_H */
--
1.8.3.1
v31-0004-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v31-0004-default-to-with-lz4.patchDownload
From 8026b815bf6383b989a0de289732466ba8cd9569 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v31 4/4] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index da7cee0..78f5065 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v31-0003-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v31-0003-Alter-table-set-compression.patchDownload
From cb0722fd4b7ff1bc2094123077fcb865b65b86ca Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sat, 6 Mar 2021 13:10:23 +0530
Subject: [PATCH v31 3/4] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 44 ++++++-
src/test/regress/sql/compression.sql | 20 +++
8 files changed, 292 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 1164837..0b2b866 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3974,6 +3976,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4501,7 +4504,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4909,6 +4913,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7774,6 +7782,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and attstorage for the respective index attribute if
+ * the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ Oid newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (OidIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7788,7 +7857,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7852,47 +7920,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidOid, newstorage,
+ lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15046,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208a..07ba55f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3b546da..36b6376 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -165,12 +165,57 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index bb58dfb..0e0df2d 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -157,12 +157,54 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: not built with lz4 support
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index ee46ba0..d2478b4 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -74,6 +74,26 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Sat, Mar 06, 2021 at 08:59:16PM +0530, Dilip Kumar wrote:
- Alter table set compression, will not rewrite the old data, so only
the new tuple will be compressed with the new compression method.
- No preserve.
+1, this simplifies things. If someone *wants* to rewrite the table, they can
VACUUM FULL, CLUSTER, or dump+restore.
I checked that it's possible to do simple column manipulations on columns
written with-lz4 with binaries built without-lz4:
- is null
- texteq if length differs
- explain analyze
If I pg_upgrade from an binary with-lz4 to one without-lz4, it fails
while restoring the schema, after running check, which is bad:
| pg_restore: error: could not execute query: ERROR: not built with lz4 support
|CREATE TABLE "public"."a" (
| "t" "text" COMPRESSION lz4,
For comparison, upgrading from binaries with-libxml to binaries without-libxml
actualy passes pg_upgrade.
It's arguable which behavior is desirable:
- allow CREATE TABLE(..COMPRESSION lz4) during pg_upgrade;
- allow CREATE TABLE(..COMPRESSION lz4) always. This has the advantage that
GetAttributeCompression() doesn't have conditional compilation. This seems
to be parallel to the libxml case - apparently, it's possible to create an
XML column, but not insert into it.
- abort pg_upgrade --check if the old cluster has lz4 and the new one doesn't,
if there are any lz4 compressed columns. This avoids the possibilty of
running an upgrade to binaries without lz4, starting a new cluster (which
leaves the old cluster unsafe to start if --link was used), and then the new
cluster may even appear to work, until an LZ4 column is accessed in a
nontrivial way. It has the disadvantage that there's no obvious parallel in
pg_upgrade (checksums and xml are the closest?). And the disadvantage that
some people might *want* the upgrade to succeed in that case to then recompile
with lz4 afterwards.
In this patch, SET default_toast_compression=lz4 "works" even if without-lz4,
but then CREATE TABLE fails. You should either allow table creation (as
above), or check in check_default_toast_compression() if lz4 is enabled.
Its comment about "catalog access" is incorrect now.
Now, I wonder if default_toast_compression should be a GUC, or a reloption.
An obvious advantage of being a GUC is that regression tests are trivial with
make installcheck.
Some minor fixes:
+ if (strcmp(def->compression, newdef->compression))
!= 0
+ * NULL for non varlena type or the uncompressed data.
remove "the"
+ * InvalidOid for the plain/external storage otherwise default
remove "the"
+ behavior is to exclude compression methods, resulting in the columns
remove "the"
+ attcompression and attstorage for the respective index attribute if ... the respective input values are
say "and/or attstorage"
+ If this variable is set to <literal>true</literal>, column's [...]
I wrote this, but I guess it should say: columns'
I think you could also say either of these:
.. column compression method details are not displayed.
.. details of column compression are not displayed.
--
Justin
Re: [HACKERS] Custom compression methods
On Sun, Mar 7, 2021 at 1:27 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Sat, Mar 06, 2021 at 08:59:16PM +0530, Dilip Kumar wrote:
- Alter table set compression, will not rewrite the old data, so only
the new tuple will be compressed with the new compression method.
- No preserve.+1, this simplifies things. If someone *wants* to rewrite the table, they can
VACUUM FULL, CLUSTER, or dump+restore.
Thanks
I checked that it's possible to do simple column manipulations on columns
written with-lz4 with binaries built without-lz4:
- is null
- texteq if length differs
- explain analyze
That's because unless you really need to compress/decompress the data
it won't error out. IMHO this behavior is fine as this is the
behavior with other libraries also e.g libxml
If I pg_upgrade from an binary with-lz4 to one without-lz4, it fails
while restoring the schema, after running check, which is bad:
| pg_restore: error: could not execute query: ERROR: not built with lz4 support
|CREATE TABLE "public"."a" (
| "t" "text" COMPRESSION lz4,For comparison, upgrading from binaries with-libxml to binaries without-libxml
actualy passes pg_upgrade.It's arguable which behavior is desirable:
- allow CREATE TABLE(..COMPRESSION lz4) during pg_upgrade;
- allow CREATE TABLE(..COMPRESSION lz4) always. This has the advantage that
GetAttributeCompression() doesn't have conditional compilation. This seems
to be parallel to the libxml case - apparently, it's possible to create an
XML column, but not insert into it.
IMHO we can always allow creating the table with lz4 and only error
out when we really need to compress/decompress the data. I like this
behavior because it is the same as libxml. But I am fine with
allowing it only in binary upgrade also. Another option could be to
fall back to default "pglz" in binary upgrade mode if it is built
without-lz4 but the problem is this will change the table
specification after the upgrade. So maybe we can go with either of
the first two options. Any other thoughts on this?
- abort pg_upgrade --check if the old cluster has lz4 and the new one doesn't,
if there are any lz4 compressed columns. This avoids the possibilty of
running an upgrade to binaries without lz4, starting a new cluster (which
leaves the old cluster unsafe to start if --link was used), and then the new
cluster may even appear to work, until an LZ4 column is accessed in a
nontrivial way. It has the disadvantage that there's no obvious parallel in
pg_upgrade (checksums and xml are the closest?). And the disadvantage that
some people might *want* the upgrade to succeed in that case to then recompile
with lz4 afterwards.
Yeah.
In this patch, SET default_toast_compression=lz4 "works" even if without-lz4,
but then CREATE TABLE fails. You should either allow table creation (as
above), or check in check_default_toast_compression() if lz4 is enabled.
Its comment about "catalog access" is incorrect now.
I will fix the comment, I agree that only if we allow to always
create a table then only it makes sense to set the default as lz4 if
it is compiled without lz4.
Now, I wonder if default_toast_compression should be a GUC, or a reloption.
An obvious advantage of being a GUC is that regression tests are trivial with
make installcheck.
I don't think it makes much sense to give a table-wise option, we
anyways have the option to give a compression method per attribute and
I think selecting the compression method more depends upon the
attribute data and type. So I think providing the GUC makes more
sense when the user wants to select some default compression method
for most of its attributes.
Some minor fixes:
+ if (strcmp(def->compression, newdef->compression))
!= 0+ * NULL for non varlena type or the uncompressed data.
remove "the"+ * InvalidOid for the plain/external storage otherwise default
remove "the"+ behavior is to exclude compression methods, resulting in the columns
remove "the"+ attcompression and attstorage for the respective index attribute if ... the respective input values are
say "and/or attstorage"+ If this variable is set to <literal>true</literal>, column's [...]
I wrote this, but I guess it should say: columns'I think you could also say either of these:
.. column compression method details are not displayed.
.. details of column compression are not displayed.
Thanks, I will fix these in the next version.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sun, Mar 07, 2021 at 12:16:41PM +0530, Dilip Kumar wrote:
If I pg_upgrade from an binary with-lz4 to one without-lz4, it fails
while restoring the schema, after running check, which is bad:
| pg_restore: error: could not execute query: ERROR: not built with lz4 support
|CREATE TABLE "public"."a" (
| "t" "text" COMPRESSION lz4,
Actually, it looks like pg_upgrading an xml column works, but calling xml
functions fails.
I think that's a deficiency in pg_upgrade - it should be caught early during
the --check phase and not after dumping and in the middle of restoring the
schema (which can sometimes take significant time).
For comparison, upgrading from binaries with-libxml to binaries without-libxml
actualy passes pg_upgrade.It's arguable which behavior is desirable:
- allow CREATE TABLE(..COMPRESSION lz4) during pg_upgrade;
- allow CREATE TABLE(..COMPRESSION lz4) always. This has the advantage that
GetAttributeCompression() doesn't have conditional compilation. This seems
to be parallel to the libxml case - apparently, it's possible to create an
XML column, but not insert into it.IMHO we can always allow creating the table with lz4 and only error
out when we really need to compress/decompress the data. I like this
behavior because it is the same as libxml. But I am fine with
allowing it only in binary upgrade also. Another option could be to
fall back to default "pglz" in binary upgrade mode if it is built
without-lz4 but the problem is this will change the table
specification after the upgrade.
No, you certainly can't do that.
You'd have a table defined as pglz but with lz4 in the data files.
In the best case, it would give errors about corrupt lz4 data.
--
Justin
Re: [HACKERS] Custom compression methods
On Sun, Mar 7, 2021 at 12:47 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
IMHO we can always allow creating the table with lz4 and only error
out when we really need to compress/decompress the data. I like this
behavior because it is the same as libxml. But I am fine with
allowing it only in binary upgrade also. Another option could be to
fall back to default "pglz" in binary upgrade mode if it is built
without-lz4 but the problem is this will change the table
specification after the upgrade.No, you certainly can't do that.
You'd have a table defined as pglz but with lz4 in the data files.
In the best case, it would give errors about corrupt lz4 data.
Yeah, we can not do that. Just missed that part :)
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sun, Mar 07, 2021 at 01:36:50PM +0530, Dilip Kumar wrote:
On Sun, Mar 7, 2021 at 12:47 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
IMHO we can always allow creating the table with lz4 and only error
out when we really need to compress/decompress the data. I like this
behavior because it is the same as libxml. But I am fine with
allowing it only in binary upgrade also. Another option could be to
fall back to default "pglz" in binary upgrade mode if it is built
without-lz4 but the problem is this will change the table
specification after the upgrade.No, you certainly can't do that.
You'd have a table defined as pglz but with lz4 in the data files.
In the best case, it would give errors about corrupt lz4 data.Yeah, we can not do that. Just missed that part :)
But I believe what you're thinking is that it ought to be possible to restore a
backup, even if the binaries in the target cluster don't support the
compression used by the source tables.
Earlier in this thread, I suggested to implement an option to pg_restore to
avoid outputting compression, in order to allow restoring with a different
compression (by using the default_toast_compression GUC). Now, it seems like
that's even more important, to allow restoring into binaries --without-lz4.
(the pg_dump isn't in LZ4 format, it just needs to not say "COMPRESSION LZ4").
I think you're planning to allow the CREATE TABLE to succeed in any case, but
it's not helpful if the DBA has to restore the schema, and then alter all the
text columns to set PGLZ, and then restore the data and post-data.
Also, I suggest to rename the pg_dump option:
| --no-compression-methods do not dump compression methods
I have a patch to pg_dump to support alternate compression in the dump itself
(in addition to zlib), so the name will be confusing. I suggest
--no-toast-compression, like the GUC. And the same for pg_restore.
--
Justin
Re: [HACKERS] Custom compression methods
On Sun, Mar 7, 2021 at 2:19 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Earlier in this thread, I suggested to implement an option to pg_restore to
avoid outputting compression, in order to allow restoring with a different
compression (by using the default_toast_compression GUC). Now, it seems like
that's even more important, to allow restoring into binaries --without-lz4.
(the pg_dump isn't in LZ4 format, it just needs to not say "COMPRESSION LZ4").
IMHO, we have an option with pg_dump that should be sufficient, no?
but I agree that having such an option with restore will give more
flexibility basically, by using the same dump we can restore to binary
--with-lz4 as well as without-lz4 if such option exists with restore
as well. But it seems in pg_restore we process token by token so if
we want to implement such an option then I think we will have to parse
the complete string of CREATE TABLE command and remove the compression
option if it exists for any attribute. I am not sure whether providing
this option is worth the complexity?
I think you're planning to allow the CREATE TABLE to succeed in any case, but
it's not helpful if the DBA has to restore the schema, and then alter all the
text columns to set PGLZ, and then restore the data and post-data.Also, I suggest to rename the pg_dump option:
| --no-compression-methods do not dump compression methodsI have a patch to pg_dump to support alternate compression in the dump itself
(in addition to zlib), so the name will be confusing. I suggest
--no-toast-compression, like the GUC. And the same for pg_restore.
Ok.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sun, Mar 7, 2021 at 1:27 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Sat, Mar 06, 2021 at 08:59:16PM +0530, Dilip Kumar wrote:
- Alter table set compression, will not rewrite the old data, so only
the new tuple will be compressed with the new compression method.
- No preserve.In this patch, SET default_toast_compression=lz4 "works" even if without-lz4,
but then CREATE TABLE fails. You should either allow table creation (as
above), or check in check_default_toast_compression() if lz4 is enabled.
Its comment about "catalog access" is incorrect now.
As of now I have made GUC behavior similar to the CREATE TABLE, in
both case it will throw an error if it is not compiled with lz4
method.
+ if (strcmp(def->compression, newdef->compression))
!= 0+ * NULL for non varlena type or the uncompressed data.
remove "the"+ * InvalidOid for the plain/external storage otherwise default
remove "the"+ behavior is to exclude compression methods, resulting in the columns
remove "the"+ attcompression and attstorage for the respective index attribute if ... the respective input values are
say "and/or attstorage"+ If this variable is set to <literal>true</literal>, column's [...]
I wrote this, but I guess it should say: columns'I think you could also say either of these:
.. column compression method details are not displayed.
.. details of column compression are not displayed.
I have fixed the above comments, and also some other minor fixup.
So now only pending point is, how do we handle the upgrade when you
are upgrading from --with-lz4 to --without-lz4 binary and a couple of
options discussed here are
a) Should we allow table creation with lz4 even if it is compiled
--without-lz4? In case of xml we always allow table creation even if
it is compiled --wthout-libxml
b) Instead of allowing this always, only allow during binary upgrade.
With this we will be able to make binary upgrades successful.
However, if users have to access the data compiled with the lz4 then
then need to enable the lz4 support. But I think that is true with
any case for example with a hot standby setup if the user configures
the standby without lz4 then also for accessing the lz4 compressed
data user needs to enable the lz4 support.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v32-0002-Add-default_toast_compression-GUC.patchapplication/x-patch; name=v32-0002-Add-default_toast_compression-GUC.patchDownload
From 5a3367b9b905b1f5b349bd49ab8b37c474d08490 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sat, 6 Mar 2021 16:05:12 +0530
Subject: [PATCH v32 2/4] Add default_toast_compression GUC
Justin Pryzby with modification from Dilip Kumar
---
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 10 +++--
src/backend/utils/misc/guc.c | 64 +++++++++++++++++++++++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/compressapi.h | 18 +++++++-
6 files changed, 89 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index d9f018c..94e50f9 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 3263460..4142e8c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 405d1cbc..45d3496 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -121,6 +121,8 @@ typedef struct OnCommitItem
static List *on_commits = NIL;
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
/*
* State information for ALTER TABLE
@@ -11930,7 +11932,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17744,9 +17746,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..5d0b7ec 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/compressapi.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -227,6 +228,7 @@ static bool check_recovery_target_lsn(char **newval, void **extra, GucSource sou
static void assign_recovery_target_lsn(const char *newval, void *extra);
static bool check_primary_slot_name(char **newval, void **extra, GucSource source);
static bool check_default_with_oids(bool *newval, void **extra, GucSource source);
+static bool check_default_toast_compression(char **newval, void **extra, GucSource source);
/* Private functions in guc-file.l that need to be called from guc.c */
static ConfigVariable *ProcessConfigFileInternal(GucContext context,
@@ -3917,6 +3919,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
@@ -12229,4 +12242,55 @@ check_default_with_oids(bool *newval, void **extra, GucSource source)
return true;
}
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ /*
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.
+ */
+ if (IsTransactionState() && MyDatabaseId != InvalidOid)
+ {
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+ }
+
+ return true;
+}
+
#include "guc-file.c"
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/compressapi.h b/src/include/access/compressapi.h
index e9073b3..fa988ed 100644
--- a/src/include/access/compressapi.h
+++ b/src/include/access/compressapi.h
@@ -43,12 +43,15 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
+
+extern char *default_toast_compression;
+
/* compression handler routines */
typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
@@ -160,4 +163,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
+
#endif /* COMPRESSAPI_H */
--
1.8.3.1
v32-0004-default-to-with-lz4.patchapplication/x-patch; name=v32-0004-default-to-with-lz4.patchDownload
From a8c05203659c9e6138494d570b16ea1a969452de Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 13 Feb 2021 22:11:24 -0600
Subject: [PATCH v32 4/4] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 0895b2f..79e9d22 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index da7cee0..78f5065 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_SUBST(with_lz4)
#
--
1.8.3.1
v32-0003-Alter-table-set-compression.patchapplication/x-patch; name=v32-0003-Alter-table-set-compression.patchDownload
From 5f7eeb3a21160e59841a57d03854f6ca56bdc24d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sat, 6 Mar 2021 13:10:23 +0530
Subject: [PATCH v32 3/4] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 48 ++++++-
src/test/regress/sql/compression.sql | 20 +++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 45d3496..c75c94a 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -530,6 +530,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3974,6 +3976,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4501,7 +4504,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4909,6 +4913,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7774,6 +7782,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7788,7 +7857,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7852,47 +7920,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15018,6 +15047,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208a..07ba55f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3b546da..36b6376 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -165,12 +165,57 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index c7fd3cc..62ce8cd 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -161,12 +161,58 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index ee46ba0..d2478b4 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -74,6 +74,26 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v32-0001-Built-in-compression-method.patchapplication/x-patch; name=v32-0001-Built-in-compression-method.patchDownload
From 6a4fc6b03d0f75d473953ff224f535343fc68ec3 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v32 1/4] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 116 ++++++++++++++
configure.ac | 17 +++
doc/src/sgml/ref/create_table.sgml | 33 +++-
doc/src/sgml/ref/psql-ref.sgml | 11 ++
src/backend/access/Makefile | 2 +-
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/detoast.c | 71 ++++++---
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_internals.c | 54 +++----
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/compression/Makefile | 17 +++
src/backend/access/compression/compress_lz4.c | 149 ++++++++++++++++++
src/backend/access/compression/compress_pglz.c | 137 +++++++++++++++++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 111 +++++++++++++-
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +++-
src/backend/parser/parse_utilcmd.c | 9 ++
src/backend/utils/adt/varlena.c | 41 +++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 +++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 ++
src/include/access/compressapi.h | 163 ++++++++++++++++++++
src/include/access/detoast.h | 8 +
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +--
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 12 +-
src/test/regress/expected/compression.out | 201 +++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 193 ++++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 84 +++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
53 files changed, 1558 insertions(+), 87 deletions(-)
create mode 100644 src/backend/access/compression/Makefile
create mode 100644 src/backend/access/compression/compress_lz4.c
create mode 100644 src/backend/access/compression/compress_pglz.c
create mode 100644 src/include/access/compressapi.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..0895b2f 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,39 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+
+
+#
# Assignments
#
@@ -12054,6 +12090,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13408,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..da7cee0 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,14 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1181,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1418,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..7c8fc77 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/Makefile b/src/backend/access/Makefile
index 0880e0a..ba08bdd 100644
--- a/src/backend/access/Makefile
+++ b/src/backend/access/Makefile
@@ -8,7 +8,7 @@ subdir = src/backend/access
top_builddir = ../../..
include $(top_builddir)/src/Makefile.global
-SUBDIRS = brin common gin gist hash heap index nbtree rmgrdesc spgist \
+SUBDIRS = brin common compression gin gist hash heap index nbtree rmgrdesc spgist \
table tablesample transam
include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..a3d259c 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,6 +458,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+Oid
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
* toast_decompress_datum -
*
* Decompress a compressed version of a varlena datum
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..69dd949 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..d9f018c 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/compression/Makefile b/src/backend/access/compression/Makefile
new file mode 100644
index 0000000..5c48b2e
--- /dev/null
+++ b/src/backend/access/compression/Makefile
@@ -0,0 +1,17 @@
+#-------------------------------------------------------------------------
+#
+# Makefile for access/compression
+#
+# Copyright (c) 2021, PostgreSQL Global Development Group
+#
+# src/backend/access/compression/Makefile
+#
+#-------------------------------------------------------------------------
+
+subdir = src/backend/access/compression
+top_builddir = ../../../..
+include $(top_builddir)/src/Makefile.global
+
+OBJS = compress_pglz.o compress_lz4.o
+
+include $(top_srcdir)/src/backend/common.mk
diff --git a/src/backend/access/compression/compress_lz4.c b/src/backend/access/compression/compress_lz4.c
new file mode 100644
index 0000000..75a057b
--- /dev/null
+++ b/src/backend/access/compression/compress_lz4.c
@@ -0,0 +1,149 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_lz4.c
+ * lz4 compression method.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_lz4.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/compressapi.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* Definition of the lz4 compression method */
+const CompressionRoutine lz4_compress_methods = {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+};
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
diff --git a/src/backend/access/compression/compress_pglz.c b/src/backend/access/compression/compress_pglz.c
new file mode 100644
index 0000000..9bd5370
--- /dev/null
+++ b/src/backend/access/compression/compress_pglz.c
@@ -0,0 +1,137 @@
+/*-------------------------------------------------------------------------
+ *
+ * compress_pglz.c
+ * pglz compression method
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/compression/compress_pglz.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#include "access/compressapi.h"
+#include "common/pg_lzcompress.h"
+
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* Definition of the pglz compression method */
+const CompressionRoutine pglz_compress_methods = {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..3263460 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/compressapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..558bc17 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..9383afd 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/compressapi.h"
#include "access/heapam.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..405d1cbc 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/compressapi.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -558,7 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
-
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
* DefineRelation
@@ -852,6 +853,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2409,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2459,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2709,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6387,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11918,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17717,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..e254806 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/compressapi.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..e4266e0 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,6 +17,7 @@
#include <ctype.h>
#include <limits.h>
+#include "access/compressapi.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -5300,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..5269f1f 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compressions;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..09ce898 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compressions", no_argument, &dopt.no_toast_compressions, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compressions do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compressions &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..d956b03 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a43a8f5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/compressapi.h b/src/include/access/compressapi.h
new file mode 100644
index 0000000..e9073b3
--- /dev/null
+++ b/src/include/access/compressapi.h
@@ -0,0 +1,163 @@
+/*-------------------------------------------------------------------------
+ *
+ * compressapi.h
+ * API for Postgres compression methods.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/compressapi.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef COMPRESSAPI_H
+#define COMPRESSAPI_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern const CompressionRoutine pglz_compress_methods;
+extern const CompressionRoutine lz4_compress_methods;
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in compressapi.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in compressapi.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+static inline char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(pglz_compress_methods.cmname, compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(lz4_compress_methods.cmname, compression) == 0)
+ {
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+static inline const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return &pglz_compress_methods;
+ case LZ4_COMPRESSION:
+ return &lz4_compress_methods;
+ default:
+ elog(ERROR, "invalid compression method %u", method);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+#endif /* COMPRESSAPI_H */
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..d958af0 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_oid -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern Oid toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..15dc5e6 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/compressapi.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..faa5456 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7098,6 +7098,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..488413a 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..695b475 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * flags */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,21 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/* va_tcinfo in va_compress contains raw size of datum and optional flags */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARFLAGS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3b546da
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,201 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..c7fd3cc
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,193 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..ee46ba0
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,84 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..daf163c 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..0fc7017 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Sun, Mar 07, 2021 at 06:04:41PM +0530, Dilip Kumar wrote:
On Sun, Mar 7, 2021 at 2:19 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Earlier in this thread, I suggested to implement an option to pg_restore to
avoid outputting compression, in order to allow restoring with a different
compression (by using the default_toast_compression GUC). Now, it seems like
that's even more important, to allow restoring into binaries --without-lz4.
(the pg_dump isn't in LZ4 format, it just needs to not say "COMPRESSION LZ4").IMHO, we have an option with pg_dump that should be sufficient, no?
I think it's insufficient since people may be unable to restore from backup, or
can only restore backup by resorting to this:
pg_restore -f- |sed 's/COMPRESSION lz4//' |psql -d mydb
I think there's a parallel with --no-tablespaces. But if a tablespace is
missing/renamed, the table is still restored (with errors during SET
default_tablespace), whereas if lz4 is missing, the table is not restored.
Or actually the table would be created, but the COPY/INSERTs would fail.
There's an argument to be made that this is already an issue - for example,
I've numerous times done a partial restore of a single partition, where the
column types have changed in the parent, and I need to use sed to restore the
partition. However, that's improving - in v14: "attach table" is a separate
pg_dump object, so the table *is* restored, and only the ATTACH command fails.
(See 9a4c0e36f).
I wonder if COMPRESSION should be dumped as ALTER statements, not in the
CREATE. In fact, the CREATE syntax is optional and could be removed. Similar
to ALTER TABLE t ALTER c SET STATISTICS 99 - there's no CREATE grammar for
that.
Note that I think that using ALTER doesn't resolves this issue, since the
createStmt is sent using the simple query protocol (PQexec), which means that
all its commands are executed as a single transaction, and if the ALTER to LZ4
fails, so does the preceding CREATE. This is the same issue I see with
"CREATE..ATTACH PARTITION", above.
but I agree that having such an option with restore will give more
flexibility basically, by using the same dump we can restore to binary
--with-lz4 as well as without-lz4 if such option exists with restore
as well. But it seems in pg_restore we process token by token so if
we want to implement such an option then I think we will have to parse
the complete string of CREATE TABLE command and remove the compression
option if it exists for any attribute. I am not sure whether providing
this option is worth the complexity?
Oh...I realize now that this is different from the "tablespace" case in that
the column compression is not stored separately in the dump. And because it
exists for each column, not for the whole table. I suppose one answer to that
would be to make compression a per-table reloption, rather than a per-attribute
option. (I can anticipate that you'll hate this idea.)
--
Justin
Re: [HACKERS] Custom compression methods
On Mon, Mar 8, 2021 at 5:02 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
So now only pending point is, how do we handle the upgrade when you
are upgrading from --with-lz4 to --without-lz4 binary and a couple of
options discussed here are
a) Should we allow table creation with lz4 even if it is compiled
--without-lz4? In case of xml we always allow table creation even if
it is compiled --wthout-libxml
b) Instead of allowing this always, only allow during binary upgrade.
I think the basic answer to (a) that it doesn't matter. Suppose the
user is not upgrading but just feels like creating a table that is
configured to use LZ4 compression. Does it really matter whether they
get the error when they create the table or when they load the data?
Personally, I think it is slightly more user-friendly to give the
error when they try to create the table, because the problem doesn't
occur when inserting ANY row, but only when inserting rows that are
wide enough that compression will occur. It's not that great to have
people create a table and then find out only much later that it
doesn't work. On the other hand, consistency with the way the xml data
type already works seems like a fair enough argument for letting the
error happen when we try to actually use the compression method. So I
can't get worked up about it either way.
Regarding (b), it seems to me that with this approach, we have to
document that pg_upgrade from binaries that support LZ4 to binaries
that don't support LZ4 is fundamentally unsafe. You might have
LZ4-compressed values in your columns even if they are now set to use
PGLZ, and you might have LZ4'd data inside composite values that are
on disk someplace. We have no idea whether those things are true or
not, and we can't prevent you from upgrading to something that makes
part of your data inaccessible. Given that, if we go with this
approach, I think we should expend exactly 0 code trying to making
pg_upgrade pass in cases where there are LZ4 columns in the database
and the new binaries don't support LZ4. Just because the user goes and
gets rid of all the LZ4 columns before upgrading doesn't mean that the
upgrade is safe, but if they haven't even done that much, maybe they
should reconsider things a bit.
Some other review comments:
toast_get_compression_method() should now return char, not Oid.
With this design, we can support changing the compression method on a
column quite easily. It's just a hint, like the STORAGE parameter. It
has no bearing on what can be present in the table, but just controls
how new values are stored. It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.
I am tempted to propose that we collapse compress_lz4.c and
compress_pglz.c into a single file, get rid of the directory, and just
have something like src/backend/access/common/toast_compression.c. The
files are awfully short, and making a whole new directory for that
small amount of code seems like overkill.
I think the pg_dump argument should be --no-toast-compression, not
--no-toast-compressions. I agree with Justin that pg_restore should
have the option also.
Man, it would be really nice to be able to set the default for new
tables, rather than having all these places hard-coded to use
DefaultCompressionMethod. Surely lotsa people are going to want to set
toast_compression = lz4 in postgresql.conf and forget about it.
Is there any reason not to change varattrib_4b's description of
va_tcinfo that says "and flags" to instead say "and compression
method"? And rename VARFLAGS_4B_C to VARCOMPRESS_4B_C? I don't know
why we should call it flags when we know it's specifically compression
information.
You should probably have a test that involves altering the type of a
varlena column to non-varlena, and the other way around, and make sure
that changing integer -> text sets attcompression and doing the
reverse clears it.
You need to update catalogs.sgml.
On the whole I don't see a whole lot to complain about here. I don't
love giving up on the idea of tracking which compression methods are
used where, but making that work without performance regressions seems
very difficult and perhaps just outright impossible, and dealing with
all the concurrency problems that introduces is a pain, too. I think
accepting a feature that gives us LZ4 compression is better than
rejecting it because we can't solve those problems.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 08, 2021 at 03:26:04PM -0500, Robert Haas wrote:
On Mon, Mar 8, 2021 at 5:02 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
So now only pending point is, how do we handle the upgrade when you
are upgrading from --with-lz4 to --without-lz4 binary and a couple of
options discussed here are
a) Should we allow table creation with lz4 even if it is compiled
--without-lz4? In case of xml we always allow table creation even if
it is compiled --wthout-libxml
b) Instead of allowing this always, only allow during binary upgrade.
It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.
Doesn't vacuum full/cluster/dump+restore do that ?
I think the pg_dump argument should be --no-toast-compression, not
--no-toast-compressions. I agree with Justin that pg_restore should
have the option also.
I mentioned that this is hard to do, since the compression is stored inside the
text blob that creates the whole table...Unless toast compression is a
per-relation property rather than per-attribute. I don't think pg_restore
should try to reverse-engineer the text output by pg_dump to elide the
"COMPRESSION lz4".
I think maybe CREATE shouldn't support COMPRESSION at all, and pg_dump/restore
would use ALTER. That makes this very slightly less of an issue, as one can
use pg_restore -f- |grep -v '^ALTER TABLE .* SET COMPRESSION' |psql -d,
rather than sed 's/COMPRESSION lz4//'
Man, it would be really nice to be able to set the default for new
tables, rather than having all these places hard-coded to use
DefaultCompressionMethod. Surely lotsa people are going to want to set
toast_compression = lz4 in postgresql.conf and forget about it.
I don't understand - isn't that what 0002 does ?
Subject: [PATCH v32 2/4] Add default_toast_compression GUC
--
Justin
Re: [HACKERS] Custom compression methods
On Mon, Mar 8, 2021 at 3:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.Doesn't vacuum full/cluster/dump+restore do that ?
Well, dump and restore will do it, certainly, but I don't think VACUUM
FULL or CLUSTER will. I haven't tested it, though, so maybe I'm wrong.
I think the pg_dump argument should be --no-toast-compression, not
--no-toast-compressions. I agree with Justin that pg_restore should
have the option also.I mentioned that this is hard to do, since the compression is stored inside the
text blob that creates the whole table...Unless toast compression is a
per-relation property rather than per-attribute. I don't think pg_restore
should try to reverse-engineer the text output by pg_dump to elide the
"COMPRESSION lz4".
Oh, yeah. I guess we have to leave that out then.
I think maybe CREATE shouldn't support COMPRESSION at all, and pg_dump/restore
would use ALTER. That makes this very slightly less of an issue, as one can
use pg_restore -f- |grep -v '^ALTER TABLE .* SET COMPRESSION' |psql -d,
rather than sed 's/COMPRESSION lz4//'
TBH, that doesn't seem very nice to me. I think it's usually better to
create objects with the right properties initially rather than
creating them wrong and then fixing them afterwards.
Man, it would be really nice to be able to set the default for new
tables, rather than having all these places hard-coded to use
DefaultCompressionMethod. Surely lotsa people are going to want to set
toast_compression = lz4 in postgresql.conf and forget about it.I don't understand - isn't that what 0002 does ?
Oh, man, you want me to look at all the patches and not just the first one? :-)
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 08, 2021 at 03:32:39PM +0530, Dilip Kumar wrote:
On Sun, Mar 7, 2021 at 1:27 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Sat, Mar 06, 2021 at 08:59:16PM +0530, Dilip Kumar wrote:
- Alter table set compression, will not rewrite the old data, so only
the new tuple will be compressed with the new compression method.
- No preserve.In this patch, SET default_toast_compression=lz4 "works" even if without-lz4,
but then CREATE TABLE fails. You should either allow table creation (as
above), or check in check_default_toast_compression() if lz4 is enabled.
Its comment about "catalog access" is incorrect now.As of now I have made GUC behavior similar to the CREATE TABLE, in
both case it will throw an error if it is not compiled with lz4
method.
In the latest patch, CREATE TABLE (t text COMPRESS lz4) fails if --without-lz4.
I think that's the right choice, since otherwise we should also allow ALTER SET
COMPRESSION lz4, which feels wrong.
This comment and associated conditional is still wrong, since there's no
catalog access anymore:
+ * If we aren't inside a transaction, or not connected to a database, we
+ * cannot do the catalog access necessary to verify the method. Must
+ * accept the value on faith.This shouldn't refer to "access method" (probably originally my error):
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent table access method, only a NOTICE. See comments in
+ * guc.h.I'm not sure why that function is now in guc.c ?
Now I think this comment should just say /* GUC */
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;It sounds like after that, you should merge that part into 0001.
In 0001, configure.ac is missing this :
AC_MSG_RESULT([$with_lz4])
Also, Thomas updated the mac CI to installed LZ4.
However it fails to find the library. Maybe configure.ac needs to use
pkg-config. Or maybe the mac build needs to use this - we're not sure.
--with-includes=/usr/local/opt --with-libraries=/usr/local/opt
--
Justin
Re: [HACKERS] Custom compression methods
On Tue, Mar 9, 2021 at 2:45 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Mar 8, 2021 at 3:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.Doesn't vacuum full/cluster/dump+restore do that ?
Well, dump and restore will do it, certainly, but I don't think VACUUM
FULL or CLUSTER will. I haven't tested it, though, so maybe I'm wrong.
Yeah, vacuum full or cluster will not re-compress the data. How about
providing syntax ALTER TABLE <tab-name> ALTER COLUMN <col_name> SET
COMPRESSION <com_name> REWRITE? So if we have given a rewrite then we
will always rewrite the table and in an attempt to rewrite we will
re-compress the data. If REWRITE is given and the compression method
is the same as the existing then also we can not skip the rewrite
because we don't know the history, the user might alter the
compression method multiple times without rewrite.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Mar 09, 2021 at 01:04:10PM +0530, Dilip Kumar wrote:
On Tue, Mar 9, 2021 at 2:45 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Mar 8, 2021 at 3:59 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.Doesn't vacuum full/cluster/dump+restore do that ?
Well, dump and restore will do it, certainly, but I don't think VACUUM
FULL or CLUSTER will. I haven't tested it, though, so maybe I'm wrong.Yeah, vacuum full or cluster will not re-compress the data. How about
providing syntax ALTER TABLE <tab-name> ALTER COLUMN <col_name> SET
COMPRESSION <com_name> REWRITE?
It'd be strange to me if "rewrite" were associated with a column.
Depending on what data strucutures you use, you might accidentally fail to
rewrite the table if someone wrote something like:
postgres=# alter table t alter a set compression pglz REWRITE, alter a set compression pglz;
Although I hope that's something to support someday for columnar AMs, I think
table-rewriting now is done in entirety, and the syntax should reflect that.
--
Justin
Re: [HACKERS] Custom compression methods
On Tue, Mar 9, 2021 at 1:13 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Tue, Mar 09, 2021 at 01:04:10PM +0530, Dilip Kumar wrote:
On Tue, Mar 9, 2021 at 2:45 AM Robert Haas <robertmhaas@gmail.com> wrote:
Yeah, vacuum full or cluster will not re-compress the data. How about
providing syntax ALTER TABLE <tab-name> ALTER COLUMN <col_name> SET
COMPRESSION <com_name> REWRITE?It'd be strange to me if "rewrite" were associated with a column.
Depending on what data strucutures you use, you might accidentally fail to
rewrite the table if someone wrote something like:
postgres=# alter table t alter a set compression pglz REWRITE, alter a set compression pglz;
I think that should not be a problem because in ATControlle, first we
execute all the AlterExec* command and therein we set various flags to
AlteredTableInfo->rewrite, so we can set one flag to compression
rewrite as well if rewrite is given for any column while updating the
compression method.
Although I hope that's something to support someday for columnar AMs, I think
table-rewriting now is done in entirety, and the syntax should reflect that.
I thought it would be good to give the rewrite along with the set
compression, and since we are setting compression column wise so
better to give rewrite also there. If we give rewrite as alter table
syntax and not with the compression then it will be like we can always
give the rewrite option for the table whether we are setting the
compression method or not.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Mar 9, 2021 at 1:56 AM Robert Haas <robertmhaas@gmail.com> wrote:
With this design, we can support changing the compression method on a
column quite easily. It's just a hint, like the STORAGE parameter. It
has no bearing on what can be present in the table, but just controls
how new values are stored. It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.
So you mean if we are not able to decompress the old data because the
binary was not compiled with lz4 then don't give error and continue.
I think that depends upon how we are going to support this option for
example suppose we are setting as ALTER COLUMN f1 SET COMPRESSION pglz
REWRITE, then maybe it make sense that even we are not able to rewrite
because it was not compiled with lz4 we can successfully set the new
compression method to pglz.
Another thing is that if the table has some rowtype column then we
will have to process that and decompress any compressed field inside
that right? I haven't yet thought how complex it will be to
decompress the data stored inside an already formed composite type but
I will analyze this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Mar 9, 2021 at 1:56 AM Robert Haas <robertmhaas@gmail.com> wrote:
Some other review comments:
I have worked on these review comments. Please find my response inline
toast_get_compression_method() should now return char, not Oid.
Fixed
With this design, we can support changing the compression method on a
column quite easily. It's just a hint, like the STORAGE parameter. It
has no bearing on what can be present in the table, but just controls
how new values are stored. It would be nice to have a way to force
anything compressed with the old method to be re-compressed with the
new method, but not having that doesn't preclude allowing the
parameter to be changed.
As responded upthread, as of now I am planning to provide a syntax as
ALTER COLUMN col SET COMPRESSION method REWRITE, if user wants to
rewrite the table.
I am tempted to propose that we collapse compress_lz4.c and
compress_pglz.c into a single file, get rid of the directory, and just
have something like src/backend/access/common/toast_compression.c. The
files are awfully short, and making a whole new directory for that
small amount of code seems like overkill.
I have done that, along with that I have also renamed compressapi.h to
toast_compression.h, and along with that I have done some more
refactoring of the code especially in toast_compression.c and
toast_compression.h, please have a look.
I think the pg_dump argument should be --no-toast-compression, not
--no-toast-compressions.
Done
I agree with Justin that pg_restore should
have the option also.
Not done anything for pg_restore as we already agreed upon this.
Man, it would be really nice to be able to set the default for new
tables, rather than having all these places hard-coded to use
DefaultCompressionMethod. Surely lotsa people are going to want to set
toast_compression = lz4 in postgresql.conf and forget about it.
As Justine pointed out we are doing in 0002, maybe we should merge
0001 and 0002 but I kept is that way so that the review can be easy.
Is there any reason not to change varattrib_4b's description of
va_tcinfo that says "and flags" to instead say "and compression
method"? And rename VARFLAGS_4B_C to VARCOMPRESS_4B_C? I don't know
why we should call it flags when we know it's specifically compression
information.
Done.
You should probably have a test that involves altering the type of a
varlena column to non-varlena, and the other way around, and make sure
that changing integer -> text sets attcompression and doing the
reverse clears it.
Done, also added the test case to see that setting the storage type to
plain on varlena type should not clear the compression method.
You need to update catalogs.sgml.
Done
On the whole I don't see a whole lot to complain about here. I don't
love giving up on the idea of tracking which compression methods are
used where, but making that work without performance regressions seems
very difficult and perhaps just outright impossible, and dealing with
all the concurrency problems that introduces is a pain, too. I think
accepting a feature that gives us LZ4 compression is better than
rejecting it because we can't solve those problems.
Right.
Apart from this I have also fixed the comment given by Justin.
The pending comment is providing a way to rewrite a table and
re-compress the data with the current compression method.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v33-0004-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v33-0004-default-to-with-lz4.patchDownload
From 9bc1e97e7b61ad29622f4f888d8dcc006309076b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v33 4/4] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 8cce80c..637d441 100755
--- a/configure
+++ b/configure
@@ -1571,7 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8592,7 +8592,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 2c7f65b..8aa1793 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
v33-0002-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v33-0002-Add-default_toast_compression-GUC.patchDownload
From ab7efdb1fac9b60ba0873115373f7f2c92c23dc7 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:05:05 +0530
Subject: [PATCH v33 2/4] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++---
src/backend/utils/misc/guc.c | 12 +++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 +++++++++++--
src/test/regress/expected/compression.out | 16 ++++++++++
src/test/regress/expected/compression_1.out | 19 +++++++++++
src/test/regress/sql/compression.sql | 8 +++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 3463b42..e33f926 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -306,3 +309,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 503d64d..a4b34ec 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 45e1cfa..def0ad1 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d295d85..745dfe9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..7809867 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/toast_compression.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3917,6 +3918,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 38800cb..e9d9ce5 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -15,6 +15,14 @@
#include "postgres.h"
+#include "utils/guc.h"
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/* GUCs */
+extern char *default_toast_compression;
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -36,8 +44,6 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -67,6 +73,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -115,5 +123,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 1de3ae2..12ab404 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -211,6 +211,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 2999b3b..cc9b913 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -208,6 +208,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index ab35f3b..0d5a223 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -92,6 +92,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v33-0003-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v33-0003-Alter-table-set-compression.patchDownload
From 83c198679eb365f2b0edaf192d62f9021c5e4032 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:34:08 +0530
Subject: [PATCH v33 3/4] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 48 ++++++-
src/test/regress/sql/compression.sql | 20 +++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9..f9bda94 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7773,6 +7781,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15046,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208a..07ba55f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404..3bfe235 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913..59175b9 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a223..1f417b7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v33-0001-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v33-0001-Built-in-compression-method.patchDownload
From 70799bbf622ff8066584bc627e49035b67a84d89 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v33 1/4] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 118 ++++++++++
configure.ac | 18 ++
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 71 ++++--
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 308 ++++++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 54 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 +++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/utils/adt/varlena.c | 41 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 ++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 8 +
src/include/access/toast_compression.h | 119 ++++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/compression.out | 247 +++++++++++++++++++++
src/test/regress/expected/compression_1.out | 240 ++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 102 +++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
52 files changed, 1645 insertions(+), 85 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..8cce80c 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8564,6 +8567,41 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+#
# Assignments
#
@@ -12054,6 +12092,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13410,36 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..2c7f65b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,15 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1182,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1419,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0..5fdc80f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,16 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..7c8fc77 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..031d20d 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -457,6 +458,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
* toast_decompress_datum -
*
* Decompress a compressed version of a varlena datum
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..3463b42
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,308 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..69dd949 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..503d64d 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..45e1cfa 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/toast_compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d8b71e5 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..2196b44 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/heapam.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..d295d85 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/toast_compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..af772ba 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/toast_compression.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..37876a8 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,6 +17,7 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_compression.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -5300,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..cef9bbd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..d956b03 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a43a8f5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..9a428aa 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..38800cb
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,119 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..284a157 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..faa5456 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7098,6 +7098,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..488413a 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..e98af05 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..1de3ae2
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,247 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..2999b3b
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,240 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..ab35f3b
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,102 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..daf163c 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..0fc7017 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
The pending comment is providing a way to rewrite a table and
re-compress the data with the current compression method.
I spent some time poking at this yesterday and ran couldn't figure out
what was going on here. There are two places where we rewrite tables.
One is the stuff in cluter.c, which handles VACUUM FULL and CLUSTER.
That eventually calls reform_and_rewrite_tuple(), which deforms the
old tuple and creates a new one, but it doesn't seem like there's
anything in there that would expand toasted values, whether external
or inline compressed. But I think that can't be right, because it
seems like then you'd end up with toast pointers into the old TOAST
relation, not the new one, which would cause failures later. So I must
be missing something here. The other place where we rewrite tables is
in ATRewriteTable() as part of the ALTER TABLE machinery. I don't see
anything there to force detoasting either.
That said, I think that using the word REWRITE may not really capture
what we're on about. Leaving aside the question of exactly what the
CLUSTER code does today, you could in theory rewrite the main table by
just taking all the tuples and putting them into a new relfilenode.
And then you could do the same thing with the TOAST table. And despite
having fully rewritten both tables, you wouldn't have done anything
that helps with this problem because you haven't deformed the tuples
at any point. Now as it happens we do have code -- in
reform_and_rewrite_tuple() -- that does deform and reform the tuples,
but it doesn't take care of this problem either. We might need to
distinguish between rewriting the table, which is mostly about getting
a new relfilenode, and some other word that means doing this.
But, I am not really convinced that we need to solve this problem by
adding new ALTER TABLE syntax. I'd be happy enough if CLUSTER, VACUUM
FULL, and versions of ALTER TABLE that already force a rewrite would
cause the compression to be redone also. Honestly, even if the user
had to fall back on creating a new table and doing INSERT INTO newtab
SELECT * FROM oldtab I would consider that to be not a total
showstopper for this .. assuming of course that it actually works. If
it doesn't, we have big problems. Even without the pg_am stuff, we
still need to make sure that we don't just blindly let compressed
values wander around everywhere. When we insert into a table column
with a compression method, we should recompress any data that is
compressed using some other method.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 10, 2021 at 03:50:48PM -0500, Robert Haas wrote:
On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
The pending comment is providing a way to rewrite a table and
re-compress the data with the current compression method.I spent some time poking at this yesterday and ran couldn't figure out
what was going on here. There are two places where we rewrite tables.
One is the stuff in cluter.c, which handles VACUUM FULL and CLUSTER.
That eventually calls reform_and_rewrite_tuple(), which deforms the
old tuple and creates a new one, but it doesn't seem like there's
anything in there that would expand toasted values, whether external
or inline compressed. But I think that can't be right, because it
seems like then you'd end up with toast pointers into the old TOAST
relation, not the new one, which would cause failures later. So I must
be missing something here.
I did this the empirical way.
postgres=# CREATE TABLE t (a text compression lz4);
postgres=# INSERT INTO t SELECT repeat('a',99999);
postgres=# ALTER TABLE t ALTER a SET COMPRESSION pglz;
postgres=# VACUUM FULL t;
postgres=# SELECT pg_column_compression(a) FROM t;
pg_column_compression | lz4
Actually, a --without-lz4 build can *also* VACUUM FULL the lz4 table.
It looks like VACUUM FULL t processes t but not its toast table (which is
strange to me, since VACUUM processes both, but then autovacuum also processes
them independently).
--
Justin
Re: [HACKERS] Custom compression methods
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:
https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 support
--
Justin
Attachments:
0001-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From 601ed9e2966b29e1603fd07f69b8068404753810 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH 1/5] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 118 +++++++
configure.ac | 18 +
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 +-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 71 ++--
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 308 ++++++++++++++++++
src/backend/access/common/toast_internals.c | 54 ++-
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 +++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/utils/adt/varlena.c | 41 +++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 +-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 8 +
src/include/access/toast_compression.h | 119 +++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/compression.out | 247 ++++++++++++++
src/test/regress/expected/compression_1.out | 240 ++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 102 ++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
52 files changed, 1645 insertions(+), 85 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index fad817bb38..761a27965d 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +865,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -1569,6 +1571,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8563,6 +8566,41 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
#
# Assignments
#
@@ -12110,6 +12148,56 @@ fi
fi
+if test "$with_lz4" = yes ; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13376,6 +13464,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes ; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 0ed53571dd..616ce5e1a2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,15 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1174,6 +1183,10 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1407,6 +1420,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes ; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0674..5fdc80ff3d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1355,6 +1355,16 @@
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227683..7c8fc77983 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -288,6 +288,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
@@ -605,6 +625,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edfa4d..01ec9b8b0a 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3863,6 +3863,17 @@ bar
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d63f1..b9aff0ccfd 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..031d20d5fc 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -13,6 +13,7 @@
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
@@ -456,6 +457,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1f6b7b77d4 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000000..3463b42438
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,308 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..69dd9492f6 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..503d64df38 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -19,6 +19,7 @@
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/htup_details.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..45e1cfa56c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -17,6 +17,7 @@
#include <unistd.h>
#include <signal.h>
+#include "access/toast_compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/htup_details.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..9586c29ad0 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1f55..d8b71e5bc3 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -29,6 +29,7 @@
*/
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/genam.h"
#include "access/htup_details.h"
#include "access/multixact.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..397d70d226 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..2196b44aae 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -14,6 +14,7 @@
*/
#include "postgres.h"
+#include "access/toast_compression.h"
#include "access/heapam.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d2e5..d295d85cd2 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/attmap.h"
+#include "access/toast_compression.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heapam_xlog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index da91cbd2b1..0d3b923a38 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2988,6 +2988,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6493a03ff8..75343b8aaf 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2876,6 +2876,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b96d..9d923b5d95 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266caeb4..af772ba6bc 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -27,6 +27,7 @@
#include "postgres.h"
#include "access/amapi.h"
+#include "access/toast_compression.h"
#include "access/htup_details.h"
#include "access/relation.h"
#include "access/reloptions.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..37876a8cb7 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -17,6 +17,7 @@
#include <ctype.h>
#include <limits.h>
+#include "access/toast_compression.h"
#include "access/detoast.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -5299,6 +5300,46 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30a79..0296b9bb5e 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7eb4..cef9bbd76a 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213fb06..d956b035f3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e46464a..bc91bb12ac 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..b284113d55 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index daa5081eac..99a59470c5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -372,6 +372,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d65990059d..83f2e6f254 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..110906a4e9 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1159,6 +1159,13 @@ show_context_hook(const char *newval)
return true;
}
+static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..9a428aae2d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000000..38800cb97a
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,119 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..05104ce237 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..284a15761a 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..560f8f00bb 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7619f8cd3..91c1c5b720 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7099,6 +7099,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a2ca..19d2ba26bf 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 7a7cc21d8d..6007d72a73 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..e98af05b30 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..1de3ae2646
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,247 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..2999b3bb79
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,240 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e280198b17..70c38309d7 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -115,7 +115,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941c24..1524676f3b 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 6a57e889a1..d81d04136c 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -201,6 +201,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..ab35f3b4ae
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,102 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 select large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index a4f5cc4bdb..5f39a92111 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 574a8a94fa..e6cb35a16f 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
0002-f-use-pkgconfig.patchtext/x-diff; charset=us-asciiDownload
From 007c6fa5f25f49c129b3489086c054e7a4227707 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Wed, 10 Mar 2021 19:02:02 -0600
Subject: [PATCH 2/5] f! use pkgconfig
---
configure | 99 +++++++++++++++++++++++++++++++++++++++++++++++++++-
configure.ac | 3 +-
2 files changed, 100 insertions(+), 2 deletions(-)
diff --git a/configure b/configure
index 761a27965d..7ee514e101 100755
--- a/configure
+++ b/configure
@@ -654,6 +654,8 @@ UUID_LIBS
LDAP_LIBS_BE
LDAP_LIBS_FE
with_ssl
+LZ4_LIBS
+LZ4_CFLAGS
PTHREAD_CFLAGS
PTHREAD_LIBS
PTHREAD_CC
@@ -897,6 +899,8 @@ LDFLAGS_EX
LDFLAGS_SL
PERL
PYTHON
+LZ4_CFLAGS
+LZ4_LIBS
MSGFMT
TCLSH'
@@ -1603,6 +1607,8 @@ Some influential environment variables:
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
PYTHON Python program
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
MSGFMT msgfmt program for NLS
TCLSH Tcl interpreter program (tclsh)
@@ -12149,6 +12155,97 @@ fi
fi
if test "$with_lz4" = yes ; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
@@ -13466,7 +13563,7 @@ fi
fi
-if test "$with_lz4" = yes ; then
+if test "$with_lz4" = yes; then
for ac_header in lz4/lz4.h
do :
ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
diff --git a/configure.ac b/configure.ac
index 616ce5e1a2..0697701bfe 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1184,6 +1184,7 @@ Use --without-zlib to disable zlib support.])])
fi
if test "$with_lz4" = yes ; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
fi
@@ -1420,7 +1421,7 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
-if test "$with_lz4" = yes ; then
+if test "$with_lz4" = yes; then
AC_CHECK_HEADERS(lz4/lz4.h, [],
[AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
fi
--
2.17.0
0003-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From 4dfa0e65e32a52208fe6d86b591a8ffea4018385 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:05:05 +0530
Subject: [PATCH 3/5] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++--
src/backend/utils/misc/guc.c | 12 +++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 ++++++++-
src/test/regress/expected/compression.out | 16 +++++++
src/test/regress/expected/compression_1.out | 19 ++++++++
src/test/regress/sql/compression.sql | 8 ++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 3463b42438..e33f92687c 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -306,3 +309,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 503d64df38..a4b34ec570 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 45e1cfa56c..def0ad1dcf 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index d295d85cd2..745dfe9570 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 855076b1fd..321d2eb21e 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -30,6 +30,7 @@
#include <unistd.h>
#include "access/commit_ts.h"
+#include "access/toast_compression.h"
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index f46c2dd7a8..7d7a433dcc 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -659,6 +659,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 38800cb97a..e9d9ce5634 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -15,6 +15,14 @@
#include "postgres.h"
+#include "utils/guc.h"
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/* GUCs */
+extern char *default_toast_compression;
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -36,8 +44,6 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -67,6 +73,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -115,5 +123,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 1de3ae2646..12ab404900 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -211,6 +211,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 2999b3bb79..cc9b913418 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -208,6 +208,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index ab35f3b4ae..0d5a2231d0 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -92,6 +92,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0004-Alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From 1396ab54354d8935cd959291e1e81d976156e4ea Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:34:08 +0530
Subject: [PATCH 4/5] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5abd6..0bd0c1a503 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -383,6 +385,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 745dfe9570..f9bda942a3 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7772,6 +7780,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15016,6 +15045,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5d95..5c4e779ca0 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index ecdb8d752b..2071a29bf0 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba26bf..f9a87dee02 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 12ab404900..3bfe2358f4 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -227,12 +227,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index cc9b913418..59175b935b 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 0d5a2231d0..1f417b7d94 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0005-default-to-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From f39801fc8eb11ae3e4e1979cba1ec209b0e90035 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH 5/5] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 7ee514e101..e0bb5de975 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 0697701bfe..f3837d470f 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
2.17.0
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 2:21 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
The pending comment is providing a way to rewrite a table and
re-compress the data with the current compression method.I spent some time poking at this yesterday and ran couldn't figure out
what was going on here. There are two places where we rewrite tables.
One is the stuff in cluter.c, which handles VACUUM FULL and CLUSTER.
That eventually calls reform_and_rewrite_tuple(), which deforms the
old tuple and creates a new one, but it doesn't seem like there's
anything in there that would expand toasted values, whether external
or inline compressed. But I think that can't be right, because it
seems like then you'd end up with toast pointers into the old TOAST
relation, not the new one, which would cause failures later. So I must
be missing something here. The other place where we rewrite tables is
in ATRewriteTable() as part of the ALTER TABLE machinery. I don't see
anything there to force detoasting either.
We do expand the external values, see below call stack. See below call stack.
#0 detoast_external_attr (attr=0x2d61a80) at detoast.c:50
#1 0x000000000055bd53 in toast_tuple_init
#2 0x000000000050554d in heap_toast_insert_or_update
#3 0x000000000050ad5b in raw_heap_insert
#4 0x000000000050a9a1 in rewrite_heap_tuple
#5 0x0000000000502325 in reform_and_rewrite_tuple
#6 0x00000000004ff47c in heapam_relation_copy_for_cluster
But that is only if there are external attributes, we do nothing for
the inline compressed values. In raw_heap_insert only if
HeapTupleHasExternal(tup) is true then we call raw_heap_insert. So if
we want to do something about inline compressed data then we will have
to do something in reform_and_rewrite_tuple because there we deform
and form the tuple again so we have an opportunity to decompress any
compressed data.
But, I am not really convinced that we need to solve this problem by
adding new ALTER TABLE syntax. I'd be happy enough if CLUSTER, VACUUM
FULL, and versions of ALTER TABLE that already force a rewrite would
cause the compression to be redone also.
Okay, Maybe for directly compressed data we can do that without
affecting the performance of unrelated paths but I am again worried
about the composite type. Basically, if we have some row type then we
have to deform the tuple stored in the row type and process it fully.
Said that I think in the older version we already had a pacthes at
[1]: /messages/by-id/CAFiTN-u=2-qaLTod3isQmXuSU0s0_bR+RcUQL-vSvH=MJbEd7Q@mail.gmail.com
any compressed data in the composite type so we will not have to worry
about those as well.
Honestly, even if the user
had to fall back on creating a new table and doing INSERT INTO newtab
SELECT * FROM oldtab I would consider that to be not a total
showstopper for this .. assuming of course that it actually works. If
it doesn't, we have big problems. Even without the pg_am stuff, we
still need to make sure that we don't just blindly let compressed
values wander around everywhere. When we insert into a table column
with a compression method, we should recompress any data that is
compressed using some other method.
Well, it used to work in the older version[1]/messages/by-id/CAFiTN-u=2-qaLTod3isQmXuSU0s0_bR+RcUQL-vSvH=MJbEd7Q@mail.gmail.com so we can make it work
here as well without much effort.
[1]: /messages/by-id/CAFiTN-u=2-qaLTod3isQmXuSU0s0_bR+RcUQL-vSvH=MJbEd7Q@mail.gmail.com
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 08:17:46AM +0530, Dilip Kumar wrote:
On Thu, Mar 11, 2021 at 2:21 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Mar 10, 2021 at 6:52 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
The pending comment is providing a way to rewrite a table and
re-compress the data with the current compression method.I spent some time poking at this yesterday and ran couldn't figure out
what was going on here. There are two places where we rewrite tables.
One is the stuff in cluter.c, which handles VACUUM FULL and CLUSTER.
That eventually calls reform_and_rewrite_tuple(), which deforms the
old tuple and creates a new one, but it doesn't seem like there's
anything in there that would expand toasted values, whether external
or inline compressed. But I think that can't be right, because it
seems like then you'd end up with toast pointers into the old TOAST
relation, not the new one, which would cause failures later. So I must
be missing something here. The other place where we rewrite tables is
in ATRewriteTable() as part of the ALTER TABLE machinery. I don't see
anything there to force detoasting either.We do expand the external values, see below call stack. See below call stack.
#0 detoast_external_attr (attr=0x2d61a80) at detoast.c:50
#1 0x000000000055bd53 in toast_tuple_init
#2 0x000000000050554d in heap_toast_insert_or_update
#3 0x000000000050ad5b in raw_heap_insert
#4 0x000000000050a9a1 in rewrite_heap_tuple
#5 0x0000000000502325 in reform_and_rewrite_tuple
#6 0x00000000004ff47c in heapam_relation_copy_for_clusterBut that is only if there are external attributes, we do nothing for
the inline compressed values. In raw_heap_insert only if
HeapTupleHasExternal(tup) is true then we call raw_heap_insert. So if
we want to do something about inline compressed data then we will have
to do something in reform_and_rewrite_tuple because there we deform
and form the tuple again so we have an opportunity to decompress any
compressed data.But, I am not really convinced that we need to solve this problem by
adding new ALTER TABLE syntax. I'd be happy enough if CLUSTER, VACUUM
FULL, and versions of ALTER TABLE that already force a rewrite would
cause the compression to be redone also.Okay, Maybe for directly compressed data we can do that without
affecting the performance of unrelated paths but I am again worried
about the composite type. Basically, if we have some row type then we
have to deform the tuple stored in the row type and process it fully.
Said that I think in the older version we already had a pacthes at
[1], basically the first 3 patches will ensure that we will never have
any compressed data in the composite type so we will not have to worry
about those as well.Honestly, even if the user
had to fall back on creating a new table and doing INSERT INTO newtab
SELECT * FROM oldtab I would consider that to be not a total
showstopper for this .. assuming of course that it actually works. If
it doesn't, we have big problems. Even without the pg_am stuff, we
still need to make sure that we don't just blindly let compressed
values wander around everywhere. When we insert into a table column
with a compression method, we should recompress any data that is
compressed using some other method.Well, it used to work in the older version[1] so we can make it work
here as well without much effort.
Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was calling
CompareCompressionMethodAndDecompress().
I think what's wanted here is that decompression should only happen when the
tuple uses a different compression than the column's currently set compression.
So there's no overhead in the usual case. I guess CLUSTER and INSERT SELECT
should do the same.
This is important to allow someone to get rid of LZ4 compression, if they want
to get rid of that dependency.
But it's also really desirable for admins to be able to "migrate" existing
data. People will want to test this, and I guess INSERT SELECT and CLUSTER are
the obvious ways.
--
Justin
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was calling
CompareCompressionMethodAndDecompress().
While changing the compression method user might be just interested
to compress the future tuple with the new compression method but
doesn't want to rewrite all the old tuple. So IMHO without any option
just force
rewrite whenever changing the compression method doesn't look that
great.
I think what's wanted here is that decompression should only happen when the
tuple uses a different compression than the column's currently set compression.
So there's no overhead in the usual case. I guess CLUSTER and INSERT SELECT
should do the same.This is important to allow someone to get rid of LZ4 compression, if they want
to get rid of that dependency.But it's also really desirable for admins to be able to "migrate" existing
data. People will want to test this, and I guess INSERT SELECT and CLUSTER are
the obvious ways.
For INSERT SELECT we were already doing in the older version so we can
include that code here, we will also have to include the patches for
decompressing data before forming the composite types because without
that we can not ensure that lz4 does not exist anywhere in the table.
Said that with that also we can not ensure that it doesn't exist anywhere
in the system because it might exist in the WAL and if you do the crash
recovery then might get those lz4 compressed data back.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 10:07 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
For INSERT SELECT we were already doing in the older version so we can
include that code here, we will also have to include the patches for
decompressing data before forming the composite types because without
that we can not ensure that lz4 does not exist anywhere in the table.
Said that with that also we can not ensure that it doesn't exist anywhere
in the system because it might exist in the WAL and if you do the crash
recovery then might get those lz4 compressed data back.
In updated patches, now INSERT INTO SELECT/VACUUM FULL/ CLUSTER will
re compress the data as per the latest compression method.
create table t(a text compression lz4);
insert into t values(repeat('a', 9000));
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
lz4
(1 row)
postgres[129360]=# alter TABLE t ALTER COLUMN a SET COMPRESSION pglz;
ALTER TABLE
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
lz4
(1 row)
postgres[129360]=# VACUUM FULL t;
VACUUM
postgres[129360]=# select pg_column_compression(a) from t;
pg_column_compression
-----------------------
pglz
(1 row)
IMHO, now we have a way for user to rewrite table using VACUUM
FULL/CLUSTER so I don't think we should force rewrite in ALTER SET
COMPRESSION.
In attached patch I am re compressing in INSERT INTO SELECT as well,
but honestly I think maybe we don't need to force that also on user
and we can only do this in VACUUM FULL or CLUSTER, thoughts?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v34-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v34-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From 4a4609207f24d739cd89d3d3befdfde8a4360b1d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v34 2/7] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/executor/execExprInterp.c | 29 +++++++++++++++++++++++++++--
src/backend/utils/adt/jsonfuncs.c | 8 ++++++--
src/pl/plpgsql/src/pl_exec.c | 5 ++++-
3 files changed, 37 insertions(+), 5 deletions(-)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 20949a5..c3754ac 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ continue;
+
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3098,24 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..c3d464f 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,10 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3769,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..fd07376 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,9 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v34-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchtext/x-patch; charset=US-ASCII; name=v34-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchDownload
From 16a138bd774ddd99924d35d52d1dcc1d4bbaf9b8 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH v34 1/7] Get datum from tuple which doesn't contain external
data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 17 +++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 1 +
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 125 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..43bb2ab 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..c36c283 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,21 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 80d2d20..67385e3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..20949a5 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to flatten
+ * any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index e5f8a06..f973aea 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1420,5 +1420,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..5936c39 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2259,7 +2260,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..079a5af 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..59eb42f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9798,7 +9798,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..8771ccd 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..8ba7ae2 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v34-0005-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v34-0005-Add-default_toast_compression-GUC.patchDownload
From 026b0d29906843ad225b1d08d765c988af94289a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:05:05 +0530
Subject: [PATCH v34 5/7] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++---
src/backend/utils/misc/guc.c | 12 +++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 +++++++++++--
src/test/regress/expected/compression.out | 16 ++++++++++
src/test/regress/expected/compression_1.out | 19 +++++++++++
src/test/regress/sql/compression.sql | 8 +++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 3463b42..e33f926 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -306,3 +309,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index a66d111..362a371 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index af0c0aa..9be0841 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 18e30cf..fdefe87 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 59eb42f..e1dd4cf 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3917,6 +3918,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 38800cb..e9d9ce5 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -15,6 +15,14 @@
#include "postgres.h"
+#include "utils/guc.h"
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/* GUCs */
+extern char *default_toast_compression;
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -36,8 +44,6 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -67,6 +73,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -115,5 +123,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 6981d58..9ddffc8 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -231,6 +231,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 893c18c..03a7f46 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -237,6 +237,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b085c9f..6c9b24f 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -103,6 +103,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v34-0003-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v34-0003-Disallow-compressed-data-inside-container-types.patchDownload
From 6561677a5eabacabbcd0463273720ed17b9e74a8 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 16:33:10 +0530
Subject: [PATCH v34 3/7] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the compressed field but while constructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we only decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
Dilip Kumar based on idea from Robert Haas
---
src/backend/access/common/heaptuple.c | 9 ++--
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 6 +--
src/backend/executor/execTuples.c | 4 --
src/backend/utils/adt/expandedrecord.c | 76 ++++++++++++----------------------
src/backend/utils/adt/jsonfuncs.c | 3 +-
src/include/funcapi.h | 4 +-
src/pl/plpgsql/src/pl_exec.c | 3 +-
8 files changed, 38 insertions(+), 71 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c36c283..eb9f016 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -984,15 +984,12 @@ Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * We have to inline any external/compressed data to meet the conventions
+ * for composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
+ return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
- else
- return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
}
/* ----------------
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index c3754ac..71e6f41 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2845,8 +2845,7 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
- if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
continue;
op->d.row.elemvalues[i] =
@@ -3103,8 +3102,7 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
- if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
continue;
op->d.fieldstore.values[i] = PointerGetDatum(
PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..f115464 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..3cbc256 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index c3d464f..821aa8f 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3388,8 +3388,7 @@ populate_record(TupleDesc tupdesc,
&field,
&nulls[i]);
- if (!nulls[i] && att->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ if (!nulls[i] && att->attlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 8ba7ae2..c869012 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -208,10 +208,10 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Macro declarations/inline functions:
* HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
* HeapTupleHeaderGetDatum but the input tuple should not contain
- * external varlena
+ * external/compressed varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
* HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
- * but the input tuple should not contain external varlena
+ * but the input tuple should not contain external/compressed varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index fd07376..0519253 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -7300,8 +7300,7 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
- if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v34-0004-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v34-0004-Built-in-compression-method.patchDownload
From 07306a9446fb835f739732e56725aa69cac05488 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v34 4/7] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 215 ++++++++++++++++++
configure.ac | 19 ++
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 71 ++++--
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 308 ++++++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 54 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/heap/heapam_handler.c | 22 ++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 +++++++++
src/backend/executor/nodeModifyTable.c | 128 +++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/utils/adt/varlena.c | 41 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 ++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 8 +
src/include/access/toast_compression.h | 119 ++++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/compression.out | 267 ++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 269 ++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 113 ++++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
56 files changed, 1962 insertions(+), 86 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..457ce84 100755
--- a/configure
+++ b/configure
@@ -654,6 +654,8 @@ UUID_LIBS
LDAP_LIBS_BE
LDAP_LIBS_FE
with_ssl
+LZ4_LIBS
+LZ4_CFLAGS
PTHREAD_CFLAGS
PTHREAD_LIBS
PTHREAD_CC
@@ -699,6 +701,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -895,6 +899,8 @@ LDFLAGS_EX
LDFLAGS_SL
PERL
PYTHON
+LZ4_CFLAGS
+LZ4_LIBS
MSGFMT
TCLSH'
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1600,6 +1607,8 @@ Some influential environment variables:
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
PYTHON Python program
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
MSGFMT msgfmt program for NLS
TCLSH Tcl interpreter program (tclsh)
@@ -8564,6 +8573,41 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+#
# Assignments
#
@@ -12054,6 +12098,147 @@ fi
fi
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13322,6 +13507,36 @@ fi
fi
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..2bd24a5 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,15 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+#
# Assignments
#
@@ -1173,6 +1182,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1406,6 +1420,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0..5fdc80f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,16 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..7c8fc77 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..9f05a45 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -16,6 +16,7 @@
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/toast_internals.h"
#include "common/int.h"
#include "common/pg_lzcompress.h"
@@ -457,6 +458,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
* toast_decompress_datum -
*
* Decompress a compressed version of a varlena datum
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..3463b42
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,308 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..69dd949 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..a66d111 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0..99f5f63 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,26 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+ /*
+ * Since we are rewriting the table, use this opportunity to recompress
+ * any compressed attribute with current compression method of the
+ * attribute. Basically, if the compression method of the compressed
+ * varlena is not same as current compression method of the attribute
+ * then decompress it so that if it need to be compressed then it will
+ * be compressed with the current compression method of the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmethod = toast_get_compression_method(new_value);
+
+ if (IsValidCompression(cmethod) &&
+ TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..af0c0aa 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d0ec44b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..933a073 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..18e30cf 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..631ba1f 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,11 @@
#include "postgres.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2036,6 +2038,119 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not same as the target attribute then decompress
+ * the value. If any of the value need to be decompressed then we need to
+ * store that into a new slot.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmethod = toast_get_compression_method(new_value);
+ if (IsValidCompression(cmethod) &&
+ targetTupDesc->attrs[i].attcompression != cmethod)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc,
+ slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /* copy the value/null array to the new slot */
+ memcpy(newslot->tts_values, slot->tts_values,
+ natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull,
+ natts * sizeof(bool));
+
+ }
+
+ /* detoast the value and store into the new slot */
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2359,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3000,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..681151e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..187e55b 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5300,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..cef9bbd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..d956b03 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a43a8f5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..9a428aa 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..38800cb
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,119 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..284a157 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..faa5456 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7098,6 +7098,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..6495162 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e31ad62..e388336 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..488413a 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..e98af05 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..6981d58
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,267 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+DROP TABLE cmdata3;
+DROP TABLE cmdata2;
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..893c18c
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,269 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+ERROR: type "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+ ^
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+ERROR: relation "cmdata2" does not exist
+LINE 1: INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+ ^
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ERROR: relation "cmdata2" does not exist
+LINE 1: SELECT pg_column_compression((x).f1) FROM cmdata2;
+ ^
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+ERROR: type "cmdata2" does not exist
+LINE 1: CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+ ^
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+ERROR: relation "cmdata3" does not exist
+LINE 1: INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata...
+ ^
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ERROR: relation "cmdata2" does not exist
+LINE 1: SELECT pg_column_compression((x).f1) FROM cmdata2;
+ ^
+DROP TABLE cmdata3;
+ERROR: table "cmdata3" does not exist
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..b085c9f
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,113 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+DROP TABLE cmdata3;
+DROP TABLE cmdata2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..daf163c 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..0fc7017 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v34-0006-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v34-0006-Alter-table-set-compression.patchDownload
From 280bcf0688aad0ae9462dd05c5f7569725c3abad Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:34:08 +0530
Subject: [PATCH v34 6/7] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 48 ++++++-
src/test/regress/sql/compression.sql | 20 +++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index fdefe87..e6a0b69 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7773,6 +7781,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15046,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208a..07ba55f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8..1962570 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46..bd64263 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f..00b1ff1 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v34-0007-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v34-0007-default-to-with-lz4.patchDownload
From 3ceca5abc4b81be3a9c91e5675eb3d87e2c97f34 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v34 7/7] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 457ce84..be39718 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 2bd24a5..5037d51 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 support
This includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.
Attachments:
0001-Get-datum-from-tuple-which-doesn-t-contain-external-.patchtext/x-diff; charset=us-asciiDownload
From db1e2cbe5cd008d1e042f409705e43b992eb9b7d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH 1/8] Get datum from tuple which doesn't contain external data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 17 +++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 1 +
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 125 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa88e..5a41116e8e 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff844..cc7a8e0eef 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01cac..d466f6cc46 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df07177ed..5d517c8afc 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2deb66..626294685a 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b572f..f0028e4cc4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbcdb8..77dd559c7e 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f6303df..dbf34f87d4 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119474..957ee5bc14 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb31e..3a050ee88c 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21016..e7aab866b8 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579fcbb..1fd405e5df 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbfa44..7eb80d5b78 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124e62..43bb2ab852 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb892..333f7fd391 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193bb25..147c8d22eb 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb30f0..7d74c316c7 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff8af..70b8bf0855 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0bb98..53801e92cd 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0fa5a..c36c283253 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,21 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66286..015c84179e 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1fa1..b44066a4eb 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6023e7c16f..3f48597c2a 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6dc2..53f84de41c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b690d8..7cd62e70b8 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9ccb..f566e1e0a5 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286d8c..20949a5dec 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to flatten
+ * any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44113..f92628bc59 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index e5f8a06fea..f973aea8e5 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1420,5 +1420,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1ba8..115131c707 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dced9..633c90d08d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e218..5871c0b404 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c55ea..ce11554b3b 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8d76..d3d386b748 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265c12..081844939b 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574d7e..57e0ea09a3 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5db6..6915939f43 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5102227a60..b93ad73241 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1856,7 +1856,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[8] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2272,7 +2273,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6ae7..079a5af5fb 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebcc8f..6679493247 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 855076b1fd..953837085f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9795,7 +9795,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a882..195c127739 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852e7f..7ac3c2382d 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c86e..8771ccdac6 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2a1f..8ba7ae211f 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf71a..cdf79f78e8 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e11837559d..b4e710a22a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd61c..ec83645bbe 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
2.17.0
0002-Expand-the-external-data-before-forming-the-tuple.patchtext/x-diff; charset=us-asciiDownload
From 64d02424b961406600d479a95a0d74fec5a385ce Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH 2/8] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/executor/execExprInterp.c | 29 +++++++++++++++++++++++++--
src/backend/utils/adt/jsonfuncs.c | 8 ++++++--
src/pl/plpgsql/src/pl_exec.c | 5 ++++-
3 files changed, 37 insertions(+), 5 deletions(-)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 20949a5dec..c3754acca4 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ continue;
+
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3098,24 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 511467280f..c3d464f42b 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,10 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3769,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aaa7f..fd073767bc 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,9 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
2.17.0
0003-Disallow-compressed-data-inside-container-types.patchtext/x-diff; charset=us-asciiDownload
From 4489b8e89557c92d932dd944db07c99eea8c088c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 16:33:10 +0530
Subject: [PATCH 3/8] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the compressed field but while constructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we only decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
Dilip Kumar based on idea from Robert Haas
---
src/backend/access/common/heaptuple.c | 9 +--
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 6 +-
src/backend/executor/execTuples.c | 4 --
src/backend/utils/adt/expandedrecord.c | 76 +++++++++-----------------
src/backend/utils/adt/jsonfuncs.c | 3 +-
src/include/funcapi.h | 4 +-
src/pl/plpgsql/src/pl_exec.c | 3 +-
8 files changed, 38 insertions(+), 71 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c36c283253..eb9f016dfa 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -984,15 +984,12 @@ Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * We have to inline any external/compressed data to meet the conventions
+ * for composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
+ return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
- else
- return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
}
/* ----------------
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d584..b09462348b 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index c3754acca4..71e6f41fee 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2845,8 +2845,7 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
- if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
continue;
op->d.row.elemvalues[i] =
@@ -3103,8 +3102,7 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
- if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
continue;
op->d.fieldstore.values[i] = PointerGetDatum(
PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df9c9..f11546468e 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491ecf7..3cbc256671 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index c3d464f42b..821aa8fbdb 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3388,8 +3388,7 @@ populate_record(TupleDesc tupdesc,
&field,
&nulls[i]);
- if (!nulls[i] && att->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ if (!nulls[i] && att->attlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 8ba7ae211f..c869012873 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -208,10 +208,10 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Macro declarations/inline functions:
* HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
* HeapTupleHeaderGetDatum but the input tuple should not contain
- * external varlena
+ * external/compressed varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
* HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
- * but the input tuple should not contain external varlena
+ * but the input tuple should not contain external/compressed varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index fd073767bc..0519253cbe 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -7300,8 +7300,7 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
- if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
2.17.0
0004-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From d0d7896a88f009181f9540eaea8cf2a89a552637 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH 4/8] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 215 ++++++++++++
configure.ac | 19 ++
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 +-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 71 ++--
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 308 ++++++++++++++++++
src/backend/access/common/toast_internals.c | 54 ++-
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/heap/heapam_handler.c | 22 ++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 +++++++
src/backend/executor/nodeModifyTable.c | 128 ++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/utils/adt/varlena.c | 41 +++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 +-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 8 +
src/include/access/toast_compression.h | 119 +++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/compression.out | 267 +++++++++++++++
src/test/regress/expected/compression_1.out | 269 +++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 113 +++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
56 files changed, 1962 insertions(+), 86 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index fad817bb38..87ed16060e 100755
--- a/configure
+++ b/configure
@@ -654,6 +654,8 @@ UUID_LIBS
LDAP_LIBS_BE
LDAP_LIBS_FE
with_ssl
+LZ4_LIBS
+LZ4_CFLAGS
PTHREAD_CFLAGS
PTHREAD_LIBS
PTHREAD_CC
@@ -699,6 +701,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -895,6 +899,8 @@ LDFLAGS_EX
LDFLAGS_SL
PERL
PYTHON
+LZ4_CFLAGS
+LZ4_LIBS
MSGFMT
TCLSH'
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1600,6 +1607,8 @@ Some influential environment variables:
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
PYTHON Python program
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
MSGFMT msgfmt program for NLS
TCLSH Tcl interpreter program (tclsh)
@@ -8563,6 +8572,41 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
#
# Assignments
#
@@ -12110,6 +12154,147 @@ fi
fi
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13376,6 +13561,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 0ed53571dd..c8f199dc5e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,15 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1174,6 +1183,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1407,6 +1421,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0674..5fdc80ff3d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1355,6 +1355,16 @@
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227683..7c8fc77983 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -288,6 +288,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
@@ -605,6 +625,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edfa4d..01ec9b8b0a 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3863,6 +3863,17 @@ bar
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d63f1..b9aff0ccfd 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..9f05a4502b 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -16,6 +16,7 @@
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/toast_internals.h"
#include "common/int.h"
#include "common/pg_lzcompress.h"
@@ -456,6 +457,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1f6b7b77d4 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000000..3463b42438
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,308 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..69dd9492f6 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..a66d1113db 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0c1f..99f5f63ea7 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,26 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+ /*
+ * Since we are rewriting the table, use this opportunity to recompress
+ * any compressed attribute with current compression method of the
+ * attribute. Basically, if the compression method of the compressed
+ * varlena is not same as current compression method of the attribute
+ * then decompress it so that if it need to be compressed then it will
+ * be compressed with the current compression method of the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmethod = toast_get_compression_method(new_value);
+
+ if (IsValidCompression(cmethod) &&
+ TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..af0c0aa77e 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..9586c29ad0 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 693a94107d..7033ba9bed 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1716,6 +1718,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..397d70d226 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..933a0734d1 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8745c1e722..e20dd5f63c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -24,6 +24,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -562,6 +563,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -855,6 +857,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2399,6 +2413,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2433,6 +2463,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2678,6 +2713,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6380,6 +6428,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11899,6 +11959,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17743,3 +17820,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba43e3..631ba1f193 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,11 @@
#include "postgres.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2036,6 +2038,119 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not same as the target attribute then decompress
+ * the value. If any of the value need to be decompressed then we need to
+ * store that into a new slot.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmethod = toast_get_compression_method(new_value);
+ if (IsValidCompression(cmethod) &&
+ targetTupDesc->attrs[i].attcompression != cmethod)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc,
+ slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /* copy the value/null array to the new slot */
+ memcpy(newslot->tts_values, slot->tts_values,
+ natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull,
+ natts * sizeof(bool));
+
+ }
+
+ /* detoast the value and store into the new slot */
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2359,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3000,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index da91cbd2b1..0d3b923a38 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2988,6 +2988,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6493a03ff8..75343b8aaf 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2876,6 +2876,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 2ae41f8f4d..f8bc11d28b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3429,11 +3431,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,8 +3445,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3488,6 +3491,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3718,6 +3729,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15304,6 +15316,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15824,6 +15837,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266caeb4..681151e32c 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..187e55bd4e 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5299,6 +5300,46 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 99ace71a2a..f54f285f38 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -161,6 +161,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTableam;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3fd7f48605..be5d369efa 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1048,6 +1049,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8626,6 +8628,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8711,6 +8714,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8756,6 +8768,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8784,6 +8797,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15900,6 +15914,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213fb06..d956b035f3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e46464a..bc91bb12ac 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..b284113d55 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index daa5081eac..99a59470c5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -372,6 +372,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d65990059d..83f2e6f254 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..110906a4e9 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1159,6 +1159,13 @@ show_context_hook(const char *newval)
return true;
}
+static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..9a428aae2d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000000..38800cb97a
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,119 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..05104ce237 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..284a15761a 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..560f8f00bb 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7619f8cd3..91c1c5b720 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7099,6 +7099,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363d54..6495162a33 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e31ad6204e..e388336e02 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ce54f76610..56e24529f1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 7a7cc21d8d..6007d72a73 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..e98af05b30 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..6981d580dc
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,267 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+DROP TABLE cmdata3;
+DROP TABLE cmdata2;
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..893c18c7fa
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,269 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+ERROR: type "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+ ^
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+ERROR: relation "cmdata2" does not exist
+LINE 1: INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+ ^
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ERROR: relation "cmdata2" does not exist
+LINE 1: SELECT pg_column_compression((x).f1) FROM cmdata2;
+ ^
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+ERROR: type "cmdata2" does not exist
+LINE 1: CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+ ^
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+ERROR: relation "cmdata3" does not exist
+LINE 1: INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata...
+ ^
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ERROR: relation "cmdata2" does not exist
+LINE 1: SELECT pg_column_compression((x).f1) FROM cmdata2;
+ ^
+DROP TABLE cmdata3;
+ERROR: table "cmdata3" does not exist
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e280198b17..70c38309d7 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -115,7 +115,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941c24..1524676f3b 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 6a57e889a1..d81d04136c 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -201,6 +201,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..b085c9fbd9
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,113 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+DROP TABLE cmdata3;
+DROP TABLE cmdata2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index a4f5cc4bdb..5f39a92111 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 574a8a94fa..e6cb35a16f 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
0005-f-2nd-attempt-to-use-pkgconfig-to-allow-compiling-on.patchtext/x-diff; charset=us-asciiDownload
From ef577dfd2bcae3594d7239d4714f1dad8c8e9d79 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Thu, 11 Mar 2021 09:18:46 -0600
Subject: [PATCH 5/8] f!2nd attempt to use pkgconfig to allow compiling on OSX
---
configure | 279 +++++++++++++++++++--------------------------------
configure.ac | 16 ++-
2 files changed, 108 insertions(+), 187 deletions(-)
diff --git a/configure b/configure
index 87ed16060e..387c553d5a 100755
--- a/configure
+++ b/configure
@@ -654,8 +654,6 @@ UUID_LIBS
LDAP_LIBS_BE
LDAP_LIBS_FE
with_ssl
-LZ4_LIBS
-LZ4_CFLAGS
PTHREAD_CFLAGS
PTHREAD_LIBS
PTHREAD_CC
@@ -701,6 +699,8 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
with_lz4
with_zlib
with_system_tzdata
@@ -895,12 +895,12 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
PYTHON
-LZ4_CFLAGS
-LZ4_LIBS
MSGFMT
TCLSH'
@@ -1603,12 +1603,12 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
PYTHON Python program
- LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
- LZ4_LIBS linker flags for LZ4, overriding pkg-config
MSGFMT msgfmt program for NLS
TCLSH Tcl interpreter program (tclsh)
@@ -8607,6 +8607,102 @@ fi
$as_echo "$with_lz4" >&6; }
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
#
# Assignments
#
@@ -12154,147 +12250,6 @@ fi
fi
-if test "$with_lz4" = yes; then
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
-$as_echo_n "checking for liblz4... " >&6; }
-
-if test -n "$LZ4_CFLAGS"; then
- pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
- if test -n "$PKG_CONFIG" && \
- { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
- ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
- test "x$?" != "x0" && pkg_failed=yes
-else
- pkg_failed=yes
-fi
- else
- pkg_failed=untried
-fi
-if test -n "$LZ4_LIBS"; then
- pkg_cv_LZ4_LIBS="$LZ4_LIBS"
- elif test -n "$PKG_CONFIG"; then
- if test -n "$PKG_CONFIG" && \
- { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
- ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
- test "x$?" != "x0" && pkg_failed=yes
-else
- pkg_failed=yes
-fi
- else
- pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
- _pkg_short_errors_supported=yes
-else
- _pkg_short_errors_supported=no
-fi
- if test $_pkg_short_errors_supported = yes; then
- LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
- else
- LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
- fi
- # Put the nasty error message in config.log where it belongs
- echo "$LZ4_PKG_ERRORS" >&5
-
- as_fn_error $? "Package requirements (liblz4) were not met:
-
-$LZ4_PKG_ERRORS
-
-Consider adjusting the PKG_CONFIG_PATH environment variable if you
-installed software in a non-standard prefix.
-
-Alternatively, you may set the environment variables LZ4_CFLAGS
-and LZ4_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details." "$LINENO" 5
-elif test $pkg_failed = untried; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
- { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
-$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
-as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
-is in your PATH or set the PKG_CONFIG environment variable to the full
-path to pkg-config.
-
-Alternatively, you may set the environment variables LZ4_CFLAGS
-and LZ4_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details.
-
-To get pkg-config, see <http://pkg-config.freedesktop.org/>.
-See \`config.log' for more details" "$LINENO" 5; }
-else
- LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
- LZ4_LIBS=$pkg_cv_LZ4_LIBS
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-
-fi
- { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
-$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
-if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- ac_check_lib_save_LIBS=$LIBS
-LIBS="-llz4 $LIBS"
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-
-/* Override any GCC internal prototype to avoid an error.
- Use char because int might match the return type of a GCC
- builtin and then its argument prototype would still apply. */
-#ifdef __cplusplus
-extern "C"
-#endif
-char LZ4_compress ();
-int
-main ()
-{
-return LZ4_compress ();
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_link "$LINENO"; then :
- ac_cv_lib_lz4_LZ4_compress=yes
-else
- ac_cv_lib_lz4_LZ4_compress=no
-fi
-rm -f core conftest.err conftest.$ac_objext \
- conftest$ac_exeext conftest.$ac_ext
-LIBS=$ac_check_lib_save_LIBS
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
-$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
-if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE_LIBLZ4 1
-_ACEOF
-
- LIBS="-llz4 $LIBS"
-
-else
- as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
-fi
-
-fi
-
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13561,36 +13516,6 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
-fi
-
-if test "$with_lz4" = yes; then
- for ac_header in lz4/lz4.h
-do :
- ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
-if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE_LZ4_LZ4_H 1
-_ACEOF
-
-else
- for ac_header in lz4.h
-do :
- ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
-if test "x$ac_cv_header_lz4_h" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE_LZ4_H 1
-_ACEOF
-
-else
- as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
-fi
-
-done
-
-fi
-
-done
-
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index c8f199dc5e..4cd683afb3 100644
--- a/configure.ac
+++ b/configure.ac
@@ -995,6 +995,12 @@ PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
#
# Assignments
#
@@ -1183,11 +1189,6 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
-if test "$with_lz4" = yes; then
- PKG_CHECK_MODULES(LZ4, liblz4)
- AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
-fi
-
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1421,11 +1422,6 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
-if test "$with_lz4" = yes; then
- AC_CHECK_HEADERS(lz4/lz4.h, [],
- [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
-fi
-
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
--
2.17.0
0006-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From 685d3e6178ce1443220940052cac9f5d8a08a504 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:05:05 +0530
Subject: [PATCH 6/8] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++--
src/backend/utils/misc/guc.c | 12 +++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 ++++++++-
src/test/regress/expected/compression.out | 16 +++++++
src/test/regress/expected/compression_1.out | 19 ++++++++
src/test/regress/sql/compression.sql | 8 ++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 3463b42438..e33f92687c 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -306,3 +309,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index a66d1113db..362a371a6c 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index af0c0aa77e..9be0841d08 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e20dd5f63c..a0d2af0cde 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11971,7 +11971,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17847,9 +17847,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 953837085f..e59ed5b214 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index f46c2dd7a8..7d7a433dcc 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -659,6 +659,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 38800cb97a..e9d9ce5634 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -15,6 +15,14 @@
#include "postgres.h"
+#include "utils/guc.h"
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/* GUCs */
+extern char *default_toast_compression;
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -36,8 +44,6 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -67,6 +73,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -115,5 +123,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 6981d580dc..9ddffc8b18 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -231,6 +231,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 893c18c7fa..03a7f46554 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -237,6 +237,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b085c9fbd9..6c9b24f1f7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -103,6 +103,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0007-Alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From 99876667a05a13d751c3e5fb8f4a7cb3b81a986c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:34:08 +0530
Subject: [PATCH 7/8] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0008-default-to-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From a1cc5febe7065f777b66470162cea5eb5899b7cc Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH 8/8] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 387c553d5a..66b9edcc7b 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 4cd683afb3..7907aded9a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
2.17.0
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 10:07:30AM +0530, Dilip Kumar wrote:
On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was calling
CompareCompressionMethodAndDecompress().While changing the compression method user might be just interested
to compress the future tuple with the new compression method but
doesn't want to rewrite all the old tuple. So IMHO without any option
just force rewrite whenever changing the compression method doesn't look that
great.
I mean to keep the current behavior where SET is only a catalog change.
But I'm comparing with earlier implementation.
Does your new patch avoid recompressing things if the compression is unchanged?
I think what's wanted here is that decompression should only happen when the
tuple uses a different compression than the column's currently set compression.
So there's no overhead in the usual case. I guess CLUSTER and INSERT SELECT
should do the same.This is important to allow someone to get rid of LZ4 compression, if they want
to get rid of that dependency.But it's also really desirable for admins to be able to "migrate" existing
data. People will want to test this, and I guess INSERT SELECT and CLUSTER are
the obvious ways.For INSERT SELECT we were already doing in the older version so we can
include that code here, we will also have to include the patches for
decompressing data before forming the composite types because without
that we can not ensure that lz4 does not exist anywhere in the table.
Said that with that also we can not ensure that it doesn't exist anywhere
in the system because it might exist in the WAL and if you do the crash
recovery then might get those lz4 compressed data back.
I think this is no concern except for PITR, in which case it's working as
intended (if someone does a partial replay to an intermediate state where
tables were still LZ4). If they replay to a point following the SET pglz +
CLUSTER, then LZ4 doesn't exist in the heap, and anything in the WAL is pretty
uninteresting.
My patch this morning compiled ok on mac, so you should include it.
|0005-f-2nd-attempt-to-use-pkgconfig-to-allow-compiling-on.patch
I mailed to Thomas offlist about getting pkg-config installed on the BSD CI
environment.
--
Justin
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 11:55 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.
Thanks for your help. I did not understand the reason for removal of
lz4.h header check?
-if test "$with_lz4" = yes; then
- AC_CHECK_HEADERS(lz4/lz4.h, [],
- [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file
is required for LZ4])])])
-fi
-
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 01, 2021 at 08:53:09PM +0530, Dilip Kumar wrote:
On Mon, Mar 1, 2021 at 5:36 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Mon, Mar 1, 2021 at 11:06 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks. It seems like that explains it.
I think if that's a problem with recent versions, then you'll have to
conditionally disable slicing.
https://packages.debian.org/liblz4-devSlicing isn't generally usable if it sometimes makes people's data inaccessible
and gives errors about corruption.I guess you could make it a compile time test on these constants (I don't know
the necessary version, though)#define LZ4_VERSION_MAJOR 1 /* for breaking interface changes */
#define LZ4_VERSION_MINOR 7 /* for new (non-breaking) interface capabilities */
#define LZ4_VERSION_RELEASE 1 /* for tweaks, bug-fixes, or development */
#define LZ4_VERSION_NUMBER (LZ4_VERSION_MAJOR *100*100 + LZ4_VERSION_MINOR *100 + LZ4_VERSION_RELEASE)If the version is too low, either make it #error, or disable slicing.
The OS usual library version infrastructure will make sure the runtime version
is at least the MAJOR+MINOR of the compile time version.I think we can check the version and if it too low i.e. below1.8.3 (
in this release the slicing issue was fixed) then we can call the full
decompression routine from the slicing function.
Thank you
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else It occurred to me that this should actually compare the runtime version with
LZ4_versionNumber(). That way, a library upgrade can enable the slice
behavior.
--
Justin
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 08:38:41AM +0530, Dilip Kumar wrote:
On Thu, Mar 11, 2021 at 11:55 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.Thanks for your help. I did not understand the reason for removal of
lz4.h header check?
It can stay. I tried to base this on ICU and LIBXML, but I see now they both
have that.
-if test "$with_lz4" = yes; then
- AC_CHECK_HEADERS(lz4/lz4.h, [],
- [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file
is required for LZ4])])])
-fi
-
--
Justin
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 8:54 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
+#elif LZ4_VERSION_NUMBER < 10803 + return lz4_cmdecompress(value); +#elseIt occurred to me that this should actually compare the runtime version with
LZ4_versionNumber(). That way, a library upgrade can enable the slice
behavior.
Yeah, that makes sense, I will change in the next version.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 9:03 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Fri, Mar 12, 2021 at 08:38:41AM +0530, Dilip Kumar wrote:
On Thu, Mar 11, 2021 at 11:55 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.Thanks for your help. I did not understand the reason for removal of
lz4.h header check?It can stay. I tried to base this on ICU and LIBXML, but I see now they both
have that.
Ok
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 2:54 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Thu, Mar 11, 2021 at 10:07:30AM +0530, Dilip Kumar wrote:
On Thu, Mar 11, 2021 at 8:50 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Looking at v23-0002-alter-table-set-compression, ATRewriteTable() was calling
CompareCompressionMethodAndDecompress().While changing the compression method user might be just interested
to compress the future tuple with the new compression method but
doesn't want to rewrite all the old tuple. So IMHO without any option
just force rewrite whenever changing the compression method doesn't look that
great.I mean to keep the current behavior where SET is only a catalog change.
But I'm comparing with earlier implementation.Does your new patch avoid recompressing things if the compression is unchanged?
Currently, my patch is not at all re-compressing on ALTER SET
COMPRESSION METHOD. Yesterday I had an offlist discussion with Robert
and the idea is that whenever we are rewriting the table that time we
can use the opportunity to compare the compression method of the
compressed data with the current compression method of the attribute,
and if they are not the same then decompress so that they can be
compressed back as per the current compression method if required. So
that will be true for VACUUM FULL, CLUSTER, INSERT INTO SELECT,
MATVIEW, CTAS. I think for alter table also if we are rewriting a
table for some reason then we can use the opportunity to re-compress,
but we are not planning to force a rewrite for ALTER SET COMPRESSION,
even if we are changing the compression method. I will make all these
changes in the next version of the patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.
The 2nd attempt passed ./configure on mac (and BSD after Thomas installed
pkg-config), but I eventually realized that LZ4 was effectively disabled,
because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4.
--
Justin
Attachments:
0001-Get-datum-from-tuple-which-doesn-t-contain-external-.patchtext/x-diff; charset=us-asciiDownload
From db1e2cbe5cd008d1e042f409705e43b992eb9b7d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH 1/8] Get datum from tuple which doesn't contain external data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 17 +++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 1 +
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 125 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa88e..5a41116e8e 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff844..cc7a8e0eef 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01cac..d466f6cc46 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df07177ed..5d517c8afc 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2deb66..626294685a 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b572f..f0028e4cc4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbcdb8..77dd559c7e 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f6303df..dbf34f87d4 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119474..957ee5bc14 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb31e..3a050ee88c 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21016..e7aab866b8 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579fcbb..1fd405e5df 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbfa44..7eb80d5b78 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124e62..43bb2ab852 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb892..333f7fd391 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193bb25..147c8d22eb 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb30f0..7d74c316c7 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff8af..70b8bf0855 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0bb98..53801e92cd 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0fa5a..c36c283253 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,21 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66286..015c84179e 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1fa1..b44066a4eb 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6023e7c16f..3f48597c2a 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6dc2..53f84de41c 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b690d8..7cd62e70b8 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9ccb..f566e1e0a5 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286d8c..20949a5dec 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to flatten
+ * any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44113..f92628bc59 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index e5f8a06fea..f973aea8e5 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1420,5 +1420,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1ba8..115131c707 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dced9..633c90d08d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e218..5871c0b404 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c55ea..ce11554b3b 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8d76..d3d386b748 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265c12..081844939b 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574d7e..57e0ea09a3 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5db6..6915939f43 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5102227a60..b93ad73241 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1856,7 +1856,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[8] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2272,7 +2273,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6ae7..079a5af5fb 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebcc8f..6679493247 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 855076b1fd..953837085f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9795,7 +9795,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a882..195c127739 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852e7f..7ac3c2382d 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c86e..8771ccdac6 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2a1f..8ba7ae211f 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf71a..cdf79f78e8 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e11837559d..b4e710a22a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd61c..ec83645bbe 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
2.17.0
0002-Expand-the-external-data-before-forming-the-tuple.patchtext/x-diff; charset=us-asciiDownload
From 64d02424b961406600d479a95a0d74fec5a385ce Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH 2/8] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/executor/execExprInterp.c | 29 +++++++++++++++++++++++++--
src/backend/utils/adt/jsonfuncs.c | 8 ++++++--
src/pl/plpgsql/src/pl_exec.c | 5 ++++-
3 files changed, 37 insertions(+), 5 deletions(-)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 20949a5dec..c3754acca4 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ continue;
+
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3098,24 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* Retrieve externally stored values and decompress. */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ continue;
+ op->d.fieldstore.values[i] = PointerGetDatum(
+ PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 511467280f..c3d464f42b 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,10 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3769,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aaa7f..fd073767bc 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -5326,7 +5326,7 @@ exec_eval_datum(PLpgSQL_execstate *estate,
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7300,6 +7300,9 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
2.17.0
0003-Disallow-compressed-data-inside-container-types.patchtext/x-diff; charset=us-asciiDownload
From 4489b8e89557c92d932dd944db07c99eea8c088c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 16:33:10 +0530
Subject: [PATCH 3/8] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the compressed field but while constructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we only decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
Dilip Kumar based on idea from Robert Haas
---
src/backend/access/common/heaptuple.c | 9 +--
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 6 +-
src/backend/executor/execTuples.c | 4 --
src/backend/utils/adt/expandedrecord.c | 76 +++++++++-----------------
src/backend/utils/adt/jsonfuncs.c | 3 +-
src/include/funcapi.h | 4 +-
src/pl/plpgsql/src/pl_exec.c | 3 +-
8 files changed, 38 insertions(+), 71 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c36c283253..eb9f016dfa 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -984,15 +984,12 @@ Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * We have to inline any external/compressed data to meet the conventions
+ * for composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
+ return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
- else
- return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
}
/* ----------------
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d584..b09462348b 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index c3754acca4..71e6f41fee 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2845,8 +2845,7 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
- if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
continue;
op->d.row.elemvalues[i] =
@@ -3103,8 +3102,7 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
- if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
continue;
op->d.fieldstore.values[i] = PointerGetDatum(
PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df9c9..f11546468e 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491ecf7..3cbc256671 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work
+ * of detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the
+ * flag about them. This won't cause ER_flatten_into() to mistakenly
+ * take the fast path, since expanded_record_set_field() will have
+ * cleared ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1155,11 +1132,12 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index c3d464f42b..821aa8fbdb 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3388,8 +3388,7 @@ populate_record(TupleDesc tupdesc,
&field,
&nulls[i]);
- if (!nulls[i] && att->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ if (!nulls[i] && att->attlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 8ba7ae211f..c869012873 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -208,10 +208,10 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Macro declarations/inline functions:
* HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
* HeapTupleHeaderGetDatum but the input tuple should not contain
- * external varlena
+ * external/compressed varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
* HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
- * but the input tuple should not contain external varlena
+ * but the input tuple should not contain external/compressed varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index fd073767bc..0519253cbe 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -7300,8 +7300,7 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
- if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ if (!nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
2.17.0
0004-Built-in-compression-method.patchtext/x-diff; charset=us-asciiDownload
From d0d7896a88f009181f9540eaea8cf2a89a552637 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH 4/8] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 215 ++++++++++++
configure.ac | 19 ++
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 +-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 71 ++--
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 308 ++++++++++++++++++
src/backend/access/common/toast_internals.c | 54 ++-
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/heap/heapam_handler.c | 22 ++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 +++++++
src/backend/executor/nodeModifyTable.c | 128 ++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/utils/adt/varlena.c | 41 +++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 +-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 8 +
src/include/access/toast_compression.h | 119 +++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 20 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/compression.out | 267 +++++++++++++++
src/test/regress/expected/compression_1.out | 269 +++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 113 +++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
56 files changed, 1962 insertions(+), 86 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index fad817bb38..87ed16060e 100755
--- a/configure
+++ b/configure
@@ -654,6 +654,8 @@ UUID_LIBS
LDAP_LIBS_BE
LDAP_LIBS_FE
with_ssl
+LZ4_LIBS
+LZ4_CFLAGS
PTHREAD_CFLAGS
PTHREAD_LIBS
PTHREAD_CC
@@ -699,6 +701,7 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -895,6 +899,8 @@ LDFLAGS_EX
LDFLAGS_SL
PERL
PYTHON
+LZ4_CFLAGS
+LZ4_LIBS
MSGFMT
TCLSH'
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1600,6 +1607,8 @@ Some influential environment variables:
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
PYTHON Python program
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
MSGFMT msgfmt program for NLS
TCLSH Tcl interpreter program (tclsh)
@@ -8563,6 +8572,41 @@ fi
+#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
#
# Assignments
#
@@ -12110,6 +12154,147 @@ fi
fi
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
+$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_compress ();
+int
+main ()
+{
+return LZ4_compress ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_compress=yes
+else
+ ac_cv_lib_lz4_LZ4_compress=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
@@ -13376,6 +13561,36 @@ Use --without-zlib to disable zlib support." "$LINENO" 5
fi
+fi
+
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index 0ed53571dd..c8f199dc5e 100644
--- a/configure.ac
+++ b/configure.ac
@@ -986,6 +986,15 @@ PGAC_ARG_BOOL(with, zlib, yes,
[do not use Zlib])
AC_SUBST(with_zlib)
+#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
#
# Assignments
#
@@ -1174,6 +1183,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1407,6 +1421,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0674..5fdc80ff3d 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1355,6 +1355,16 @@
</para></entry>
</row>
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
<row>
<entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227683..7c8fc77983 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -288,6 +288,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
@@ -605,6 +625,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edfa4d..01ec9b8b0a 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3863,6 +3863,17 @@ bar
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9473..0ab5712c71 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d63f1..b9aff0ccfd 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf648..9f05a4502b 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -16,6 +16,7 @@
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/toast_internals.h"
#include "common/int.h"
#include "common/pg_lzcompress.h"
@@ -456,6 +457,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
return result;
}
+/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
/* ----------
* toast_decompress_datum -
*
@@ -464,21 +496,18 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +521,18 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in the
+ * toast header.
+ */
+ cmroutine = GetCompressionRoutines(
+ CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr)));
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138497..1f6b7b77d4 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000000..3463b42438
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,308 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef HAVE_LIBLZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#elif LZ4_VERSION_NUMBER < 10803
+ return lz4_cmdecompress(value);
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef HAVE_LIBLZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine*
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f41b..69dd9492f6 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f59440c..a66d1113db 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0c1f..99f5f63ea7 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,26 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+ /*
+ * Since we are rewriting the table, use this opportunity to recompress
+ * any compressed attribute with current compression method of the
+ * attribute. Basically, if the compression method of the compressed
+ * varlena is not same as current compression method of the attribute
+ * then decompress it so that if it need to be compressed then it will
+ * be compressed with the current compression method of the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmethod = toast_get_compression_method(new_value);
+
+ if (IsValidCompression(cmethod) &&
+ TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151ce5..53f78f9c3e 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6622..af0c0aa77e 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958112..9586c29ad0 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 693a94107d..7033ba9bed 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1716,6 +1718,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5efd..397d70d226 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b806020d..933a0734d1 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8745c1e722..e20dd5f63c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -24,6 +24,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -562,6 +563,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -855,6 +857,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2399,6 +2413,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression = GetCompressionMethodName(
+ attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2433,6 +2463,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2678,6 +2713,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6380,6 +6428,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11899,6 +11959,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17743,3 +17820,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba43e3..631ba1f193 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,11 @@
#include "postgres.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2036,6 +2038,119 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not same as the target attribute then decompress
+ * the value. If any of the value need to be decompressed then we need to
+ * store that into a new slot.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmethod = toast_get_compression_method(new_value);
+ if (IsValidCompression(cmethod) &&
+ targetTupDesc->attrs[i].attcompression != cmethod)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc,
+ slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /* copy the value/null array to the new slot */
+ memcpy(newslot->tts_values, slot->tts_values,
+ natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull,
+ natts * sizeof(bool));
+
+ }
+
+ /* detoast the value and store into the new slot */
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2359,15 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * Compare the compression method of the compressed attribute in the
+ * source tuple with target attribute and if those are different then
+ * decompress those attributes.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3000,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index da91cbd2b1..0d3b923a38 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2988,6 +2988,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d73626fc..f3592003da 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac5c2..38226530c6 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6493a03ff8..75343b8aaf 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2876,6 +2876,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 2ae41f8f4d..f8bc11d28b 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3429,11 +3431,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3442,8 +3445,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3488,6 +3491,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3718,6 +3729,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15304,6 +15316,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15824,6 +15837,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266caeb4..681151e32c 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9ae54..187e55bd4e 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5299,6 +5300,46 @@ pg_column_size(PG_FUNCTION_ARGS)
PG_RETURN_INT32(result);
}
+/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
/*
* string_agg - Concatenates values and returns string.
*
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 99ace71a2a..f54f285f38 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -161,6 +161,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTableam;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 3fd7f48605..be5d369efa 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1048,6 +1049,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8626,6 +8628,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8711,6 +8714,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8756,6 +8768,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8784,6 +8797,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15900,6 +15914,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213fb06..d956b035f3 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e46464a..bc91bb12ac 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a92b4..b284113d55 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index daa5081eac..99a59470c5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -372,6 +372,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d65990059d..83f2e6f254 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c8d7..110906a4e9 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1159,6 +1159,13 @@ show_context_hook(const char *newval)
return true;
}
+static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
static bool
hide_tableam_hook(const char *newval)
{
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c77b..9a428aae2d 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000000..38800cb97a
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,119 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define PGLZ_COMPRESSION 'p'
+#define LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+} CompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline CompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case PGLZ_COMPRESSION:
+ return PGLZ_COMPRESSION_ID;
+ case LZ4_COMPRESSION:
+ return LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop CompressionId in toast_compression.h
+ */
+static inline Oid
+CompressionIdToMethod(CompressionId cmid)
+{
+ switch (cmid)
+ {
+ case PGLZ_COMPRESSION_ID:
+ return PGLZ_COMPRESSION;
+ case LZ4_COMPRESSION_ID:
+ return LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char*
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d644bc..05104ce237 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb890d8..284a15761a 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,23 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) >= PGLZ_COMPRESSION_ID && (cm_method) <= LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42abf08..560f8f00bb 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index c7619f8cd3..91c1c5b720 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7099,6 +7099,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363d54..6495162a33 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e31ad6204e..e388336e02 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index ce54f76610..56e24529f1 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aaac9..ca1f950cbe 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 7a7cc21d8d..6007d72a73 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,6 +346,9 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed572004d..e98af05b30 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000000..6981d580dc
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,267 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+DROP TABLE cmdata3;
+DROP TABLE cmdata2;
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000000..893c18c7fa
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,269 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+ERROR: type "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+ ^
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+ERROR: relation "cmdata2" does not exist
+LINE 1: INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+ ^
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ERROR: relation "cmdata2" does not exist
+LINE 1: SELECT pg_column_compression((x).f1) FROM cmdata2;
+ ^
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+ERROR: type "cmdata2" does not exist
+LINE 1: CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+ ^
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+ERROR: relation "cmdata3" does not exist
+LINE 1: INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata...
+ ^
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+ERROR: relation "cmdata2" does not exist
+LINE 1: SELECT pg_column_compression((x).f1) FROM cmdata2;
+ ^
+DROP TABLE cmdata3;
+ERROR: table "cmdata3" does not exist
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e280198b17..70c38309d7 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -115,7 +115,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941c24..1524676f3b 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 6a57e889a1..d81d04136c 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -201,6 +201,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000000..b085c9fbd9
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,113 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+--test composite type
+CREATE TABLE cmdata2 (x cmdata1 compression pglz);
+INSERT INTO cmdata2 SELECT cmdata1 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+
+CREATE TABLE cmdata3 (x cmdata2 compression pglz);
+INSERT INTO cmdata3 SELECT row(cmdata1)::cmdata2 FROM cmdata1;
+SELECT pg_column_compression((x).f1) FROM cmdata2;
+DROP TABLE cmdata3;
+DROP TABLE cmdata2;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index a4f5cc4bdb..5f39a92111 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,6 +307,7 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 574a8a94fa..e6cb35a16f 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
2.17.0
0005-f-3nd-attempt-to-use-pkgconfig-to-allow-compiling-on.patchtext/x-diff; charset=us-asciiDownload
From 749cbdbed1c0a9cf80ca2010e5bee7cc29bcfba0 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Thu, 11 Mar 2021 09:18:46 -0600
Subject: [PATCH 5/8] f!3nd attempt to use pkgconfig to allow compiling on OSX
---
configure | 249 +++++++-----------
configure.ac | 13 +-
src/backend/access/common/toast_compression.c | 10 +-
src/include/pg_config.h.in | 6 +-
src/tools/msvc/Solution.pm | 2 +-
5 files changed, 118 insertions(+), 162 deletions(-)
diff --git a/configure b/configure
index 87ed16060e..440d1e8ce5 100755
--- a/configure
+++ b/configure
@@ -654,8 +654,6 @@ UUID_LIBS
LDAP_LIBS_BE
LDAP_LIBS_FE
with_ssl
-LZ4_LIBS
-LZ4_CFLAGS
PTHREAD_CFLAGS
PTHREAD_LIBS
PTHREAD_CC
@@ -701,6 +699,8 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
with_lz4
with_zlib
with_system_tzdata
@@ -895,12 +895,12 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
PYTHON
-LZ4_CFLAGS
-LZ4_LIBS
MSGFMT
TCLSH'
@@ -1603,12 +1603,12 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
PYTHON Python program
- LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
- LZ4_LIBS linker flags for LZ4, overriding pkg-config
MSGFMT msgfmt program for NLS
TCLSH Tcl interpreter program (tclsh)
@@ -8607,6 +8607,102 @@ fi
$as_echo "$with_lz4" >&6; }
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
#
# Assignments
#
@@ -12154,147 +12250,6 @@ fi
fi
-if test "$with_lz4" = yes; then
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
-$as_echo_n "checking for liblz4... " >&6; }
-
-if test -n "$LZ4_CFLAGS"; then
- pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
- if test -n "$PKG_CONFIG" && \
- { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
- ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
- test "x$?" != "x0" && pkg_failed=yes
-else
- pkg_failed=yes
-fi
- else
- pkg_failed=untried
-fi
-if test -n "$LZ4_LIBS"; then
- pkg_cv_LZ4_LIBS="$LZ4_LIBS"
- elif test -n "$PKG_CONFIG"; then
- if test -n "$PKG_CONFIG" && \
- { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
- ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
- test "x$?" != "x0" && pkg_failed=yes
-else
- pkg_failed=yes
-fi
- else
- pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
- _pkg_short_errors_supported=yes
-else
- _pkg_short_errors_supported=no
-fi
- if test $_pkg_short_errors_supported = yes; then
- LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
- else
- LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
- fi
- # Put the nasty error message in config.log where it belongs
- echo "$LZ4_PKG_ERRORS" >&5
-
- as_fn_error $? "Package requirements (liblz4) were not met:
-
-$LZ4_PKG_ERRORS
-
-Consider adjusting the PKG_CONFIG_PATH environment variable if you
-installed software in a non-standard prefix.
-
-Alternatively, you may set the environment variables LZ4_CFLAGS
-and LZ4_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details." "$LINENO" 5
-elif test $pkg_failed = untried; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
- { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
-$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
-as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
-is in your PATH or set the PKG_CONFIG environment variable to the full
-path to pkg-config.
-
-Alternatively, you may set the environment variables LZ4_CFLAGS
-and LZ4_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details.
-
-To get pkg-config, see <http://pkg-config.freedesktop.org/>.
-See \`config.log' for more details" "$LINENO" 5; }
-else
- LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
- LZ4_LIBS=$pkg_cv_LZ4_LIBS
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-
-fi
- { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_compress in -llz4" >&5
-$as_echo_n "checking for LZ4_compress in -llz4... " >&6; }
-if ${ac_cv_lib_lz4_LZ4_compress+:} false; then :
- $as_echo_n "(cached) " >&6
-else
- ac_check_lib_save_LIBS=$LIBS
-LIBS="-llz4 $LIBS"
-cat confdefs.h - <<_ACEOF >conftest.$ac_ext
-/* end confdefs.h. */
-
-/* Override any GCC internal prototype to avoid an error.
- Use char because int might match the return type of a GCC
- builtin and then its argument prototype would still apply. */
-#ifdef __cplusplus
-extern "C"
-#endif
-char LZ4_compress ();
-int
-main ()
-{
-return LZ4_compress ();
- ;
- return 0;
-}
-_ACEOF
-if ac_fn_c_try_link "$LINENO"; then :
- ac_cv_lib_lz4_LZ4_compress=yes
-else
- ac_cv_lib_lz4_LZ4_compress=no
-fi
-rm -f core conftest.err conftest.$ac_objext \
- conftest$ac_exeext conftest.$ac_ext
-LIBS=$ac_check_lib_save_LIBS
-fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_compress" >&5
-$as_echo "$ac_cv_lib_lz4_LZ4_compress" >&6; }
-if test "x$ac_cv_lib_lz4_LZ4_compress" = xyes; then :
- cat >>confdefs.h <<_ACEOF
-#define HAVE_LIBLZ4 1
-_ACEOF
-
- LIBS="-llz4 $LIBS"
-
-else
- as_fn_error $? "library 'lz4' is required for LZ4 support" "$LINENO" 5
-fi
-
-fi
-
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/configure.ac b/configure.ac
index c8f199dc5e..780791ae8a 100644
--- a/configure.ac
+++ b/configure.ac
@@ -995,6 +995,12 @@ PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
#
# Assignments
#
@@ -1183,11 +1189,6 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
-if test "$with_lz4" = yes; then
- PKG_CHECK_MODULES(LZ4, liblz4)
- AC_CHECK_LIB(lz4, LZ4_compress, [], [AC_MSG_ERROR([library 'lz4' is required for LZ4 support])])
-fi
-
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
@@ -1423,7 +1424,7 @@ fi
if test "$with_lz4" = yes; then
AC_CHECK_HEADERS(lz4/lz4.h, [],
- [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
fi
if test "$with_gssapi" = yes ; then
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 3463b42438..db4911ce43 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -13,7 +13,7 @@
*/
#include "postgres.h"
-#ifdef HAVE_LIBLZ4
+#ifdef USE_LZ4
#include <lz4.h>
#endif
@@ -168,7 +168,7 @@ pglz_cmdecompress_slice(const struct varlena *value,
static struct varlena *
lz4_cmcompress(const struct varlena *value)
{
-#ifndef HAVE_LIBLZ4
+#ifndef USE_LZ4
NO_LZ4_SUPPORT();
#else
int32 valsize;
@@ -212,7 +212,7 @@ lz4_cmcompress(const struct varlena *value)
static struct varlena *
lz4_cmdecompress(const struct varlena *value)
{
-#ifndef HAVE_LIBLZ4
+#ifndef USE_LZ4
NO_LZ4_SUPPORT();
#else
int32 rawsize;
@@ -246,7 +246,7 @@ lz4_cmdecompress(const struct varlena *value)
static struct varlena *
lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
{
-#ifndef HAVE_LIBLZ4
+#ifndef USE_LZ4
NO_LZ4_SUPPORT();
#elif LZ4_VERSION_NUMBER < 10803
return lz4_cmdecompress(value);
@@ -289,7 +289,7 @@ CompressionNameToMethod(char *compression)
else if (strcmp(toast_compression[LZ4_COMPRESSION_ID].cmname,
compression) == 0)
{
-#ifndef HAVE_LIBLZ4
+#ifndef USE_LZ4
NO_LZ4_SUPPORT();
#endif
return LZ4_COMPRESSION;
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 6007d72a73..0a6422da4f 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -346,9 +346,6 @@
/* Define to 1 if you have the `z' library (-lz). */
#undef HAVE_LIBZ
-/* Define to 1 if you have the `lz4' library (-llz4). */
-#undef HAVE_LIBLZ4
-
/* Define to 1 if you have the `link' function. */
#undef HAVE_LINK
@@ -902,6 +899,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to build with LZ4 support (--with-lz4) */
+#undef USE_LZ4
+
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 5f39a92111..14605371bb 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -307,7 +307,6 @@ sub GenerateFiles
HAVE_LIBXML2 => undef,
HAVE_LIBXSLT => undef,
HAVE_LIBZ => $self->{options}->{zlib} ? 1 : undef,
- HAVE_LIBLZ4 => undef,
HAVE_LINK => undef,
HAVE_LOCALE_T => 1,
HAVE_LONG_INT_64 => undef,
@@ -486,6 +485,7 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
+ USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
--
2.17.0
0006-Add-default_toast_compression-GUC.patchtext/x-diff; charset=us-asciiDownload
From a898f7a3532a62075d6d3379606b75f1607b458d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:05:05 +0530
Subject: [PATCH 6/8] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++--
src/backend/utils/misc/guc.c | 12 +++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 ++++++++-
src/test/regress/expected/compression.out | 16 +++++++
src/test/regress/expected/compression_1.out | 19 ++++++++
src/test/regress/sql/compression.sql | 8 ++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index db4911ce43..27fe47b201 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -306,3 +309,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index a66d1113db..362a371a6c 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index af0c0aa77e..9be0841d08 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index e20dd5f63c..a0d2af0cde 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11971,7 +11971,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17847,9 +17847,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 953837085f..e59ed5b214 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3915,6 +3916,17 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index f46c2dd7a8..7d7a433dcc 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -659,6 +659,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 38800cb97a..e9d9ce5634 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -15,6 +15,14 @@
#include "postgres.h"
+#include "utils/guc.h"
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/* GUCs */
+extern char *default_toast_compression;
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -36,8 +44,6 @@ typedef enum CompressionId
LZ4_COMPRESSION_ID = 1
} CompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -67,6 +73,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -115,5 +123,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 6981d580dc..9ddffc8b18 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -231,6 +231,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 893c18c7fa..03a7f46554 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -237,6 +237,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index b085c9fbd9..6c9b24f1f7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -103,6 +103,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0007-Alter-table-set-compression.patchtext/x-diff; charset=us-asciiDownload
From fff541f8c6b4abbda1fb3ac25d89dcb84217a00c Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:34:08 +0530
Subject: [PATCH 7/8] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/commands/tablecmds.c | 198 +++++++++++++++-----
src/backend/parser/gram.y | 9 +
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++-
src/test/regress/expected/compression_1.out | 48 ++++-
src/test/regress/sql/compression.sql | 20 ++
8 files changed, 296 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index 38e416d183..e147831640 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -104,6 +105,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</listitem>
</varlistentry>
+ <varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index a0d2af0cde..bb284741ce 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -532,6 +532,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3977,6 +3979,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4511,7 +4514,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4924,6 +4928,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7812,6 +7820,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
return address;
}
+/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -7827,7 +7896,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7891,47 +7959,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15118,6 +15147,89 @@ ATExecGenericOptions(Relation rel, List *options)
heap_freetuple(tuple);
}
+/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
/*
* Preparation phase for SET LOGGED/UNLOGGED
*
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index f8bc11d28b..07aa3c9330 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index fdd1bdd92a..ebe07a17cd 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 56e24529f1..b20c03f691 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1902,7 +1902,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 9ddffc8b18..19625708a9 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -247,12 +247,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 03a7f46554..bd642634a6 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -256,12 +256,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 6c9b24f1f7..00b1ff1137 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -111,6 +111,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
2.17.0
0008-default-to-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From 7aaa792891efc6cfa6688c1c4d33b03be3d3f094 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH 8/8] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 440d1e8ce5..b411ed1cb4 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 780791ae8a..392784d55c 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
2.17.0
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 10:45 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.The 2nd attempt passed ./configure on mac (and BSD after Thomas installed
pkg-config), but I eventually realized that LZ4 was effectively disabled,
because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4.
So is it working on the Mac with your latest changes in 0005 or we
need to wait for the result? Also is there any reason for changing
HAVE_LIBLZ4 to USE_LZ4?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 11:24:57AM +0530, Dilip Kumar wrote:
On Fri, Mar 12, 2021 at 10:45 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.The 2nd attempt passed ./configure on mac (and BSD after Thomas installed
pkg-config), but I eventually realized that LZ4 was effectively disabled,
because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4.So is it working on the Mac with your latest changes in 0005 or we
need to wait for the result? Also is there any reason for changing
HAVE_LIBLZ4 to USE_LZ4?
It worked everywhere (but everytime someone mail this thread, it queues a
rebuild).
http://cfbot.cputube.org/dilip-kumar.html
HAVE_LIBLZ4 was being set by AC_CHECK_LIB(), which is no longer used in favour
of pkgconfig.
https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Libraries.html
I saw that PGAC_ARG_BOOL() is setting other USE variables but not HAVE vars, so
I used USE_LZ4 and changed the code for consistency.
--
Justin
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 11:55 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
It worked everywhere (but everytime someone mail this thread, it queues a
rebuild).
http://cfbot.cputube.org/dilip-kumar.html
Okay
HAVE_LIBLZ4 was being set by AC_CHECK_LIB(), which is no longer used in favour
of pkgconfig.
https://www.gnu.org/software/autoconf/manual/autoconf-2.67/html_node/Libraries.htmlI saw that PGAC_ARG_BOOL() is setting other USE variables but not HAVE vars, so
I used USE_LZ4 and changed the code for consistency.
Okay thanks, I will include the changes in the next version.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 10:45 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.The 2nd attempt passed ./configure on mac (and BSD after Thomas installed
pkg-config), but I eventually realized that LZ4 was effectively disabled,
because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4.
With this patch, I see USE_LZ4 is never defined in my centos
machine(even --with-lz4), however it was working fine without the 0005
patch. I will have a look why it is behaving like this so I will not
include these changes until I figure out what is going on.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 2:12 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Fri, Mar 12, 2021 at 10:45 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Thu, Mar 11, 2021 at 12:25:26PM -0600, Justin Pryzby wrote:
On Wed, Mar 10, 2021 at 08:28:58PM -0600, Justin Pryzby wrote:
This includes a patch to use pkgconfig, in an attempt to build on mac, which
currently fails like:https://cirrus-ci.com/task/5993712963551232?command=build#L126
checking for LZ4_compress in -llz4... no
configure: error: library 'lz4' is required for LZ4 supportThis includes a 2nd attempt to use pkg-config to build on mac.
If this doesn't work, we should ask for help from a mac user who wants to take
on a hopefully-quick project.The 2nd attempt passed ./configure on mac (and BSD after Thomas installed
pkg-config), but I eventually realized that LZ4 was effectively disabled,
because we set HAVE_LZ4, but the code tested instead WITH_LIBLZ4.With this patch, I see USE_LZ4 is never defined in my centos
machine(even --with-lz4), however it was working fine without the 0005
patch. I will have a look why it is behaving like this so I will not
include these changes until I figure out what is going on.
Just realized I was still checking for HAVE_LIBLZ4 not USE_LZ4, sorry
for the noise its working fine. And thanks for making it work for
mac.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
I think these names need to be more specific.
+typedef enum CompressionId
+{
+ PGLZ_COMPRESSION_ID = 0,
+ LZ4_COMPRESSION_ID = 1
+CompressionId, PGLZ_COMPRESSION_ID, LZ4_COMPRESSION_ID are also being used by
Andrey's WAL compression patch. I suggested he use a prefix, but your patch is
also of limited scope (TOAST).
--
Justin
Re: [HACKERS] Custom compression methods
On Fri, Mar 12, 2021 at 4:06 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think these names need to be more specific.
+typedef enum CompressionId +{ + PGLZ_COMPRESSION_ID = 0, + LZ4_COMPRESSION_ID = 1 +CompressionId, PGLZ_COMPRESSION_ID, LZ4_COMPRESSION_ID are also being used by
Andrey's WAL compression patch. I suggested he use a prefix, but your patch is
also of limited scope (TOAST).
In the attached patches I have changed this, other than this I have
run pg_indent on all the patches and now CTAS and Matview will compare
the compression method with the target attribute and it will
recompress if required. CLUSTER, VACUUM
FULL(reform_and_rewrite_tuple) and ATRewriteTable will also recompress
as per the target attribute. But the changes of
reform_and_rewrite_tuple and ATRewriteTable are in patch 0005, because
now whenever we are moving to target table we are always recompress so
there is no use-case to recompress in reform_and_rewrite_tuple and
ATRewriteTable unless we provide an option to alter the attribute
compression. Basically, we are allowed to change the attribute
compression without any rewrite but on the very next opportunity when
the table is getting rewritten the data will be recompressed based on
the current compression method. Also done some improvement and
comment changes in patch 0002 and patch 0006.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v35-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchtext/x-patch; charset=US-ASCII; name=v35-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchDownload
From 84512feb6d557e7ff32c42dcb71881af47ec646b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH v35 1/7] Get datum from tuple which doesn't contain external
data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 17 +++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 1 +
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 125 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..43bb2ab 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..c36c283 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,21 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 80d2d20..67385e3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..e47ab1e 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to
+ * flatten any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index e5f8a06..f973aea 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1420,5 +1420,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..5936c39 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2259,7 +2260,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..079a5af 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..59eb42f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9798,7 +9798,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..8771ccd 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,7 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..8ba7ae2 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v35-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v35-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From 2bc9f6abfb87a40cf24e60ef41a7929d19cfbb9b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v35 2/7] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/executor/execExprInterp.c | 29 +++++++++++++++++++++++++++--
src/backend/utils/adt/jsonfuncs.c | 9 +++++++--
src/pl/plpgsql/src/pl_exec.c | 19 ++++++++++++++-----
3 files changed, 48 insertions(+), 9 deletions(-)
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index e47ab1e..772ab11 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,12 +2840,25 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
+ /* detoast any external data before forming the tuple */
+ for (int i = 0; i < op->d.row.tupdesc->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
+
+ if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ continue;
+
+ op->d.row.elemvalues[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.row.elemvalues[i]));
+ }
+
/* build tuple from evaluated field values */
tuple = heap_form_tuple(op->d.row.tupdesc,
op->d.row.elemvalues,
op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3085,12 +3098,24 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
+ /* detoast any external data before forming the tuple */
+ for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
+
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ continue;
+ op->d.fieldstore.values[i] =
+ PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
+ }
+
/* argdesc should already be valid from the DeForm step */
tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
op->d.fieldstore.values,
op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..dc3f9d4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,11 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ /* detoast any external data before forming the tuple */
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3770,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..d4fcb70 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -404,7 +404,8 @@ static void exec_move_row_from_fields(PLpgSQL_execstate *estate,
static bool compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc);
static HeapTuple make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc);
+ TupleDesc tupdesc,
+ bool flatten);
static TupleDesc deconstruct_composite_datum(Datum value,
HeapTupleData *tmptup);
static void exec_move_row_from_datum(PLpgSQL_execstate *estate,
@@ -3418,7 +3419,7 @@ exec_stmt_return_next(PLpgSQL_execstate *estate,
/* Use eval_mcontext for tuple conversion work */
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tuple = make_tuple_from_row(estate, row, tupdesc);
+ tuple = make_tuple_from_row(estate, row, tupdesc, false);
if (tuple == NULL) /* should not happen */
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
@@ -5321,12 +5322,12 @@ exec_eval_datum(PLpgSQL_execstate *estate,
/* Make sure we have a valid type/typmod setting */
BlessTupleDesc(row->rowtupdesc);
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tup = make_tuple_from_row(estate, row, row->rowtupdesc);
+ tup = make_tuple_from_row(estate, row, row->rowtupdesc, true);
if (tup == NULL) /* should not happen */
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7265,12 +7266,15 @@ compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc)
*
* The result tuple is freshly palloc'd in caller's context. Some junk
* may be left behind in eval_mcontext, too.
+ *
+ * flatten - if this is passed true then any external data will be flattened.
* ----------
*/
static HeapTuple
make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc)
+ TupleDesc tupdesc,
+ bool flatten)
{
int natts = tupdesc->natts;
HeapTuple tuple;
@@ -7300,6 +7304,11 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+
+ /* detoast any external data if the caller has asked to do so */
+ if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v35-0004-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v35-0004-Add-default_toast_compression-GUC.patchDownload
From 39b895a720e5202ec67a536d4509da9996d86c89 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Mon, 15 Mar 2021 14:21:19 +0530
Subject: [PATCH v35 4/7] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++---
src/backend/utils/misc/guc.c | 12 +++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 +++++++++++--
src/test/regress/expected/compression.out | 16 ++++++++++
src/test/regress/expected/compression_1.out | 19 +++++++++++
src/test/regress/sql/compression.sql | 8 +++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index f8d7fdb..d337b7c 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -308,3 +311,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index a66d111..362a371 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index af0c0aa..9be0841 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7d160d5..cf90f40 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 59eb42f..e1dd4cf 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3917,6 +3918,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 1ccf5c7..e73c453 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -15,6 +15,14 @@
#include "postgres.h"
+#include "utils/guc.h"
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/* GUCs */
+extern char *default_toast_compression;
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -36,8 +44,6 @@ typedef enum ToastCompressionId
TOAST_LZ4_COMPRESSION_ID = 1
} ToastCompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -67,6 +73,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -115,5 +123,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3706e76..1773aeb 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -212,6 +212,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3fb8307..136e750 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -208,6 +208,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 168825d..1834ba7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -92,6 +92,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v35-0005-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v35-0005-Alter-table-set-compression.patchDownload
From 71104db9a2cad3fb8f64cb43e7f62ccbbe9a1952 Mon Sep 17 00:00:00 2001
From: dilipkumar <dilipbalaut@gmail.com>
Date: Mon, 15 Mar 2021 15:08:02 +0530
Subject: [PATCH v35 5/7] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 16 ++
src/backend/access/heap/heapam_handler.c | 22 +++
src/backend/commands/createas.c | 3 +-
src/backend/commands/matview.c | 3 +-
src/backend/commands/tablecmds.c | 228 ++++++++++++++++++++++------
src/backend/executor/nodeModifyTable.c | 9 +-
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/executor/executor.h | 3 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 +++++-
src/test/regress/expected/compression_1.out | 48 +++++-
src/test/regress/sql/compression.sql | 20 +++
13 files changed, 356 insertions(+), 57 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index 7664619..b74ba00 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -2471,6 +2471,28 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+
+ /*
+ * Since we are rewriting the table, use this opportunity to
+ * recompress any compressed attribute with current compression method
+ * of the attribute. Basically, if the compression method of the
+ * compressed varlena is not same as current compression method of the
+ * attribute then decompress it so that if it need to be compressed
+ * then it will be compressed with the current compression method of
+ * the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmethod = toast_get_compression_method(new_value);
+
+ if (IsValidCompression(cmethod) &&
+ TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index c4ade98..0489fd0 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -590,7 +590,8 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->rel->rd_att);
+ myState->rel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index 1e31122..4fc0677 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -495,7 +495,8 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&myState->decompress_tuple_slot,
- myState->transientrel->rd_att);
+ myState->transientrel->rd_att,
+ NULL);
/*
* Note that the input slot might not be of the type of the target
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cf90f40..4476e2c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -5528,15 +5536,35 @@ ATRewriteTable(AlteredTableInfo *tab, Oid OIDNewHeap, LOCKMODE lockmode)
if (tab->rewrite > 0)
{
+ bool decompressed = false;
+
/* Extract data from old tuple */
slot_getallattrs(oldslot);
ExecClearTuple(newslot);
- /* copy attributes */
- memcpy(newslot->tts_values, oldslot->tts_values,
- sizeof(Datum) * oldslot->tts_nvalid);
- memcpy(newslot->tts_isnull, oldslot->tts_isnull,
- sizeof(bool) * oldslot->tts_nvalid);
+ /*
+ * Compare the compression method of the compressed data in
+ * the source tuple with that of target attribute and if those
+ * are different then decompress those data.
+ */
+ (void) CompareCompressionMethodAndDecompress(oldslot,
+ &newslot,
+ newTupDesc,
+ &decompressed);
+
+ /*
+ * copy attributes, if we have decompressed some attribute
+ * then the values and nulls array is already copied
+ */
+ if (!decompressed)
+ {
+ memcpy(newslot->tts_values, oldslot->tts_values,
+ sizeof(Datum) * oldslot->tts_nvalid);
+ memcpy(newslot->tts_isnull, oldslot->tts_isnull,
+ sizeof(bool) * oldslot->tts_nvalid);
+ }
+ else
+ ExecClearTuple(newslot);
/* Set dropped attributes to null in new tuple */
foreach(lc, dropped_attrs)
@@ -7773,6 +7801,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7787,7 +7876,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7939,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15066,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index dfedd70..cafd772 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -2048,7 +2048,8 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
TupleTableSlot *
CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc)
+ TupleDesc targetTupDesc,
+ bool *decompressed)
{
int i;
int attnum;
@@ -2143,6 +2144,9 @@ CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
if (TTS_SHOULDFREE(slot))
ExecMaterializeSlot(newslot);
+ if (decompressed != NULL)
+ *decompressed = true;
+
*outslot = newslot;
return newslot;
@@ -2367,7 +2371,8 @@ ExecModifyTable(PlanState *pstate)
*/
slot = CompareCompressionMethodAndDecompress(slot,
&node->mt_decompress_tuple_slot,
- resultRelInfo->ri_RelationDesc->rd_att);
+ resultRelInfo->ri_RelationDesc->rd_att,
+ NULL);
switch (operation)
{
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208a..07ba55f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 483b9f9..65a2f8d 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -623,5 +623,6 @@ extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
TupleTableSlot **outslot,
- TupleDesc targetTupDesc);
+ TupleDesc targetTupDesc,
+ bool *decompressed);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 1773aeb..f206a69 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -228,12 +228,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 136e750..2292870 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 1834ba7..12cbfe8 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v35-0003-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v35-0003-Built-in-compression-method.patchDownload
From 918557fb6207604431930797c7368f13e9bb5473 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v35 3/7] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Justin Pryzby, Tomas Vondra and Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 170 ++++++++++++++
configure.ac | 20 ++
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 73 ++++--
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 310 ++++++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 54 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/heap/heapam_handler.c | 2 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/createas.c | 15 ++
src/backend/commands/matview.c | 15 ++
src/backend/commands/tablecmds.c | 110 +++++++++
src/backend/executor/nodeModifyTable.c | 129 +++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/utils/adt/varlena.c | 41 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 ++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 8 +
src/include/access/toast_compression.h | 119 ++++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 23 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/executor/executor.h | 4 +-
src/include/nodes/execnodes.h | 6 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 14 +-
src/test/regress/expected/compression.out | 248 +++++++++++++++++++++
src/test/regress/expected/compression_1.out | 240 ++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 102 +++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
58 files changed, 1877 insertions(+), 86 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..a0ae4b4 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,9 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -891,6 +895,8 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1596,6 +1603,8 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
@@ -8564,6 +8573,137 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -13322,6 +13462,36 @@ fi
fi
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..aa57091 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,21 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -1406,6 +1421,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0..5fdc80f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,16 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..7c8fc77 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..7bf1b8f 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -16,6 +16,7 @@
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/toast_internals.h"
#include "common/int.h"
#include "common/pg_lzcompress.h"
@@ -457,6 +458,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+
+ /*
+ * Fetch just enough of the value to examine the compression header,
+ * so that we can find out the compression method.
+ */
+ attr = toast_fetch_datum_slice(attr, 0, VARHDRSZ);
+ }
+ else if (!VARATT_IS_COMPRESSED(attr))
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+}
+
+/* ----------
* toast_decompress_datum -
*
* Decompress a compressed version of a varlena datum
@@ -464,21 +496,19 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ char cmethod;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in
+ * the toast header.
+ */
+ cmethod = CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+ cmroutine = GetCompressionRoutines(cmethod);
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +522,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ char cmethod;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in
+ * the toast header.
+ */
+ cmethod = CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+ cmroutine = GetCompressionRoutines(cmethod);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..f8d7fdb
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,310 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef USE_LZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* slice decompression not supported prior to 1.8.3 */
+ if (LZ4_versionNumber() < 10803)
+ return lz4_cmdecompress(value);
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[TOAST_PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[TOAST_LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine *
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..69dd949 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..a66d111 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0..7664619 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..af0c0aa 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d0ec44b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..933a073 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/createas.c b/src/backend/commands/createas.c
index dce8820..c4ade98 100644
--- a/src/backend/commands/createas.c
+++ b/src/backend/commands/createas.c
@@ -61,6 +61,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_intorel;
/* utility functions for CTAS definition creation */
@@ -582,6 +583,16 @@ intorel_receive(TupleTableSlot *slot, DestReceiver *self)
if (!myState->into->skipData)
{
/*
+ * If the compression method of the compressed data is not the same as
+ * the compression method of the target attribute then decompress the
+ * data so that data can be compressed back as per the target
+ * attribute's current compression method if required.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->rel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly
* less efficient than inserting with the right slot - but the
@@ -619,6 +630,10 @@ intorel_shutdown(DestReceiver *self)
/* close rel, but keep lock until commit */
table_close(myState->rel, NoLock);
myState->rel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/matview.c b/src/backend/commands/matview.c
index c5c25ce..1e31122 100644
--- a/src/backend/commands/matview.c
+++ b/src/backend/commands/matview.c
@@ -56,6 +56,7 @@ typedef struct
CommandId output_cid; /* cmin to insert in output tuples */
int ti_options; /* table_tuple_insert performance options */
BulkInsertState bistate; /* bulk insert state */
+ TupleTableSlot *decompress_tuple_slot; /* to hold the decompress tuple */
} DR_transientrel;
static int matview_maintenance_depth = 0;
@@ -487,6 +488,16 @@ transientrel_receive(TupleTableSlot *slot, DestReceiver *self)
DR_transientrel *myState = (DR_transientrel *) self;
/*
+ * If the compression method of the compressed data is not the same as the
+ * compression method of the target attribute then decompress the data so
+ * that data can be compressed back as per the target attribute's current
+ * compression method if required.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &myState->decompress_tuple_slot,
+ myState->transientrel->rd_att);
+
+ /*
* Note that the input slot might not be of the type of the target
* relation. That's supported by table_tuple_insert(), but slightly less
* efficient than inserting with the right slot - but the alternative
@@ -521,6 +532,10 @@ transientrel_shutdown(DestReceiver *self)
/* close transientrel, but keep lock until commit */
table_close(myState->transientrel, NoLock);
myState->transientrel = NULL;
+
+ /* release the slot used for decompressing the tuple */
+ if (myState->decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(myState->decompress_tuple_slot);
}
/*
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..7d160d5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression =
+ GetCompressionMethodName(attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/executor/nodeModifyTable.c b/src/backend/executor/nodeModifyTable.c
index 2993ba4..dfedd70 100644
--- a/src/backend/executor/nodeModifyTable.c
+++ b/src/backend/executor/nodeModifyTable.c
@@ -37,9 +37,11 @@
#include "postgres.h"
+#include "access/detoast.h"
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/catalog.h"
#include "commands/trigger.h"
@@ -2036,6 +2038,119 @@ ExecPrepareTupleRouting(ModifyTableState *mtstate,
return slot;
}
+/*
+ * Compare the compression method of each compressed value with the
+ * compression method of the target attribute. If the compression method
+ * of the compressed value is not same as the target attribute then decompress
+ * the value. If any of the value need to be decompressed then we need to
+ * store that into a new slot.
+ */
+TupleTableSlot *
+CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc)
+{
+ int i;
+ int attnum;
+ int natts = slot->tts_tupleDescriptor->natts;
+ bool decompressed_any = false;
+ bool slot_tup_deformed = false;
+ TupleDesc tupleDesc = slot->tts_tupleDescriptor;
+ TupleTableSlot *newslot = *outslot;
+
+ if (natts == 0)
+ return slot;
+
+ /*
+ * Loop over all the attributes in the tuple and check if any attribute is
+ * compressed and its compression method is not same as the target
+ * atrribute's compression method then decompress it.
+ */
+ for (i = 0; i < natts; i++)
+ {
+ attnum = tupleDesc->attrs[i].attnum;
+
+ if (TupleDescAttr(tupleDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ /* fetch the values of all the attribute, if not already done */
+ if (!slot_tup_deformed)
+ {
+ slot_getallattrs(slot);
+ slot_tup_deformed = true;
+ }
+
+ /* nothing to be done, if the value is null */
+ if (slot->tts_isnull[attnum - 1])
+ continue;
+
+ new_value = (struct varlena *)
+ DatumGetPointer(slot->tts_values[attnum - 1]);
+
+ /*
+ * Get the compression method stored in the toast header and
+ * compare it with the compression method of the target.
+ */
+ cmethod = toast_get_compression_method(new_value);
+ if (IsValidCompression(cmethod) &&
+ targetTupDesc->attrs[i].attcompression != cmethod)
+ {
+ if (!decompressed_any)
+ {
+ /*
+ * If the caller has passed an invalid slot then create a
+ * new slot. Otherwise, just clear the existing tuple from
+ * the slot. This slot should be stored by the caller so
+ * that it can be reused for decompressing the subsequent
+ * tuples.
+ */
+ if (newslot == NULL)
+ newslot = MakeSingleTupleTableSlot(tupleDesc,
+ slot->tts_ops);
+ else
+ ExecClearTuple(newslot);
+
+ /* copy the value/null array to the new slot */
+ memcpy(newslot->tts_values, slot->tts_values,
+ natts * sizeof(Datum));
+ memcpy(newslot->tts_isnull, slot->tts_isnull,
+ natts * sizeof(bool));
+
+ }
+
+ /* detoast the value and store into the new slot */
+ new_value = detoast_attr(new_value);
+ newslot->tts_values[attnum - 1] = PointerGetDatum(new_value);
+ decompressed_any = true;
+ }
+ }
+ }
+
+ /*
+ * If we have decompressed any of the fields then return the new slot
+ * otherwise return the original slot.
+ */
+ if (decompressed_any)
+ {
+ ExecStoreVirtualTuple(newslot);
+
+ /*
+ * If the original slot was materialized then materialize the new slot
+ * as well.
+ */
+ if (TTS_SHOULDFREE(slot))
+ ExecMaterializeSlot(newslot);
+
+ *outslot = newslot;
+
+ return newslot;
+ }
+ else
+ return slot;
+}
+
/* ----------------------------------------------------------------
* ExecModifyTable
*
@@ -2244,6 +2359,16 @@ ExecModifyTable(PlanState *pstate)
slot = ExecFilterJunk(junkfilter, slot);
}
+ /*
+ * If the compression method of the compressed data is not same as the
+ * compression method of the target attribute then decompress the data
+ * so that data can be compressed back as per the target attribute's
+ * current compression method if required.
+ */
+ slot = CompareCompressionMethodAndDecompress(slot,
+ &node->mt_decompress_tuple_slot,
+ resultRelInfo->ri_RelationDesc->rd_att);
+
switch (operation)
{
case CMD_INSERT:
@@ -2876,6 +3001,10 @@ ExecEndModifyTable(ModifyTableState *node)
ExecDropSingleTupleTableSlot(node->mt_root_tuple_slot);
}
+ /* release the slot used for decompressing the tuple */
+ if (node->mt_decompress_tuple_slot)
+ ExecDropSingleTupleTableSlot(node->mt_decompress_tuple_slot);
+
/*
* Free the exprcontext
*/
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..681151e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..187e55b 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5300,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..cef9bbd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..453f946 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a43a8f5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..9a428aa 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -89,4 +89,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..1ccf5c7
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,119 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "postgres.h"
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define TOAST_PGLZ_COMPRESSION 'p'
+#define TOAST_LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum ToastCompressionId
+{
+ TOAST_PGLZ_COMPRESSION_ID = 0,
+ TOAST_LZ4_COMPRESSION_ID = 1
+} ToastCompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop ToastCompressionId.
+ */
+static inline ToastCompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ return TOAST_PGLZ_COMPRESSION_ID;
+ case TOAST_LZ4_COMPRESSION:
+ return TOAST_LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop ToastCompressionId.
+ */
+static inline char
+CompressionIdToMethod(ToastCompressionId cmid)
+{
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return TOAST_PGLZ_COMPRESSION;
+ case TOAST_LZ4_COMPRESSION_ID:
+ return TOAST_LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char *
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..b4d0684 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,26 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) \
+ (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = \
+ ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..faa5456 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7098,6 +7098,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/executor/executor.h b/src/include/executor/executor.h
index 071e363..483b9f9 100644
--- a/src/include/executor/executor.h
+++ b/src/include/executor/executor.h
@@ -621,5 +621,7 @@ extern void CheckCmdReplicaIdentity(Relation rel, CmdType cmd);
extern void CheckSubscriptionRelkind(char relkind, const char *nspname,
const char *relname);
-
+extern TupleTableSlot *CompareCompressionMethodAndDecompress(TupleTableSlot *slot,
+ TupleTableSlot **outslot,
+ TupleDesc targetTupDesc);
#endif /* EXECUTOR_H */
diff --git a/src/include/nodes/execnodes.h b/src/include/nodes/execnodes.h
index e31ad62..e388336 100644
--- a/src/include/nodes/execnodes.h
+++ b/src/include/nodes/execnodes.h
@@ -1185,6 +1185,12 @@ typedef struct ModifyTableState
*/
TupleTableSlot *mt_root_tuple_slot;
+ /*
+ * Slot for storing the modified tuple, in case the target attribute's
+ * compression method doesn't match that of the source table.
+ */
+ TupleTableSlot *mt_decompress_tuple_slot;
+
/* Tuple-routing support info */
struct PartitionTupleRouting *mt_partition_tuple_routing;
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..3b72038 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -896,6 +896,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to build with LZ4 support (--with-lz4) */
+#undef USE_LZ4
+
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..e98af05 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -145,7 +145,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +275,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3706e76
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,248 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..3fb8307
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,240 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..168825d
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,102 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..95abad2 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -484,6 +484,7 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
+ USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..0fc7017 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v35-0006-Disallow-compressed-data-inside-container-types.patchtext/x-patch; charset=US-ASCII; name=v35-0006-Disallow-compressed-data-inside-container-types.patchDownload
From f0d531a82abae9c9c40fdae61d598d87ce303774 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Mon, 15 Mar 2021 15:57:45 +0530
Subject: [PATCH v35 6/7] Disallow compressed data inside container types
Currently, we have a general rule that Datums of container types
(rows, arrays, ranges, etc) must not contain any external TOAST
pointers. But the rule for the compressed data is not defined
and no specific rule is followed e.g. while constructing the array
we decompress the compressed field but while constructing the row
in some cases we don't decompress the compressed data whereas in
the other cases we only decompress while flattening the external
toast pointers. This patch make a general rule for the compressed
data i.e. we don't allow the compressed data in the container type.
Dilip Kumar based on idea from Robert Haas
---
src/backend/access/common/heaptuple.c | 13 +++---
src/backend/access/heap/heaptoast.c | 4 +-
src/backend/executor/execExprInterp.c | 10 ++---
src/backend/executor/execTuples.c | 4 --
src/backend/utils/adt/expandedrecord.c | 78 ++++++++++++----------------------
src/backend/utils/adt/jsonfuncs.c | 5 +--
src/include/funcapi.h | 4 +-
src/pl/plpgsql/src/pl_exec.c | 11 +++--
8 files changed, 50 insertions(+), 79 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index c36c283..12f6fdd 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -984,15 +984,12 @@ Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
/*
- * If the tuple contains any external TOAST pointers, we have to inline
- * those fields to meet the conventions for composite-type Datums.
+ * We have to inline any external/compressed data to meet the conventions
+ * for composite-type Datums.
*/
- if (HeapTupleHasExternal(tuple))
- return toast_flatten_tuple_to_datum(tuple->t_data,
- tuple->t_len,
- tupleDesc);
- else
- return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+ return toast_flatten_tuple_to_datum(tuple->t_data,
+ tuple->t_len,
+ tupleDesc);
}
/* ----------------
diff --git a/src/backend/access/heap/heaptoast.c b/src/backend/access/heap/heaptoast.c
index 55bbe1d..b094623 100644
--- a/src/backend/access/heap/heaptoast.c
+++ b/src/backend/access/heap/heaptoast.c
@@ -589,9 +589,9 @@ toast_build_flattened_tuple(TupleDesc tupleDesc,
struct varlena *new_value;
new_value = (struct varlena *) DatumGetPointer(new_values[i]);
- if (VARATT_IS_EXTERNAL(new_value))
+ if (VARATT_IS_EXTERNAL(new_value) || VARATT_IS_COMPRESSED(new_value))
{
- new_value = detoast_external_attr(new_value);
+ new_value = detoast_attr(new_value);
new_values[i] = PointerGetDatum(new_value);
freeable_values[num_to_free++] = (Pointer) new_value;
}
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 772ab11..f8ed1fa 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2840,13 +2840,12 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
{
HeapTuple tuple;
- /* detoast any external data before forming the tuple */
+ /* detoast any external/compressed data before forming the tuple */
for (int i = 0; i < op->d.row.tupdesc->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(op->d.row.tupdesc, i);
- if (op->d.row.elemnulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.row.elemvalues[i])))
+ if (op->d.row.elemnulls[i] || attr->attlen != -1)
continue;
op->d.row.elemvalues[i] =
@@ -3098,13 +3097,12 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
HeapTuple tuple;
- /* detoast any external data before forming the tuple */
+ /* detoast any external/compressed data before forming the tuple */
for (int i = 0; i < (*op->d.fieldstore.argdesc)->natts; i++)
{
Form_pg_attribute attr = TupleDescAttr(*op->d.fieldstore.argdesc, i);
- if (op->d.fieldstore.nulls[i] || attr->attlen != -1 ||
- !VARATT_IS_EXTERNAL(DatumGetPointer(op->d.fieldstore.values[i])))
+ if (op->d.fieldstore.nulls[i] || attr->attlen != -1)
continue;
op->d.fieldstore.values[i] =
PointerGetDatum(PG_DETOAST_DATUM_PACKED(op->d.fieldstore.values[i]));
diff --git a/src/backend/executor/execTuples.c b/src/backend/executor/execTuples.c
index 73c35df..f115464 100644
--- a/src/backend/executor/execTuples.c
+++ b/src/backend/executor/execTuples.c
@@ -2207,10 +2207,6 @@ HeapTupleHeaderGetDatum(HeapTupleHeader tuple)
Datum result;
TupleDesc tupDesc;
- /* No work if there are no external TOAST pointers in the tuple */
- if (!HeapTupleHeaderHasExternal(tuple))
- return PointerGetDatum(tuple);
-
/* Use the type data saved by heap_form_tuple to look up the rowtype */
tupDesc = lookup_rowtype_tupdesc(HeapTupleHeaderGetTypeId(tuple),
HeapTupleHeaderGetTypMod(tuple));
diff --git a/src/backend/utils/adt/expandedrecord.c b/src/backend/utils/adt/expandedrecord.c
index e19491e..765d58c 100644
--- a/src/backend/utils/adt/expandedrecord.c
+++ b/src/backend/utils/adt/expandedrecord.c
@@ -673,14 +673,6 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
erh->er_typmod = tupdesc->tdtypmod;
}
- /*
- * If we have a valid flattened value without out-of-line fields, we can
- * just use it as-is.
- */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- return erh->fvalue->t_len;
-
/* If we have a cached size value, believe that */
if (erh->flat_size)
return erh->flat_size;
@@ -693,38 +685,36 @@ ER_get_flat_size(ExpandedObjectHeader *eohptr)
tupdesc = erh->er_tupdesc;
/*
- * Composite datums mustn't contain any out-of-line values.
+ * Composite datums mustn't contain any out-of-line/compressed values.
*/
- if (erh->flags & ER_FLAG_HAVE_EXTERNAL)
+ for (i = 0; i < erh->nfields; i++)
{
- for (i = 0; i < erh->nfields; i++)
- {
- Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
+ Form_pg_attribute attr = TupleDescAttr(tupdesc, i);
- if (!erh->dnulls[i] &&
- !attr->attbyval && attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])))
- {
- /*
- * expanded_record_set_field_internal can do the actual work
- * of detoasting. It needn't recheck domain constraints.
- */
- expanded_record_set_field_internal(erh, i + 1,
- erh->dvalues[i], false,
- true,
- false);
- }
+ if (!erh->dnulls[i] &&
+ !attr->attbyval && attr->attlen == -1 &&
+ (VARATT_IS_EXTERNAL(DatumGetPointer(erh->dvalues[i])) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(erh->dvalues[i]))))
+ {
+ /*
+ * expanded_record_set_field_internal can do the actual work of
+ * detoasting. It needn't recheck domain constraints.
+ */
+ expanded_record_set_field_internal(erh, i + 1,
+ erh->dvalues[i], false,
+ true,
+ false);
}
-
- /*
- * We have now removed all external field values, so we can clear the
- * flag about them. This won't cause ER_flatten_into() to mistakenly
- * take the fast path, since expanded_record_set_field() will have
- * cleared ER_FLAG_FVALUE_VALID.
- */
- erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
}
+ /*
+ * We have now removed all external field values, so we can clear the flag
+ * about them. This won't cause ER_flatten_into() to mistakenly take the
+ * fast path, since expanded_record_set_field() will have cleared
+ * ER_FLAG_FVALUE_VALID.
+ */
+ erh->flags &= ~ER_FLAG_HAVE_EXTERNAL;
+
/* Test if we currently have any null values */
hasnull = false;
for (i = 0; i < erh->nfields; i++)
@@ -770,19 +760,6 @@ ER_flatten_into(ExpandedObjectHeader *eohptr,
Assert(erh->er_magic == ER_MAGIC);
- /* Easy if we have a valid flattened value without out-of-line fields */
- if (erh->flags & ER_FLAG_FVALUE_VALID &&
- !(erh->flags & ER_FLAG_HAVE_EXTERNAL))
- {
- Assert(allocated_size == erh->fvalue->t_len);
- memcpy(tuphdr, erh->fvalue->t_data, allocated_size);
- /* The original flattened value might not have datum header fields */
- HeapTupleHeaderSetDatumLength(tuphdr, allocated_size);
- HeapTupleHeaderSetTypeId(tuphdr, erh->er_typeid);
- HeapTupleHeaderSetTypMod(tuphdr, erh->er_typmod);
- return;
- }
-
/* Else allocation should match previous get_flat_size result */
Assert(allocated_size == erh->flat_size);
@@ -1151,15 +1128,16 @@ expanded_record_set_field_internal(ExpandedRecordHeader *erh, int fnumber,
{
MemoryContext oldcxt;
- /* If requested, detoast any external value */
+ /* If requested, detoast any external/compressed value */
if (expand_external)
{
if (attr->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(newValue)))
+ (VARATT_IS_EXTERNAL(DatumGetPointer(newValue)) ||
+ VARATT_IS_COMPRESSED(DatumGetPointer(newValue))))
{
/* Detoasting should be done in short-lived context. */
oldcxt = MemoryContextSwitchTo(get_short_term_cxt(erh));
- newValue = PointerGetDatum(detoast_external_attr((struct varlena *) DatumGetPointer(newValue)));
+ newValue = PointerGetDatum(detoast_attr((struct varlena *) DatumGetPointer(newValue)));
MemoryContextSwitchTo(oldcxt);
}
else
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index dc3f9d4..cccd675 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -3388,9 +3388,8 @@ populate_record(TupleDesc tupdesc,
&field,
&nulls[i]);
- /* detoast any external data before forming the tuple */
- if (!nulls[i] && att->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ /* detoast any external/compressed data before forming the tuple */
+ if (!nulls[i] && att->attlen == -1)
values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 8ba7ae2..c869012 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -208,10 +208,10 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Macro declarations/inline functions:
* HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
* HeapTupleHeaderGetDatum but the input tuple should not contain
- * external varlena
+ * external/compressed varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
* HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
- * but the input tuple should not contain external varlena
+ * but the input tuple should not contain external/compressed varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index d4fcb70..af2c85f 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -7267,7 +7267,8 @@ compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc)
* The result tuple is freshly palloc'd in caller's context. Some junk
* may be left behind in eval_mcontext, too.
*
- * flatten - if this is passed true then any external data will be flattened.
+ * flatten - if this is passed true then any external/compressed data will be
+ * flattened.
* ----------
*/
static HeapTuple
@@ -7305,9 +7306,11 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
- /* detoast any external data if the caller has asked to do so */
- if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
- VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ /*
+ * detoast any external/compressed data if the caller has asked to do
+ * so.
+ */
+ if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1)
dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v35-0007-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v35-0007-default-to-with-lz4.patchDownload
From c2bb66e98372cf3e720f397bdfd415a657d13f96 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v35 7/7] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index a0ae4b4..fc57fef 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index aa57091..0d75aa4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
In the attached patches I have changed this, ...
OK, so just looking over this patch series, here's what I think:
- 0001 and 0002 are now somewhat independent of the rest of this work,
and could be dropped, but I think they're a good idea, so I'd like to
commit them. I went over 0001 carefully this morning and didn't find
any problems. I still need to do some more review of 0002.
- 0003 through 0005 are the core of this patch set. I'd like to get
them into this release, but I think we're likely to run out of time.
- I don't think we want to proceed with 0006 at this time. It needs
broader buy-in, I think, and I think it also needs some other
improvements, at the least to the comments.
- 0007 is not intended for commit, but just exists to fool the
CommitFest bot into testing the feature.
Regarding 0003:
The biggest thing that jumps out at me while looking at this with
fresh eyes is that the patch doesn't touch varatt_external.va_extsize
at all. In a varatt_external, we can't use the va_rawsize to indicate
the compression method, because there are no bits free, because the 2
bits not required to store the size are used to indicate what type of
varlena we've got. But, that means that the size of a varlena is
limited to 1GB, so there are 2 bits free in
varatt_external.va_extsize, just like there are in
va_compressed.va_rawsize. We could store the same two bits in
varatt_external.va_extsize that we're storing in
va_compressed.va_rawsize aka va_tcinfo. That's a big deal, because
then toast_get_compression_method() doesn't have to call
toast_fetch_datum_slice() any more, which is a rather large savings.
If it's only impacting pg_column_compression() then whatever, but
that's not the case: we've got calls to
CompareCompressionMethodAndDecompress in places like intorel_receive()
and ExecModifyTable() that look pretty performance-critical.
I think that CompareCompressionMethodAndDecompress() could be
redesigned to be more efficient by moving more of the per-tuple work
into a separate setup phase. Consider a case where the tuple has 300
columns. 299 of them are fixed-with, but column 100 is a varlena. In
an ideal world, we would do slot_getsomeattrs(slot, 100). Then we
would check whether column is not null, whether it is compressed, and
whether the compression method is the one we want. If recompression is
required, then we must slot_getallattrs(slot), memcpy all the values
to the virtual slot created for this purpose, and decompress and
recompress column 100. But, if column 100 is not null, then we need
not ever deform beyond column 100, and in no case do we need to
iterate over all 300 attributes. But the current code will do just
that. For every new tuple, it loops again over every attribute and
re-discovers which ones are varlenas. That's kinda the pits.
I got thinking about this after looking at ExecFilterJunk(). That
function is extremely simple precisely because all the work of
figuring out what should be done has been precomputed. All the smarts
are in cleanmap[], which is set up before we actually begin execution.
In a similar way, you can imagine creating some sort of object, let's
call it a CompressedAttributeFilter, that looks at the tupledesc
figures out from the tupledesc which columns we need to consider
recompressing and puts them in an array. Then you have a struct that
stores a pointer to the array, the number of elements in the array,
and the value of the last array element. You pass this struct to what
is now CompareCompressionMethodAndDecompress() and it can now run more
like what I described above.
It's possible to imagine doing even better. Imagine that for every
column we maintain an attcompression value and an
attpreservecompression value. The former indicates the compression
type for the column or '\0' if it cannot be compressed, and the latter
indicates whether any other compression type might be present. Then,
when we build the CompressedAttributeFilter object, we can exclude
varlena attributes if attpreservecompression is false and
attcompression is '\0' or matches the attcompression value for the
corresponding attribute in the table into which ExecModifyTable() or
intorel_receive() will be putting the tuple. This seems quite complex
in terms of bookkeeping, but it would allow us to elide basically all
of the per-tuple bookkeeping in a lot of common cases, such as UPDATE,
or an INSERT or CTAS into a table that's using the same compression
method as the source data. You could probably contrive it so that you
have a CompressedAttributeFilter pointer that's NULL if no such
treatment is required, just like we already do for junkfilter.
There's another, rather brute-force approach to this problem, too. We
could just decide that lz4 will only be used for external data, and
that there's no such thing as an inline-compressed lz4 varlena.
deotast_fetch_datum() would just notice that the value is lz4'd and
de-lz4 it before returning it, since a compressed lz4 datum is
impossible.
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
On 2021-03-15 15:29:05 -0400, Robert Haas wrote:
On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
In the attached patches I have changed this, ...
OK, so just looking over this patch series, here's what I think:
- 0001 and 0002 are now somewhat independent of the rest of this work,
and could be dropped, but I think they're a good idea, so I'd like to
commit them. I went over 0001 carefully this morning and didn't find
any problems. I still need to do some more review of 0002.
I don't particularly like PG_RETURN_HEAPTUPLEHEADER_RAW(). What is "raw"
about it? It also seems to me like there needs to at least be a
sentence or two explaining when to use which of the functions.
I think heap_copy_tuple_as_raw_datum() should grow an assert checking
there are no external columns?
The commit messages could use a bit more explanation about motivation.
I'm don't like that after 0002 ExecEvalRow(), ExecEvalFieldStoreForm()
contain a nearly identical copy of the same code. And
make_tuple_from_row() also is similar. It seem that there should be a
heap_form_tuple() version doing this for us?
- 0003 through 0005 are the core of this patch set. I'd like to get
them into this release, but I think we're likely to run out of time.
Comments about 0003:
- why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
comparable to HIDE_TABLEAM?
- (you comment on this later): toast_get_compression_method() needing to
fetch some of the data to figure out the compression method is pretty
painful. Especially because it then goes and throws away that data!
- Adding all these indirect function calls via toast_compression[] just
for all of two builtin methods isn't fun either.
- I guess NO_LZ4_SUPPORT() is a macro so it shows the proper
file/function name?
- I wonder if adding compression to the equalTupleDesc() is really
necessary / won't cause problems (thinking of cases like the
equalTupleDesc() call in pg_proc.c).
- Is nodeModifyTable.c really the right place for the logic around
CompareCompressionMethodAndDecompress()? And is doing it in every
place that does "user initiated" inserts really the right way? Why
isn't this done on the tuptoasting level?
- CompareCompressionMethodAndDecompress() is pretty deeply
indented. Perhaps rewrite a few more of the conditions to be
continue;?
Comments about 0005:
- I'm personally not really convinced tracking the compression type in
pg_attribute the way you do is really worth it (. Especially given
that it's right now only about new rows anyway. Seems like it'd be
easier to just treat it as a default for new rows, and dispense with
all the logic around mismatching compression types etc?
The biggest thing that jumps out at me while looking at this with
fresh eyes is that the patch doesn't touch varatt_external.va_extsize
at all. In a varatt_external, we can't use the va_rawsize to indicate
the compression method, because there are no bits free, because the 2
bits not required to store the size are used to indicate what type of
varlena we've got.
Once you get to varatt_external, you could also just encode it via
vartag_external...
But, that means that the size of a varlena is limited to 1GB, so there
are 2 bits free in varatt_external.va_extsize, just like there are in
va_compressed.va_rawsize. We could store the same two bits in
varatt_external.va_extsize that we're storing in
va_compressed.va_rawsize aka va_tcinfo. That's a big deal, because
then toast_get_compression_method() doesn't have to call
toast_fetch_datum_slice() any more, which is a rather large savings.
If it's only impacting pg_column_compression() then whatever, but
that's not the case: we've got calls to
CompareCompressionMethodAndDecompress in places like intorel_receive()
and ExecModifyTable() that look pretty performance-critical.
Yea, I agree, that does seem problematic.
There's another, rather brute-force approach to this problem, too. We
could just decide that lz4 will only be used for external data, and
that there's no such thing as an inline-compressed lz4 varlena.
deotast_fetch_datum() would just notice that the value is lz4'd and
de-lz4 it before returning it, since a compressed lz4 datum is
impossible.
That seems fairly terrible.
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.
I mean, I would just define several of those places away by not caring
about tuples in a different compressino formation ending up in a
table...
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 4:28 AM Andres Freund <andres@anarazel.de> wrote:
Replying to some of the comments..
- Is nodeModifyTable.c really the right place for the logic around
CompareCompressionMethodAndDecompress()? And is doing it in every
place that does "user initiated" inserts really the right way? Why
isn't this done on the tuptoasting level?
I think if we do in tuptoasting level then it will be even costlier
because in nodeModifyTable.c at least many time we will get the
virtual tuple e.g. if a user is directly inserting the tuple but once
we go down to tuptoasting level by then we will always get the
HeapTuple and we will have to deform in every case where tupdesc has
any varlena because we don't have any flag in the tuple header to tell
us whether there are any compressed data or not. In the below
thread[1]/messages/by-id/CAFiTN-vcbfy5ScKVUp16c1N_wzP0RL6EkPBAg_Jm3eDK0ftO5Q@mail.gmail.com we have considered these 2 approaches and basically, in
unrelated paths like pg_bench, we did not see any performance
regression with any of those approaches.
[1]: /messages/by-id/CAFiTN-vcbfy5ScKVUp16c1N_wzP0RL6EkPBAg_Jm3eDK0ftO5Q@mail.gmail.com
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.I mean, I would just define several of those places away by not caring
about tuples in a different compressino formation ending up in a
table...
I am just wondering that why we don't need to process in case of
storage change, I mean if the target table has the attribute storage
as external and if there are some compressed data coming from the
source table then we will be inserting those compressed data as it is
in the target attribute without externalizing. Maybe it is done to
avoid such performance impacts? Well, we can do the same for the
compression also and just provide some mechanism to recompress maybe
in vacuum full/cluster.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
I'm a minor contributor now to a couple bits of this patch set, but I can
answer a couple of these points.
On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote:
Comments about 0003:
- why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
comparable to HIDE_TABLEAM?
That was my idea and implementation.
It's because until 3 weeks ago, the patchset supported a "plugable compression
API" like CREATE ACCESS METHOD, a suggestion from Alvaro to avoid making a new
table and everything involved just for a few rows). Now, the patch is limited
to lz4, and the "pluggable compression APIs" isn't included in the latest
patchsets.
Comments about 0005:
- I'm personally not really convinced tracking the compression type in
pg_attribute the way you do is really worth it (. Especially given
that it's right now only about new rows anyway. Seems like it'd be
easier to just treat it as a default for new rows, and dispense with
all the logic around mismatching compression types etc?
I made the half-serious suggestion to make it a per-relation relopt.
That would allow implementing pg_dump --no-toast-compression, to allow
restoring a dump from a server with LZ4 tables to a server --without-lz4.
Similar to --no-tablespaces.
That would also avoid adding a compression column in \d (which avoids the need
for HIDE_TOAST_COMPRESSION).
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.I mean, I would just define several of those places away by not caring
about tuples in a different compressino formation ending up in a
table...
If I understand you right, this is because it's desirable to allow 1) migrating
existing data from pglz to lz4; 2) also allow moving away from lz4, if need be.
--
Justin
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 11:18 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
I'm a minor contributor now to a couple bits of this patch set, but I can
answer a couple of these points.On Mon, Mar 15, 2021 at 03:58:35PM -0700, Andres Freund wrote:
Comments about 0003:
- why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
comparable to HIDE_TABLEAM?That was my idea and implementation.
It's because until 3 weeks ago, the patchset supported a "plugable compression
API" like CREATE ACCESS METHOD, a suggestion from Alvaro to avoid making a new
table and everything involved just for a few rows). Now, the patch is limited
to lz4, and the "pluggable compression APIs" isn't included in the latest
patchsets.
Yeah, but now also it makes sense to hide the compression method to
avoid unrelated regression changes. But I am okay if we think we want
to drop this?
Comments about 0005:
- I'm personally not really convinced tracking the compression type in
pg_attribute the way you do is really worth it (. Especially given
that it's right now only about new rows anyway. Seems like it'd be
easier to just treat it as a default for new rows, and dispense with
all the logic around mismatching compression types etc?I made the half-serious suggestion to make it a per-relation relopt.
That would allow implementing pg_dump --no-toast-compression, to allow
restoring a dump from a server with LZ4 tables to a server --without-lz4.
Similar to --no-tablespaces.
I am not sure how good an idea it is to support table-level options.
The attribute level option makes sense to me in case we want to
support different compression methods for different data types.
Currently, we have only pglz and lz4 but if we are not planning for
custom compression in the future then we can support 2 more built-in
compression methods so I still feel having an attribute level option
makes more sense.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 12:59 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Mar 15, 2021 at 8:14 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
In the attached patches I have changed this, ...
OK, so just looking over this patch series, here's what I think:
Regarding 0003:
The biggest thing that jumps out at me while looking at this with
fresh eyes is that the patch doesn't touch varatt_external.va_extsize
at all. In a varatt_external, we can't use the va_rawsize to indicate
the compression method, because there are no bits free, because the 2
bits not required to store the size are used to indicate what type of
varlena we've got. But, that means that the size of a varlena is
limited to 1GB, so there are 2 bits free in
varatt_external.va_extsize, just like there are in
va_compressed.va_rawsize. We could store the same two bits in
varatt_external.va_extsize that we're storing in
va_compressed.va_rawsize aka va_tcinfo. That's a big deal, because
then toast_get_compression_method() doesn't have to call
toast_fetch_datum_slice() any more, which is a rather large savings.
If it's only impacting pg_column_compression() then whatever, but
that's not the case: we've got calls to
CompareCompressionMethodAndDecompress in places like intorel_receive()
and ExecModifyTable() that look pretty performance-critical.
Yeah, right we can do this.
I got thinking about this after looking at ExecFilterJunk(). That
function is extremely simple precisely because all the work of
figuring out what should be done has been precomputed. All the smarts
are in cleanmap[], which is set up before we actually begin execution.
In a similar way, you can imagine creating some sort of object, let's
call it a CompressedAttributeFilter, that looks at the tupledesc
figures out from the tupledesc which columns we need to consider
recompressing and puts them in an array. Then you have a struct that
stores a pointer to the array, the number of elements in the array,
and the value of the last array element. You pass this struct to what
is now CompareCompressionMethodAndDecompress() and it can now run more
like what I described above.
Okay, I will work on this, basically, the main idea here is that
instead of identifying which attribute are varlena for every tuple we
already have the tupledesc so basically we can just prepare some sort
of map and in CompareCompressionMethodAndDecompress, only check those
attribute that whether they are
a) non-null b) compressed c) and, compressed with the different
compression method and if all are true then decompress. I agree this
will save the cost of processing the complete tuple descriptor for
every tuple.
It's possible to imagine doing even better. Imagine that for every
column we maintain an attcompression value and an
attpreservecompression value. The former indicates the compression
type for the column or '\0' if it cannot be compressed, and the latter
indicates whether any other compression type might be present. Then,
when we build the CompressedAttributeFilter object, we can exclude
varlena attributes if attpreservecompression is false and
attcompression is '\0' or matches the attcompression value for the
corresponding attribute in the table into which ExecModifyTable() or
intorel_receive() will be putting the tuple. This seems quite complex
in terms of bookkeeping, but it would allow us to elide basically all
of the per-tuple bookkeeping in a lot of common cases, such as UPDATE,
or an INSERT or CTAS into a table that's using the same compression
method as the source data.
I haven’t thought about this completely, but maybe it looks quite
complex. I mean maybe during the initplan time we will have to
process the complete tree to fetch the attcompression of the source
attribute because in ExecModifyTable or CTAS the source tuple could be
combinations of Join from multiple tables, function call or from any
source so keeping track of attcompression of each source attribute
seems very difficult. And, moreover, if we are always guaranteeing
that all the data in the table must be compressed as per the attribute
compression then we can not rely on the attcompression of the source
field. I mean we can alter the attribute compression without
rewriting.
There's another, rather brute-force approach to this problem, too. We
could just decide that lz4 will only be used for external data, and
that there's no such thing as an inline-compressed lz4 varlena.
deotast_fetch_datum() would just notice that the value is lz4'd and
de-lz4 it before returning it, since a compressed lz4 datum is
impossible.
This can cause multiple compression/decompression during insert no? I
mean attcompression is lz4 so first we try to compress with lz4
because it might be externalized and later we realize that compression
is reducing the size significantly and don't need to externalize then
should decompress and compress again using pglz? or now don't
compress and just externalize?
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.
I think we will have to do something about this, I have tested the
performance of just simple INSERT and CTAS for the table with 300
columns(script attached) and there is a HUGE impact on the performance
so something must be done.
INSERT TIME
Head: 17418.299 ms Patch: 20956.231 ms
CTAS TIME:
Head: 12837.872 ms Patch: 16775.739 ms
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 4:07 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
INSERT TIME
Head: 17418.299 ms Patch: 20956.231 msCTAS TIME:
Head: 12837.872 ms Patch: 16775.739 ms
On quick analysis with perf it appeared that the performance is
degrading because of deforming
- 16.19% 3.54% postgres postgres [.]
CompareCompressionMethodAndDecompress
- 12.65% CompareCompressionMethodAndDecompress
- 12.57% slot_getallattrs
- 12.56% slot_getsomeattrs
- 12.53% slot_getsomeattrs_int
- 12.50% tts_buffer_heap_getsomeattrs
slot_deform_heap_tuple
So I think in the case of direct insert it needs to deform because I
am calling CompareCompressionMethodAndDecompress after ExecCopySlot
and that is why we have to deform every time so maybe that can be
avoided by calling CompareCompressionMethodAndDecompress before
ExecCopySlot. But in the case of CTAS or INSERT INTO SELECT we can
not avoid deforming because we might get the formed tuple from the
source table. I put a temporary hack to keep the map of the varlena
attribute and use it across the tuple but it did not improve the
performance in this case because the main bottleneck is
slot_getallattrs. I think this should help where we don't have any
varlena, but first I need to test whether we can any performance
regression in those cases at all.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 15, 2021 at 6:58 PM Andres Freund <andres@anarazel.de> wrote:
I don't particularly like PG_RETURN_HEAPTUPLEHEADER_RAW(). What is "raw"
about it? It also seems to me like there needs to at least be a
sentence or two explaining when to use which of the functions.
It seemed like the natural name to me; we use "raw" elsewhere to mean
that fewer things are magically addressed on behalf of the caller,
e.g. HeapTupleHeaderGetRawXmin. I'm open to suggestions, however.
I think heap_copy_tuple_as_raw_datum() should grow an assert checking
there are no external columns?
Yeah, could be done.
I'm don't like that after 0002 ExecEvalRow(), ExecEvalFieldStoreForm()
contain a nearly identical copy of the same code. And
make_tuple_from_row() also is similar. It seem that there should be a
heap_form_tuple() version doing this for us?
I was worried about having either a performance impact or code
duplication. The actual plan where you could insert this organically
is in fill_val(), which is called from heap_fill_tuple(), which is
called from heap_form_tuple(). If you don't mind passing down 'int
flags' or similar to all those, and having additional branches to make
the behavior dependent on the flags, I'm cool with it. Or if you think
we should template-ize all those functions, that'd be another way to
go. But I was afraid I would get complaints about adding overhead to
hot code paths.
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.I mean, I would just define several of those places away by not caring
about tuples in a different compressino formation ending up in a
table...
That behavior feels unacceptable to me from a user expectations point
of view. I think there's an argument that if I update a tuple that
contains a compressed datum, and I don't update that particular
column, I think it would be OK to not recompress the column. But, if I
insert data into a table, I as a user would expect that the
compression settings for that column are going to be respected.
Deciding that's optional because we don't have a good way of making it
fast seems like a major cop-out, at least to me. I think from a user
perspective you don't expect INSERT INTO .. SELECT FROM to create a
different final state than a dump and reload, and that if we deviate
from that people are gonna be unhappy. I could be wrong; maybe it's
only me who would be unhappy.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 7:57 PM Robert Haas <robertmhaas@gmail.com> wrote:
That behavior feels unacceptable to me from a user expectations point
of view. I think there's an argument that if I update a tuple that
contains a compressed datum, and I don't update that particular
column, I think it would be OK to not recompress the column. But, if I
insert data into a table, I as a user would expect that the
compression settings for that column are going to be respected.
Deciding that's optional because we don't have a good way of making it
fast seems like a major cop-out, at least to me. I think from a user
perspective you don't expect INSERT INTO .. SELECT FROM to create a
different final state than a dump and reload, and that if we deviate
from that people are gonna be unhappy. I could be wrong; maybe it's
only me who would be unhappy.
If that is only the argument then it's possible today as well. I mean
you can INSERT INTO .. SELECT FROM where source attribute as
compressed data but the target attribute as external storage then also
we will move the compressed data as it is to the target table.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
On 2021-03-16 10:27:12 -0400, Robert Haas wrote:
I'm don't like that after 0002 ExecEvalRow(), ExecEvalFieldStoreForm()
contain a nearly identical copy of the same code. And
make_tuple_from_row() also is similar. It seem that there should be a
heap_form_tuple() version doing this for us?I was worried about having either a performance impact or code
duplication. The actual plan where you could insert this organically
is in fill_val(), which is called from heap_fill_tuple(), which is
called from heap_form_tuple().
Oh, I guess it would make sense to do it that way. However, I was just
thinking of doing the iteration over the tuples that ExecEvalRow() etc
do inside heap_form_flattened_tuple() (or whatever). That'd not be any
worse than what the patch is doing now, just less duplication, and an
easier path towards optimizing it if we notice that we need to?
If you don't mind passing down 'int flags' or similar to all those,
and having additional branches to make the behavior dependent on the
flags, I'm cool with it. Or if you think we should template-ize all
those functions, that'd be another way to go. But I was afraid I would
get complaints about adding overhead to hot code paths.
An option for fill_val() itself would probably be fine. It's already an
inline, and if it doesn't get inlined, we could force the compilers hand
with pg_attribute_always_inline.
The harder part would probably be to find a way to deal with the layers
above, without undue code duplication. I think it's not just fill_val()
that'd need to know, but also heap_compute_data_size(),
heap_fill_tuple() - both of which are externally visible (and iirc thus
not going to get inlined with many compiler options, due to symbol
interposition dangers). But we could have a
heap_compute_data_size_internal(bool flatten) that's called by
heap_compute_data_size(). And something similar for heap_form_tuple().
But that's complicated, so I'd just go with the iteration in a
heap_form_tuple() wrapper for now.
I'm open to being convinced that we don't need to do either of these
things, and that the cost of iterating over all varlenas in the tuple
is not so bad as to preclude doing things as you have them here. But,
I'm afraid it's going to be too expensive.I mean, I would just define several of those places away by not caring
about tuples in a different compressino formation ending up in a
table...That behavior feels unacceptable to me from a user expectations point
of view. I think there's an argument that if I update a tuple that
contains a compressed datum, and I don't update that particular
column, I think it would be OK to not recompress the column. But, if I
insert data into a table, I as a user would expect that the
compression settings for that column are going to be respected.
IDK. The user might also expect that INSERT .. SELECT is fast, instead
of doing expensive decompression + compression (with pglz the former can
be really slow). I think there's a good argument for having an explicit
"recompress" operation, but I'm not convincd that doing things
implicitly is good, especially if it causes complications in quite a few
places.
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 11:21 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
If that is only the argument then it's possible today as well. I mean
you can INSERT INTO .. SELECT FROM where source attribute as
compressed data but the target attribute as external storage then also
we will move the compressed data as it is to the target table.
Uggh. I don't like that behavior either, but I guess if it's the
long-established way things work then perhaps this is no worse.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Tue, Mar 16, 2021 at 2:54 PM Andres Freund <andres@anarazel.de> wrote:
Oh, I guess it would make sense to do it that way. However, I was just
thinking of doing the iteration over the tuples that ExecEvalRow() etc
do inside heap_form_flattened_tuple() (or whatever). That'd not be any
worse than what the patch is doing now, just less duplication, and an
easier path towards optimizing it if we notice that we need to?
It's a question of whether you copy the datum array. I don't think a
generic function can assume that it's OK to scribble on the input
array, or if it does, that'd better be very prominently mentioned in
the comments. And copying into a new array has its own costs. 0002 is
based on the theory that scribbling on the executor's array won't
cause any problem, which I *think* is true, but isn't correct in all
cases (e.g. if the input data is coming from a slot). If we pass a
flag down to fill_val() and friends then we don't end up having to
copy the arrays over so the problem goes away in that design.
The harder part would probably be to find a way to deal with the layers
above, without undue code duplication. I think it's not just fill_val()
that'd need to know, but also heap_compute_data_size(),
heap_fill_tuple() - both of which are externally visible (and iirc thus
not going to get inlined with many compiler options, due to symbol
interposition dangers). But we could have a
heap_compute_data_size_internal(bool flatten) that's called by
heap_compute_data_size(). And something similar for heap_form_tuple().
Hmm, yeah, that's not great. I guess there's nothing expensive we need
to repeat - I think anyway - because we should be able to get the
uncompressed size from the TOAST pointer itself. But the code would
have to know to do that, as you say.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 17, 2021 at 1:01 AM Robert Haas <robertmhaas@gmail.com> wrote:
Please find the updated version of the patch set.
Changes:
0001:
- Added comment for PG_RETURN_HEAPTUPLEHEADER_RAW().
- Added assert in heap_copy_tuple_as_raw_datum() that there should be
no external data in tuple
0002:
- Wrapper over heap_form_tuple and used in ExecEvalRow() and
ExecEvalFieldStoreForm()
0003:
1. Used 2 bits of varatt_external.va_extsize to store compression method
2. Removed logic of compressing.
3. Some other minor cleanup
0004: some minor cleanup
0005: Removed code of recompress from ATRewriteTuple, now only
recompress happens in Vacuum FULL and CLUSTER. So that there is at
least some way to recompress. Also, here we don't need to take the
overhead of deforming the tuple so it should not affect performance
also in unrelated cases. We will have to pay the penalty if we need
to decompress a lot of attributes.
0006: removed as suggested by Robert
0007: now 0006, only for CFbot
-- I still have the HIDE_TOAST_COMPRESSION code in the patch set do we
think we need to remove? I don't mind keeping it as it is because we
can avoid changes in unrelated regression tests.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v36-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v36-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From d8fc7144c39f9b3b9e26af83300938952f88957e Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v36 2/6] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/access/common/heaptuple.c | 29 +++++++++++++++++++++++++++++
src/backend/executor/execExprInterp.c | 16 ++++++++--------
src/backend/utils/adt/jsonfuncs.c | 9 +++++++--
src/include/access/htup_details.h | 2 ++
src/pl/plpgsql/src/pl_exec.c | 19 ++++++++++++++-----
5 files changed, 60 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 6061376..c4b5cd7 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -61,6 +61,7 @@
#include "access/sysattr.h"
#include "access/tupdesc_details.h"
#include "executor/tuptable.h"
+#include "fmgr.h"
#include "utils/expandeddatum.h"
@@ -1114,6 +1115,34 @@ heap_form_tuple(TupleDesc tupleDescriptor,
return tuple;
}
+
+/*
+ * heap_form_flattened_tuple
+ * wrapper over heap_form_tuple which flatten any external toast pointers
+ * before forming the tuple.
+ *
+ * The result is allocated in the current memory context.
+ */
+HeapTuple
+heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values,
+ bool *isnull)
+{
+ /* detoast any external data before forming the tuple */
+ for (int i = 0; i < tupleDescriptor->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupleDescriptor, i);
+
+ if (isnull[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ continue;
+
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
+ }
+
+ return heap_form_tuple(tupleDescriptor, values, isnull);
+}
+
/*
* heap_modify_tuple
* form a new tuple from an old tuple and a set of replacement values.
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index e47ab1e..bff6c2b 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2841,11 +2841,11 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
HeapTuple tuple;
/* build tuple from evaluated field values */
- tuple = heap_form_tuple(op->d.row.tupdesc,
- op->d.row.elemvalues,
- op->d.row.elemnulls);
+ tuple = heap_form_flattened_tuple(op->d.row.tupdesc,
+ op->d.row.elemvalues,
+ op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3086,11 +3086,11 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
HeapTuple tuple;
/* argdesc should already be valid from the DeForm step */
- tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
- op->d.fieldstore.values,
- op->d.fieldstore.nulls);
+ tuple = heap_form_flattened_tuple(*op->d.fieldstore.argdesc,
+ op->d.fieldstore.values,
+ op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..dc3f9d4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,11 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ /* detoast any external data before forming the tuple */
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3770,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7ac3c23..be6641b 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -793,6 +793,8 @@ extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
+extern HeapTuple heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
TupleDesc tupleDesc,
Datum *replValues,
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..d4fcb70 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -404,7 +404,8 @@ static void exec_move_row_from_fields(PLpgSQL_execstate *estate,
static bool compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc);
static HeapTuple make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc);
+ TupleDesc tupdesc,
+ bool flatten);
static TupleDesc deconstruct_composite_datum(Datum value,
HeapTupleData *tmptup);
static void exec_move_row_from_datum(PLpgSQL_execstate *estate,
@@ -3418,7 +3419,7 @@ exec_stmt_return_next(PLpgSQL_execstate *estate,
/* Use eval_mcontext for tuple conversion work */
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tuple = make_tuple_from_row(estate, row, tupdesc);
+ tuple = make_tuple_from_row(estate, row, tupdesc, false);
if (tuple == NULL) /* should not happen */
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
@@ -5321,12 +5322,12 @@ exec_eval_datum(PLpgSQL_execstate *estate,
/* Make sure we have a valid type/typmod setting */
BlessTupleDesc(row->rowtupdesc);
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tup = make_tuple_from_row(estate, row, row->rowtupdesc);
+ tup = make_tuple_from_row(estate, row, row->rowtupdesc, true);
if (tup == NULL) /* should not happen */
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7265,12 +7266,15 @@ compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc)
*
* The result tuple is freshly palloc'd in caller's context. Some junk
* may be left behind in eval_mcontext, too.
+ *
+ * flatten - if this is passed true then any external data will be flattened.
* ----------
*/
static HeapTuple
make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc)
+ TupleDesc tupdesc,
+ bool flatten)
{
int natts = tupdesc->natts;
HeapTuple tuple;
@@ -7300,6 +7304,11 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+
+ /* detoast any external data if the caller has asked to do so */
+ if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v36-0004-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v36-0004-Add-default_toast_compression-GUC.patchDownload
From 2507d02a8b8d75ae33b8f51c4312522d4f90cac7 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 17 Mar 2021 15:23:33 +0530
Subject: [PATCH v36 4/6] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 46 +++++++++++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++---
src/backend/utils/misc/guc.c | 12 +++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 22 +++++++++++--
src/test/regress/expected/compression.out | 16 ++++++++++
src/test/regress/expected/compression_1.out | 19 +++++++++++
src/test/regress/sql/compression.sql | 8 +++++
10 files changed, 128 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index f8d7fdb..d337b7c 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -55,6 +55,9 @@ const CompressionRoutine toast_compression[] =
}
};
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -308,3 +311,46 @@ GetCompressionRoutines(char method)
{
return &toast_compression[CompressionMethodToId(method)];
}
+
+/* check_hook: validate new default_toast_compression */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index a66d111..362a371 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -668,7 +668,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index af0c0aa..9be0841 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -733,7 +733,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 7d160d5..cf90f40 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 59eb42f..e1dd4cf 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3917,6 +3918,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index ee06528..82af1bf 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -658,6 +658,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index d87212c..c61e9f4 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,6 +13,14 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
+#include "utils/guc.h"
+
+/* GUCs */
+extern char *default_toast_compression;
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
/*
* Built-in compression methods. pg_attribute will store this in the
* attcompression column.
@@ -34,8 +42,6 @@ typedef enum ToastCompressionId
TOAST_LZ4_COMPRESSION_ID = 1
} ToastCompressionId;
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -65,6 +71,8 @@ typedef struct CompressionRoutine
extern char CompressionNameToMethod(char *compression);
extern const CompressionRoutine *GetCompressionRoutines(char method);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
/*
* CompressionMethodToId - Convert compression method to compression id.
@@ -113,5 +121,15 @@ GetCompressionMethodName(char method)
return GetCompressionRoutines(method)->cmname;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3b01933..4783952 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -212,6 +212,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3fb8307..136e750 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -208,6 +208,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 168825d..1834ba7 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -92,6 +92,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v36-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchtext/x-patch; charset=US-ASCII; name=v36-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchDownload
From 4788e0fcdef064dc7f4b782a92665256cf86d73d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH v36 1/6] Get datum from tuple which doesn't contain external
data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well. This is
the base patch for the optimization in the next patch therein we can try
to optimize the caller of the functions where we expect external varlena
by flattening any external toast pointer before forming the tuple.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 20 ++++++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 5 +++++
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 132 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..43bb2ab 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..6061376 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,24 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
+
+ /* the tuple should not contains any external TOAST pointers */
+ Assert(!HeapTupleHasExternal(tuple));
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 80d2d20..67385e3 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index d8c5bf6..53f84de 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..e47ab1e 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to
+ * flatten any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index e5f8a06..f973aea 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1420,5 +1420,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 62bff52..5936c39 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1843,7 +1843,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[4] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2259,7 +2260,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..079a5af 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4bcf705..59eb42f 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9798,7 +9798,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..cb70c05 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,11 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+/*
+ * same as PG_RETURN_HEAPTUPLEHEADER but the input tuple should not contain any
+ * external varlena.
+ */
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..8ba7ae2 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v36-0003-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v36-0003-Built-in-compression-method.patchDownload
From 547e8fb8fcab4bfc58ab5a50260f9044de5155b5 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v36 3/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Justin Pryzby, Tomas Vondra and Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 170 +++++++++++++
configure.ac | 20 ++
contrib/amcheck/verify_heapam.c | 2 +-
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 81 ++++---
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 310 ++++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 63 +++--
src/backend/access/common/tupdesc.c | 8 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 +++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 +-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/varlena.c | 41 ++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 18 +-
src/include/access/toast_compression.h | 117 +++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 23 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 46 +++-
src/test/regress/expected/compression.out | 248 +++++++++++++++++++
src/test/regress/expected/compression_1.out | 240 ++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 102 ++++++++
src/tools/msvc/Solution.pm | 1 +
src/tools/pgindent/typedefs.list | 1 +
54 files changed, 1748 insertions(+), 105 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index ce9ea36..a0ae4b4 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,9 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -891,6 +895,8 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1596,6 +1603,8 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
@@ -8564,6 +8573,137 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -13322,6 +13462,36 @@ fi
fi
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index f54f65f..aa57091 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,21 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -1406,6 +1421,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index 49f5ca0..9b17d73 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -1069,7 +1069,7 @@ check_tuple_attribute(HeapCheckContext *ctx)
*/
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ctx->attrsize = toast_pointer.va_extsize;
+ ctx->attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
ctx->endchunk = (ctx->attrsize - 1) / TOAST_MAX_CHUNK_SIZE;
ctx->totalchunks = ctx->endchunk + 1;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index b1de6d0..5fdc80f 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,16 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index 3b2b227..7c8fc77 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..ec235b6 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -16,6 +16,7 @@
#include "access/detoast.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/toast_internals.h"
#include "common/int.h"
#include "common/pg_lzcompress.h"
@@ -247,7 +248,7 @@ detoast_attr_slice(struct varlena *attr,
* of a given length (after decompression).
*/
max_size = pglz_maximum_compressed_size(slicelimit,
- toast_pointer.va_extsize);
+ VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/*
* Fetch enough compressed slices (compressed marker will get set
@@ -347,7 +348,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
result = (struct varlena *) palloc(attrsize + VARHDRSZ);
@@ -408,7 +409,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) || 0 == sliceoffset);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
if (sliceoffset >= attrsize)
{
@@ -457,6 +458,37 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
}
/* ----------
+ * toast_get_compression_method
+ *
+ * Returns compression method for the compressed varlena. If it is not
+ * compressed then returns InvalidCompressionMethod.
+ */
+char
+toast_get_compression_method(struct varlena *attr)
+{
+ ToastCompressionId cmid;
+
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ /* fast path for non-compressed external datums */
+ if (!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ return InvalidCompressionMethod;
+ else
+ cmid = VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer);
+ }
+ else if (VARATT_IS_COMPRESSED(attr))
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ else
+ return InvalidCompressionMethod;
+
+ return CompressionIdToMethod(cmid);
+}
+
+/* ----------
* toast_decompress_datum -
*
* Decompress a compressed version of a varlena datum
@@ -464,21 +496,19 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ char cmethod;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in
+ * the toast header.
+ */
+ cmethod = CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+ cmroutine = GetCompressionRoutines(cmethod);
- return result;
+ return cmroutine->datum_decompress(attr);
}
@@ -492,22 +522,19 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ char cmethod;
+ const CompressionRoutine *cmroutine;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
+ /*
+ * Get compression handler routines, using the compression id stored in
+ * the toast header.
+ */
+ cmethod = CompressionIdToMethod(TOAST_COMPRESS_METHOD(attr));
+ cmroutine = GetCompressionRoutines(cmethod);
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ return cmroutine->datum_decompress_slice(attr, slicelength);
}
/* ----------
@@ -589,7 +616,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..f8d7fdb
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,310 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef USE_LZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+static struct varlena *pglz_cmcompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress(const struct varlena *value);
+static struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+static struct varlena *lz4_cmcompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress(const struct varlena *value);
+static struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* handler routines for pglz and lz4 built-in compression methods */
+const CompressionRoutine toast_compression[] =
+{
+ {
+ .cmname = "pglz",
+ .datum_compress = pglz_cmcompress,
+ .datum_decompress = pglz_cmdecompress,
+ .datum_decompress_slice = pglz_cmdecompress_slice
+ },
+ {
+ .cmname = "lz4",
+ .datum_compress = lz4_cmcompress,
+ .datum_decompress = lz4_cmdecompress,
+ .datum_decompress_slice = lz4_cmdecompress_slice
+ }
+};
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+static struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+static struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* slice decompression not supported prior to 1.8.3 */
+ if (LZ4_versionNumber() < 10803)
+ return lz4_cmdecompress(value);
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(toast_compression[TOAST_PGLZ_COMPRESSION_ID].cmname,
+ compression) == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(toast_compression[TOAST_LZ4_COMPRESSION_ID].cmname,
+ compression) == 0)
+ {
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetCompressionRoutines - Get compression handler routines
+ */
+const CompressionRoutine *
+GetCompressionRoutines(char method)
+{
+ return &toast_compression[CompressionMethodToId(method)];
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..16508c3 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,42 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ const CompressionRoutine *cmroutine = NULL;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
- /*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
- */
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ Assert(CompressionMethodIsValid(cmethod));
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ /* get the handler routines for the compression method */
+ cmroutine = GetCompressionRoutines(cmethod);
+
+ /* call the actual compression function */
+ tmp = cmroutine->datum_compress((const struct varlena *) value);
+ if (!tmp)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ CompressionMethodToId(cmethod));
return PointerGetDatum(tmp);
}
else
@@ -164,7 +160,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -172,7 +168,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+
+ /* set external size and compression method */
+ VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, data_todo,
+ VARCOMPRESS_4B_C(dval));
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -181,7 +180,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..a66d111 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -470,6 +471,8 @@ equalTupleDescs(TupleDesc tupdesc1, TupleDesc tupdesc2)
return false;
if (attr1->attcollation != attr2->attcollation)
return false;
+ if (attr1->attcompression != attr2->attcompression)
+ return false;
/* attacl, attoptions and attfdwoptions are not even present... */
}
@@ -664,6 +667,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 6f615e6..af0c0aa 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -731,6 +732,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d0ec44b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..933a073 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 559fa1d..7d160d5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression =
+ GetCompressionMethodName(attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index aaba1ec..a8edcf7 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2987,6 +2987,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 8fc432b..641aa43 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2874,6 +2874,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index 75266ca..681151e 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1082,6 +1083,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 91600ac..c291b05 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4641,7 +4641,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..187e55b 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5300,6 +5301,46 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ Datum value = PG_GETARG_DATUM(0);
+ int typlen;
+ char method;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ method = toast_get_compression_method(
+ (struct varlena *) DatumGetPointer(value));
+
+ if (!CompressionMethodIsValid(method))
+ PG_RETURN_NULL();
+ else
+ PG_RETURN_TEXT_P(cstring_to_text(GetCompressionMethodName(method)));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..cef9bbd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..453f946 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index e44120b..a43a8f5 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -375,6 +375,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..8b6b479 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -13,16 +13,6 @@
#define DETOAST_H
/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing extsize (the actual length of the external data) to rawsize
- * (the original uncompressed datum's size). The latter includes VARHDRSZ
- * overhead, the former doesn't. We never use compression unless it actually
- * saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
-
-/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
* into a local "struct varatt_external" toast pointer. This should be
* just a memcpy, but some versions of gcc seem to produce broken code
@@ -89,4 +79,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_method -
+ *
+ * Return the compression method from the compressed value
+ * ----------
+ */
+extern char toast_get_compression_method(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..d87212c
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,117 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define TOAST_PGLZ_COMPRESSION 'p'
+#define TOAST_LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum ToastCompressionId
+{
+ TOAST_PGLZ_COMPRESSION_ID = 0,
+ TOAST_LZ4_COMPRESSION_ID = 1
+} ToastCompressionId;
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/* compression handler routines */
+typedef struct varlena *(*cmcompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_function) (const struct varlena *value);
+typedef struct varlena *(*cmdecompress_slice_function)
+ (const struct varlena *value, int32 slicelength);
+
+/*
+ * API struct for a compression routines.
+ *
+ * 'cmname' - name of the compression method
+ * 'datum_compress' - varlena compression function.
+ * 'datum_decompress' - varlena decompression function.
+ * 'datum_decompress_slice' - varlena slice decompression functions.
+ */
+typedef struct CompressionRoutine
+{
+ char cmname[64];
+ cmcompress_function datum_compress;
+ cmdecompress_function datum_decompress;
+ cmdecompress_slice_function datum_decompress_slice;
+} CompressionRoutine;
+
+extern char CompressionNameToMethod(char *compression);
+extern const CompressionRoutine *GetCompressionRoutines(char method);
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop ToastCompressionId.
+ */
+static inline ToastCompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ return TOAST_PGLZ_COMPRESSION_ID;
+ case TOAST_LZ4_COMPRESSION:
+ return TOAST_LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop ToastCompressionId.
+ */
+static inline char
+CompressionIdToMethod(ToastCompressionId cmid)
+{
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return TOAST_PGLZ_COMPRESSION;
+ case TOAST_LZ4_COMPRESSION_ID:
+ return TOAST_LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char *
+GetCompressionMethodName(char method)
+{
+ return GetCompressionRoutines(method)->cmname;
+}
+
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..b4d0684 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,26 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) \
+ (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = \
+ ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 59d2b71..faa5456 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7098,6 +7098,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 04dc330..3b72038 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -896,6 +896,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to build with LZ4 support (--with-lz4) */
+#undef USE_LZ4
+
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..9dfa85b9 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * compression method */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,7 +146,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +276,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
@@ -323,6 +334,35 @@ typedef struct
(VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
#define VARATT_IS_EXTERNAL_NON_EXPANDED(PTR) \
(VARATT_IS_EXTERNAL(PTR) && !VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
+
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and compression method if external data is compressed.
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & VARLENA_RAWSIZE_MASK)
+
+#define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, len, cm) \
+ do { \
+ Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_pointer).va_extinfo = (len) | (cm) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
+
+#define VARATT_EXTERNAL_GET_COMPRESSION(PTR) \
+ ((toast_pointer).va_extinfo >> VARLENA_RAWSIZE_BITS)
+
+/*
+ * Testing whether an externally-stored value is compressed now requires
+ * comparing size stored in va_extinfo (the actual length of the external data)
+ * to rawsize (the original uncompressed datum's size). The latter includes
+ * VARHDRSZ overhead, the former doesn't. We never use compression unless it
+ * actually saves space, so we expect either equality or less-than.
+ */
+#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
+ (toast_pointer).va_rawsize - VARHDRSZ)
+
#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3b01933
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,248 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..3fb8307
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,240 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT * FROM cmdata2;
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index c77b0d7..b3605db 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -114,7 +114,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 0264a97..15ec754 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -200,6 +200,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..168825d
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,102 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT * FROM cmdata2;
+SELECT pg_column_compression(f1) FROM cmdata1;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 69b0859..95abad2 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -484,6 +484,7 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
+ USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
diff --git a/src/tools/pgindent/typedefs.list b/src/tools/pgindent/typedefs.list
index 8bd95ae..0fc7017 100644
--- a/src/tools/pgindent/typedefs.list
+++ b/src/tools/pgindent/typedefs.list
@@ -395,6 +395,7 @@ CompositeIOData
CompositeTypeStmt
CompoundAffixFlag
CompressionAlgorithm
+CompressionRoutine
CompressorState
ComputeXidHorizonsResult
ConditionVariable
--
1.8.3.1
v36-0005-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v36-0005-Alter-table-set-compression.patchDownload
From cd3eac2e9a24ec2ffcf7238e2320eb9640591774 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 17 Mar 2021 13:53:08 +0530
Subject: [PATCH v36 5/6] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/access/heap/heapam_handler.c | 24 ++++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 48 ++++++-
src/test/regress/sql/compression.sql | 20 +++
9 files changed, 320 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index c25ef5a..0bd0c1a 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0..b74ba00 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,28 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+
+ /*
+ * Since we are rewriting the table, use this opportunity to
+ * recompress any compressed attribute with current compression method
+ * of the attribute. Basically, if the compression method of the
+ * compressed varlena is not same as current compression method of the
+ * attribute then decompress it so that if it need to be compressed
+ * then it will be compressed with the current compression method of
+ * the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmethod = toast_get_compression_method(new_value);
+
+ if (IsValidCompression(cmethod) &&
+ TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index cf90f40..20c8126 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7773,6 +7781,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15046,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index 9f0208a..07ba55f 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2115,7 +2115,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 4783952..8bf32c3 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -228,12 +228,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 136e750..2292870 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,12 +227,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 1834ba7..12cbfe8 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -100,6 +100,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v36-0006-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v36-0006-default-to-with-lz4.patchDownload
From 0e28d37b760bceadb0f94d64dc3a3cc85a2bd044 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v36 6/6] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index a0ae4b4..fc57fef 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index aa57091..0d75aa4 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 17, 2021 at 7:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
0002:
- Wrapper over heap_form_tuple and used in ExecEvalRow() and
ExecEvalFieldStoreForm()
Instead of having heap_form_flattened_tuple(), how about
heap_flatten_values(tupleDesc, values, isnull) that is documented to
modify the values array? Then instead of replacing the
heap_form_tuple() calls with a call to heap_form_flattened_tuple(),
you just insert a call to heap_flatten_values() before the call to
heap_form_tuple(). I think that might be easier for people looking at
this code in the future to understand what's happening.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
On 2021-03-17 13:31:14 -0400, Robert Haas wrote:
On Wed, Mar 17, 2021 at 7:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
0002:
- Wrapper over heap_form_tuple and used in ExecEvalRow() and
ExecEvalFieldStoreForm()Instead of having heap_form_flattened_tuple(), how about
heap_flatten_values(tupleDesc, values, isnull) that is documented to
modify the values array? Then instead of replacing the
heap_form_tuple() calls with a call to heap_form_flattened_tuple(),
you just insert a call to heap_flatten_values() before the call to
heap_form_tuple(). I think that might be easier for people looking at
this code in the future to understand what's happening.
OTOH heap_form_flattened_tuple() has the advantage that we can optimize
it further (e.g. to do the conversion to flattened values in fill_val())
without changing the outside API.
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On Wed, Mar 17, 2021 at 2:17 PM Andres Freund <andres@anarazel.de> wrote:
OTOH heap_form_flattened_tuple() has the advantage that we can optimize
it further (e.g. to do the conversion to flattened values in fill_val())
without changing the outside API.
Well, in my view, that does change the outside API, because either the
input values[] array is going to get scribbled on, or it's not. We
should either decide we're not OK with it and just do the fill_val()
thing now, or we should decide that we are and not worry about doing
the fill_val() thing later. IMHO, anyway.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
).On Mon, Mar 15, 2021 at 6:58 PM Andres Freund <andres@anarazel.de> wrote:
- Adding all these indirect function calls via toast_compression[] just
for all of two builtin methods isn't fun either.
Yeah, it feels like this has too many layers of indirection now. Like,
toast_decompress_datum() first gets TOAST_COMPRESS_METHOD(attr). Then
it calls CompressionIdToMethod to convert one constant (like
TOAST_PGLZ_COMPRESSION_ID) to another constant with a slightly
different name (like TOAST_PGLZ_COMPRESSION). Then it calls
GetCompressionRoutines() to get hold of the function pointers. Then it
does an indirect functional call. That seemed like a pretty reasonable
idea when we were trying to support arbitrary compression AMs without
overly privileging the stuff that was built into core, but if we're
just doing stuff that's built into core, then we could just switch
(TOAST_COMPRESS_METHOD(attr)) and call the correct function. In fact,
we could even move the stuff from toast_compression.c into detoast.c,
which would allow the compiler to optimize better (e.g. by inlining,
if it wants).
The same applies to toast_decompress_datum_slice().
There's a similar issue in toast_get_compression_method() and the only
caller, pg_column_compression(). Here the multiple mapping layers and
the indirect function call are split across those two functions rather
than all in the same one, but here again one could presumably find a
place to just switch on TOAST_COMPRESS_METHOD(attr) or
VARATT_EXTERNAL_GET_COMPRESSION(attr) and return "pglz" or "lz4"
directly.
In toast_compress_datum(), I think we could have a switch that invokes
the appropriate compressor based on cmethod and sets a variable to the
value to be passed as the final argument of
TOAST_COMPRESS_SET_SIZE_AND_METHOD().
Likewise, I suppose CompressionNameToMethod could at least be
simplified to use constant strings rather than stuff like
toast_compression[TOAST_PGLZ_COMPRESSION_ID].cmname.
- why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
comparable to HIDE_TABLEAM?
Andres, what do you mean by this exactly? It's exactly the same issue:
without this, if you change the default compression method, every test
that uses \d+ breaks. If you want to be able to run the whole test
suite with either compression method and get the same results, you
need this. Now, maybe you don't, because perhaps that doesn't seem so
important with compression methods as with table AMs. I think that's a
defensible position. But, it is at the underlying level, the same
thing.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
On 2021-03-17 16:01:58 -0400, Robert Haas wrote:
- why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
comparable to HIDE_TABLEAM?Andres, what do you mean by this exactly? It's exactly the same issue:
without this, if you change the default compression method, every test
that uses \d+ breaks. If you want to be able to run the whole test
suite with either compression method and get the same results, you
need this. Now, maybe you don't, because perhaps that doesn't seem so
important with compression methods as with table AMs.
I think that latter part is why I wasn't sure such an option is
warranted. Given it's a builtin feature, I didn't really forsee a need
to be able to run all the tests with a different compression method. And
it looked a like it could just have been copied from the tableam logic,
without a clear need. But if it's useful, then ...
Greetings,
Andres Freund
Re: [HACKERS] Custom compression methods
On Wed, Mar 17, 2021 at 02:50:34PM -0700, Andres Freund wrote:
On 2021-03-17 16:01:58 -0400, Robert Haas wrote:
- why is HIDE_TOAST_COMPRESSION useful? Doesn't quite seem to be
comparable to HIDE_TABLEAM?Andres, what do you mean by this exactly? It's exactly the same issue:
without this, if you change the default compression method, every test
that uses \d+ breaks. If you want to be able to run the whole test
suite with either compression method and get the same results, you
need this. Now, maybe you don't, because perhaps that doesn't seem so
important with compression methods as with table AMs.
Arguably, it's more important, since it affects every column in \d+, not just a
"footer" line.
I think that latter part is why I wasn't sure such an option is
warranted. Given it's a builtin feature, I didn't really forsee a need
to be able to run all the tests with a different compression method. And
it looked a like it could just have been copied from the tableam logic,
without a clear need. But if it's useful, then ...
This was one of my suggestions and contributions.
I copied it from tableam specifically, not incidentally.
/messages/by-id/20210214184940.GL1793@telsasoft.com
--
Justin
Re: [HACKERS] Custom compression methods
On Thu, Mar 18, 2021 at 1:32 AM Robert Haas <robertmhaas@gmail.com> wrote:
).On Mon, Mar 15, 2021 at 6:58 PM Andres Freund <andres@anarazel.de> wrote:
- Adding all these indirect function calls via toast_compression[] just
for all of two builtin methods isn't fun either.Yeah, it feels like this has too many layers of indirection now. Like,
toast_decompress_datum() first gets TOAST_COMPRESS_METHOD(attr). Then
it calls CompressionIdToMethod to convert one constant (like
TOAST_PGLZ_COMPRESSION_ID) to another constant with a slightly
different name (like TOAST_PGLZ_COMPRESSION). Then it calls
GetCompressionRoutines() to get hold of the function pointers. Then it
does an indirect functional call. That seemed like a pretty reasonable
idea when we were trying to support arbitrary compression AMs without
overly privileging the stuff that was built into core, but if we're
just doing stuff that's built into core, then we could just switch
(TOAST_COMPRESS_METHOD(attr)) and call the correct function. In fact,
we could even move the stuff from toast_compression.c into detoast.c,
which would allow the compiler to optimize better (e.g. by inlining,
if it wants).The same applies to toast_decompress_datum_slice().
Changed this, but I have still kept the functions in
toast_compression.c. I think keeping compression related
functionality in a separate file looks much cleaner. Please have a
look and let me know that whether you still feel we should move it ti
detoast.c. If the reason is that we can inline, then I feel we are
already paying cost of compression/decompression and compare to that
in lining a function will not make much difference.
There's a similar issue in toast_get_compression_method() and the only
caller, pg_column_compression(). Here the multiple mapping layers and
the indirect function call are split across those two functions rather
than all in the same one, but here again one could presumably find a
place to just switch on TOAST_COMPRESS_METHOD(attr) or
VARATT_EXTERNAL_GET_COMPRESSION(attr) and return "pglz" or "lz4"
directly.
I have simplified that, only one level of function call from
pg_column_compression, I have kept a toast_get_compression_id
function because in later patch 0005, we will be using that for
getting the compression id from the compressed data.
In toast_compress_datum(), I think we could have a switch that invokes
the appropriate compressor based on cmethod and sets a variable to the
value to be passed as the final argument of
TOAST_COMPRESS_SET_SIZE_AND_METHOD().
Done
Likewise, I suppose CompressionNameToMethod could at least be
simplified to use constant strings rather than stuff like
toast_compression[TOAST_PGLZ_COMPRESSION_ID].cmname.
Done
Other changes:
- As suggested by Andres, remove compression method comparision from
eualTupleDesc, because it is not required now.
- I found one problem in existing patch, the problem was in
detoast_attr_slice, if externally stored data is compressed then we
compute max possible compressed size to fetch based on the slice
length, for that we were using pglz_maximum_compressed_size, which is
not correct for lz4. For lz4, I think we need to fetch the complete
compressed data. We might think that for lz4 we might compute lie
Min(LZ4_compressBound(slicelength, total_compressed_size); But IMHO,
we can not do that and the reason is same that why we should not use
PGLZ_MAX_OUTPUT for pglz (explained in the comment atop
pglz_maximum_compressed_size).
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v37-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchtext/x-patch; charset=US-ASCII; name=v37-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchDownload
From 58bb2cba9b7d2340cebd7f08461080b579cf2532 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH v37 1/6] Get datum from tuple which doesn't contain external
data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well. This is
the base patch for the optimization in the next patch therein we can try
to optimize the caller of the functions where we expect external varlena
by flattening any external toast pointer before forming the tuple.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 20 ++++++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 5 +++++
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 132 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..43bb2ab 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..6061376 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,24 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
+
+ /* the tuple should not contains any external TOAST pointers */
+ Assert(!HeapTupleHasExternal(tuple));
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6023e7c..3f48597 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index f363a4c..f60d6f1 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..e47ab1e 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to
+ * flatten any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 8532296..f5425a3 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1423,5 +1423,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5102227..b93ad73 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1856,7 +1856,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[8] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2272,7 +2273,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..079a5af 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index b263e34..6b2d9d6 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9805,7 +9805,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..cb70c05 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,11 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+/*
+ * same as PG_RETURN_HEAPTUPLEHEADER but the input tuple should not contain any
+ * external varlena.
+ */
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..8ba7ae2 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v37-0003-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v37-0003-Built-in-compression-method.patchDownload
From b55fd3c679bc5ef0af3b6c0a9f8754ea3f3867d1 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v37 3/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Justin Pryzby, Tomas Vondra and Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 170 +++++++++++++++
configure.ac | 20 ++
contrib/amcheck/verify_heapam.c | 2 +-
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 99 ++++++---
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 244 ++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 75 ++++---
src/backend/access/common/tupdesc.c | 6 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 ++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/varlena.c | 49 +++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 ++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 18 +-
src/include/access/toast_compression.h | 141 +++++++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 23 ++-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 46 ++++-
src/test/regress/expected/compression.out | 261 ++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 250 +++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 104 ++++++++++
src/tools/msvc/Solution.pm | 1 +
53 files changed, 1763 insertions(+), 108 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index 3fd4cec..8176e99 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,9 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -891,6 +895,8 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1596,6 +1603,8 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
@@ -8564,6 +8573,137 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -13381,6 +13521,36 @@ fi
fi
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 2f1585a..54efbb2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,21 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -1410,6 +1425,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index e614c12..6f972e6 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -1069,7 +1069,7 @@ check_tuple_attribute(HeapCheckContext *ctx)
*/
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ctx->attrsize = toast_pointer.va_extsize;
+ ctx->attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
ctx->endchunk = (ctx->attrsize - 1) / TOAST_MAX_CHUNK_SIZE;
ctx->totalchunks = ctx->endchunk + 1;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5c9f4af..de5f02a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,16 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index ff1b642..a731239 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..fc930b7 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -240,14 +240,15 @@ detoast_attr_slice(struct varlena *attr,
*/
if (slicelimit >= 0)
{
- int32 max_size;
+ int32 max_size = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
/*
* Determine maximum amount of compressed data needed for a prefix
* of a given length (after decompression).
*/
- max_size = pglz_maximum_compressed_size(slicelimit,
- toast_pointer.va_extsize);
+ if (VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer) ==
+ TOAST_PGLZ_COMPRESSION_ID)
+ max_size = pglz_maximum_compressed_size(slicelimit, max_size);
/*
* Fetch enough compressed slices (compressed marker will get set
@@ -347,7 +348,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
result = (struct varlena *) palloc(attrsize + VARHDRSZ);
@@ -408,7 +409,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) || 0 == sliceoffset);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
if (sliceoffset >= attrsize)
{
@@ -459,26 +460,60 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
/* ----------
* toast_decompress_datum -
*
+ * Returns compression method id stored in the compressed data. Otherwise,
+ * returns TOAST_INVALID_COMPRESSION_ID for the uncompressed data.
+ */
+ToastCompressionId
+toast_get_compression_id(struct varlena *attr)
+{
+ ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID;
+
+ /*
+ * Get the compression method id stored in compressed data. If it is
+ * stored externally then fetch it from the external toast pointer
+ * otherwise for inline compressed data fetch it from the compression
+ * header.
+ */
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ cmid = VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer);
+ }
+ else if (VARATT_IS_COMPRESSED(attr))
+ cmid = VARCOMPRESS_4B_C(attr);
+
+ return cmid;
+}
+/* ----------
+ * toast_decompress_datum -
+ *
* Decompress a compressed version of a varlena datum
*/
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ ToastCompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
-
- return result;
+ /*
+ * Fetch the compression method id stored in the compression header and
+ * decompress the data using the respective decompression routine.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return pglz_cmdecompress(attr);
+ case TOAST_LZ4_COMPRESSION_ID:
+ return lz4_cmdecompress(attr);
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
}
@@ -492,22 +527,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ ToastCompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * Fetch the compression method id stored in the compression header and
+ * decompress the data slice using the respective decompression routine.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return pglz_cmdecompress_slice(attr, slicelength);
+ case TOAST_LZ4_COMPRESSION_ID:
+ return lz4_cmdecompress_slice(attr, slicelength);
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
}
/* ----------
@@ -589,7 +626,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..22a8d54
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,244 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef USE_LZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* slice decompression not supported prior to 1.8.3 */
+ if (LZ4_versionNumber() < 10803)
+ return lz4_cmdecompress(value);
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..c84521e 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,48 +44,56 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
+ Assert(CompressionMethodIsValid(cmethod));
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
/*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
+ * Call respective compression routine for the compression method and also
+ * set the size and compression method id in the compressed data header.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ switch (cmethod)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ tmp = pglz_cmcompress((const struct varlena *) value);
+ if (tmp == NULL)
+ return PointerGetDatum(NULL);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ TOAST_PGLZ_COMPRESSION_ID);
+ break;
+ case TOAST_LZ4_COMPRESSION:
+ tmp = lz4_cmcompress((const struct varlena *) value);
+ if (tmp == NULL)
+ return PointerGetDatum(NULL);
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ TOAST_LZ4_COMPRESSION_ID);
+ break;
+ default:
+ elog(ERROR, "invalid compression method %c", cmethod);
+ }
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
- {
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ if (VARSIZE(tmp) < valsize - 2)
/* successful compression */
return PointerGetDatum(tmp);
- }
else
{
/* incompressible data */
@@ -164,7 +172,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -172,7 +180,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+
+ /* set external size and compression method */
+ VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, data_todo,
+ VARCOMPRESS_4B_C(dval));
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -181,7 +192,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..c612e3c 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -664,6 +665,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 41da0c5..c57086c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -733,6 +734,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d0ec44b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..933a073 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ffb1308..08558f8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression =
+ GetCompressionMethodName(attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index da91cbd..0d3b923 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2988,6 +2988,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6493a03..75343b8 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2876,6 +2876,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index d56f81c..aa6c19a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1092,6 +1093,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 91600ac..c291b05 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4641,7 +4641,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..cbb3fd3 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5300,6 +5301,54 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ int typlen;
+ ToastCompressionId cmid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ /* get the compression method id stored in the compressed varlena */
+ cmid = toast_get_compression_id((struct varlena *)
+ DatumGetPointer(PG_GETARG_DATUM(0)));
+ if (cmid == TOAST_INVALID_COMPRESSION_ID)
+ PG_RETURN_NULL();
+
+ /* convert compression method id to compression method name */
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ PG_RETURN_TEXT_P(cstring_to_text("pglz"));
+ case TOAST_LZ4_COMPRESSION_ID:
+ PG_RETURN_TEXT_P(cstring_to_text("lz4"));
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..cef9bbd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..453f946 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index daa5081..99a5947 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -372,6 +372,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..4bfff21 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -12,15 +12,7 @@
#ifndef DETOAST_H
#define DETOAST_H
-/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing extsize (the actual length of the external data) to rawsize
- * (the original uncompressed datum's size). The latter includes VARHDRSZ
- * overhead, the former doesn't. We never use compression unless it actually
- * saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+#include "access/toast_compression.h"
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -89,4 +81,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_id -
+ *
+ * Return the compression method id from the compressed value
+ * ----------
+ */
+extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..c6d27a4
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,141 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum ToastCompressionId
+{
+ TOAST_PGLZ_COMPRESSION_ID = 0,
+ TOAST_LZ4_COMPRESSION_ID = 1,
+ TOAST_INVALID_COMPRESSION_ID = 2
+} ToastCompressionId;
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define TOAST_PGLZ_COMPRESSION 'p'
+#define TOAST_LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/*
+ * CompressionMethodToId - Convert compression method to compression id.
+ *
+ * For more details refer comment atop ToastCompressionId.
+ */
+static inline ToastCompressionId
+CompressionMethodToId(char method)
+{
+ switch (method)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ return TOAST_PGLZ_COMPRESSION_ID;
+ case TOAST_LZ4_COMPRESSION:
+ return TOAST_LZ4_COMPRESSION_ID;
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionIdToMethod - Convert compression id to compression method
+ *
+ * For more details refer comment atop ToastCompressionId.
+ */
+static inline char
+CompressionIdToMethod(ToastCompressionId cmid)
+{
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return TOAST_PGLZ_COMPRESSION;
+ case TOAST_LZ4_COMPRESSION_ID:
+ return TOAST_LZ4_COMPRESSION;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char *
+GetCompressionMethodName(char method)
+{
+ switch (method)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ return "pglz";
+ case TOAST_LZ4_COMPRESSION:
+ return "lz4";
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+static inline char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
+ {
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/* pglz compression/decompression routines */
+extern struct varlena *pglz_cmcompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* lz4 compression/decompression routines */
+extern struct varlena *lz4_cmcompress(const struct varlena *value);
+extern struct varlena *lz4_cmdecompress(const struct varlena *value);
+extern struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..b4d0684 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,26 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) \
+ (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = \
+ ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fc..e259531 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7103,6 +7103,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 7a7cc21..0a6422d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -899,6 +899,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to build with LZ4 support (--with-lz4) */
+#undef USE_LZ4
+
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..9dfa85b9 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * compression method */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,7 +146,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +276,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
@@ -323,6 +334,35 @@ typedef struct
(VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
#define VARATT_IS_EXTERNAL_NON_EXPANDED(PTR) \
(VARATT_IS_EXTERNAL(PTR) && !VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
+
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and compression method if external data is compressed.
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & VARLENA_RAWSIZE_MASK)
+
+#define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, len, cm) \
+ do { \
+ Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_pointer).va_extinfo = (len) | (cm) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
+
+#define VARATT_EXTERNAL_GET_COMPRESSION(PTR) \
+ ((toast_pointer).va_extinfo >> VARLENA_RAWSIZE_BITS)
+
+/*
+ * Testing whether an externally-stored value is compressed now requires
+ * comparing size stored in va_extinfo (the actual length of the external data)
+ * to rawsize (the original uncompressed datum's size). The latter includes
+ * VARHDRSZ overhead, the former doesn't. We never use compression unless it
+ * actually saves space, so we expect either equality or less-than.
+ */
+#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
+ (toast_pointer).va_rawsize - VARHDRSZ)
+
#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3115829
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,261 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ substr
+--------
+ 01234
+ 8f14e
+(2 rows)
+
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+ substr
+--------
+ 8f14e
+(1 row)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..e6bd377
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,250 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ ^
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+ substr
+--------
+ 8f14e
+(1 row)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e280198..70c3830 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -115,7 +115,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 6a57e88..d81d041 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -201,6 +201,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..f56c24d
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,104 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index a4f5cc4..1460537 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -485,6 +485,7 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
+ USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
--
1.8.3.1
v37-0004-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v37-0004-Add-default_toast_compression-GUC.patchDownload
From a7ae2daff77f6ee088315a7aacf35e476c5b11c0 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 18 Mar 2021 12:05:54 +0530
Subject: [PATCH v37 4/6] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 48 +++++++++++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++---
src/backend/utils/misc/guc.c | 12 +++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 24 ++++++++++++--
src/test/regress/expected/compression.out | 16 +++++++++
src/test/regress/expected/compression_1.out | 19 +++++++++++
src/test/regress/sql/compression.sql | 8 +++++
10 files changed, 132 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 22a8d54..713c3fb 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -22,6 +22,9 @@
#include "fmgr.h"
#include "utils/builtins.h"
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -242,3 +245,48 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
return result;
#endif
}
+
+/*
+ * check_default_toast_compression - validate new default_toast_compression
+ */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c612e3c..cb76465 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -666,7 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index c57086c..99e5968 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -735,7 +735,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 08558f8..b9300b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 6b2d9d6..6abbd63 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3926,6 +3927,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 6647f8f..106016d 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -660,6 +660,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index c6d27a4..aea552b 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,6 +13,14 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
+#include "utils/guc.h"
+
+/* GUCs */
+extern char *default_toast_compression;
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
@@ -42,8 +50,6 @@ typedef enum ToastCompressionId
errdetail("This functionality requires the server to be built with lz4 support."), \
errhint("You need to rebuild PostgreSQL using --with-lz4.")))
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -126,6 +132,17 @@ CompressionNameToMethod(char *compression)
return InvalidCompressionMethod;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
+
/* pglz compression/decompression routines */
extern struct varlena *pglz_cmcompress(const struct varlena *value);
extern struct varlena *pglz_cmdecompress(const struct varlena *value);
@@ -138,4 +155,7 @@ extern struct varlena *lz4_cmdecompress(const struct varlena *value);
extern struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
int32 slicelength);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
+
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3115829..ea4d76e 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -225,6 +225,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index e6bd377..f5896f8 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -218,6 +218,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index f56c24d..bc0c1bd 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -94,6 +94,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v37-0005-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v37-0005-Alter-table-set-compression.patchDownload
From 89aed3db0868b459c6a987d84c157250d6e54802 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 17 Mar 2021 13:53:08 +0530
Subject: [PATCH v37 5/6] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/access/heap/heapam_handler.c | 25 ++++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 48 ++++++-
src/test/regress/sql/compression.sql | 20 +++
9 files changed, 321 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index f82dce4..77adc9c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0..fa52f5a 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,29 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+
+ /*
+ * Since we are rewriting the table, use this opportunity to
+ * recompress any compressed attribute with current compression method
+ * of the attribute. Basically, if the compression method of the
+ * compressed varlena is not same as current compression method of the
+ * attribute then decompress it so that if it need to be compressed
+ * then it will be compressed with the current compression method of
+ * the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ ToastCompressionId cmid;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmid = toast_get_compression_id(new_value);
+
+ if (cmid != TOAST_INVALID_COMPRESSION_ID &&
+ TupleDescAttr(newTupDesc, i)->attcompression !=
+ CompressionIdToMethod(cmid))
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b9300b9..0d76ae5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7773,6 +7781,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15046,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index d3fb734..5980641 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2116,7 +2116,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index ea4d76e..995b5b5 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -241,12 +241,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index f5896f8..f2280b3 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -237,12 +237,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index bc0c1bd..2eb92ea 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -102,6 +102,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v37-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v37-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From 2daf730bda4fcd6f847e64eea6bffd72a893e432 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v37 2/6] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/access/common/heaptuple.c | 29 +++++++++++++++++++++++++++++
src/backend/executor/execExprInterp.c | 16 ++++++++--------
src/backend/utils/adt/jsonfuncs.c | 9 +++++++--
src/include/access/htup_details.h | 2 ++
src/pl/plpgsql/src/pl_exec.c | 19 ++++++++++++++-----
5 files changed, 60 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 6061376..c4b5cd7 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -61,6 +61,7 @@
#include "access/sysattr.h"
#include "access/tupdesc_details.h"
#include "executor/tuptable.h"
+#include "fmgr.h"
#include "utils/expandeddatum.h"
@@ -1114,6 +1115,34 @@ heap_form_tuple(TupleDesc tupleDescriptor,
return tuple;
}
+
+/*
+ * heap_form_flattened_tuple
+ * wrapper over heap_form_tuple which flatten any external toast pointers
+ * before forming the tuple.
+ *
+ * The result is allocated in the current memory context.
+ */
+HeapTuple
+heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values,
+ bool *isnull)
+{
+ /* detoast any external data before forming the tuple */
+ for (int i = 0; i < tupleDescriptor->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupleDescriptor, i);
+
+ if (isnull[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ continue;
+
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
+ }
+
+ return heap_form_tuple(tupleDescriptor, values, isnull);
+}
+
/*
* heap_modify_tuple
* form a new tuple from an old tuple and a set of replacement values.
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index e47ab1e..bff6c2b 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2841,11 +2841,11 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
HeapTuple tuple;
/* build tuple from evaluated field values */
- tuple = heap_form_tuple(op->d.row.tupdesc,
- op->d.row.elemvalues,
- op->d.row.elemnulls);
+ tuple = heap_form_flattened_tuple(op->d.row.tupdesc,
+ op->d.row.elemvalues,
+ op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3086,11 +3086,11 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
HeapTuple tuple;
/* argdesc should already be valid from the DeForm step */
- tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
- op->d.fieldstore.values,
- op->d.fieldstore.nulls);
+ tuple = heap_form_flattened_tuple(*op->d.fieldstore.argdesc,
+ op->d.fieldstore.values,
+ op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..dc3f9d4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,11 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ /* detoast any external data before forming the tuple */
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3770,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7ac3c23..be6641b 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -793,6 +793,8 @@ extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
+extern HeapTuple heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
TupleDesc tupleDesc,
Datum *replValues,
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..d4fcb70 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -404,7 +404,8 @@ static void exec_move_row_from_fields(PLpgSQL_execstate *estate,
static bool compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc);
static HeapTuple make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc);
+ TupleDesc tupdesc,
+ bool flatten);
static TupleDesc deconstruct_composite_datum(Datum value,
HeapTupleData *tmptup);
static void exec_move_row_from_datum(PLpgSQL_execstate *estate,
@@ -3418,7 +3419,7 @@ exec_stmt_return_next(PLpgSQL_execstate *estate,
/* Use eval_mcontext for tuple conversion work */
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tuple = make_tuple_from_row(estate, row, tupdesc);
+ tuple = make_tuple_from_row(estate, row, tupdesc, false);
if (tuple == NULL) /* should not happen */
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
@@ -5321,12 +5322,12 @@ exec_eval_datum(PLpgSQL_execstate *estate,
/* Make sure we have a valid type/typmod setting */
BlessTupleDesc(row->rowtupdesc);
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tup = make_tuple_from_row(estate, row, row->rowtupdesc);
+ tup = make_tuple_from_row(estate, row, row->rowtupdesc, true);
if (tup == NULL) /* should not happen */
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7265,12 +7266,15 @@ compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc)
*
* The result tuple is freshly palloc'd in caller's context. Some junk
* may be left behind in eval_mcontext, too.
+ *
+ * flatten - if this is passed true then any external data will be flattened.
* ----------
*/
static HeapTuple
make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc)
+ TupleDesc tupdesc,
+ bool flatten)
{
int natts = tupdesc->natts;
HeapTuple tuple;
@@ -7300,6 +7304,11 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+
+ /* detoast any external data if the caller has asked to do so */
+ if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v37-0006-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v37-0006-default-to-with-lz4.patchDownload
From a5527b73705e93d2541424b67e6990f9c62b3c69 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v37 6/6] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 8176e99..fd6d5b6 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 54efbb2..fe3bc98 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Thu, Mar 18, 2021 at 4:40 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I just realized that in the last patch (0003) I forgot to remove 2
unused functions, CompressionMethodToId and CompressionIdToMethod.
Removed in the latest patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v38-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchtext/x-patch; charset=US-ASCII; name=v38-0001-Get-datum-from-tuple-which-doesn-t-contain-exter.patchDownload
From 58bb2cba9b7d2340cebd7f08461080b579cf2532 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 13:30:47 +0530
Subject: [PATCH v38 1/6] Get datum from tuple which doesn't contain external
data
Currently, we use HeapTupleGetDatum and HeapTupleHeaderGetDatum whenever
we need to convert a tuple to datum. Introduce another version of these
routine HeapTupleGetRawDatum and HeapTupleHeaderGetRawDatum respectively
so that all the callers who doesn't expect any external data in the tuple
can call these new routines. Likewise similar treatment for
heap_copy_tuple_as_datum and PG_RETURN_HEAPTUPLEHEADER as well. This is
the base patch for the optimization in the next patch therein we can try
to optimize the caller of the functions where we expect external varlena
by flattening any external toast pointer before forming the tuple.
Dilip Kumar based on the idea by Robert Haas
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
src/backend/access/common/heaptuple.c | 20 ++++++++++++++++++--
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 ++++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 5 +++++
src/include/funcapi.h | 15 ++++++++++++++-
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
48 files changed, 132 insertions(+), 84 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..43bb2ab 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..6061376 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,6 +991,24 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
+
+ /* the tuple should not contains any external TOAST pointers */
+ Assert(!HeapTupleHasExternal(tuple));
/*
* Fast path for easy case: just make a palloc'd copy and insert the
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6023e7c..3f48597 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index f363a4c..f60d6f1 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..e47ab1e 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to
+ * flatten any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 8532296..f5425a3 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1423,5 +1423,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5102227..b93ad73 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1856,7 +1856,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[8] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2272,7 +2273,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..079a5af 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index b263e34..6b2d9d6 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9805,7 +9805,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..cb70c05 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -373,6 +373,11 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+/*
+ * same as PG_RETURN_HEAPTUPLEHEADER but the input tuple should not contain any
+ * external varlena.
+ */
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..8ba7ae2 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -205,8 +205,13 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
* Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple) - convert a
* HeapTupleHeader to a Datum.
*
- * Macro declarations:
+ * Macro declarations/inline functions:
+ * HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple) - same as
+ * HeapTupleHeaderGetDatum but the input tuple should not contain
+ * external varlena
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same as HeapTupleGetDatum
+ * but the input tuple should not contain external varlena
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -218,7 +223,15 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*----------
*/
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v38-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v38-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From 2daf730bda4fcd6f847e64eea6bffd72a893e432 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v38 2/6] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar based on idea by Robert Haas
---
src/backend/access/common/heaptuple.c | 29 +++++++++++++++++++++++++++++
src/backend/executor/execExprInterp.c | 16 ++++++++--------
src/backend/utils/adt/jsonfuncs.c | 9 +++++++--
src/include/access/htup_details.h | 2 ++
src/pl/plpgsql/src/pl_exec.c | 19 ++++++++++++++-----
5 files changed, 60 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 6061376..c4b5cd7 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -61,6 +61,7 @@
#include "access/sysattr.h"
#include "access/tupdesc_details.h"
#include "executor/tuptable.h"
+#include "fmgr.h"
#include "utils/expandeddatum.h"
@@ -1114,6 +1115,34 @@ heap_form_tuple(TupleDesc tupleDescriptor,
return tuple;
}
+
+/*
+ * heap_form_flattened_tuple
+ * wrapper over heap_form_tuple which flatten any external toast pointers
+ * before forming the tuple.
+ *
+ * The result is allocated in the current memory context.
+ */
+HeapTuple
+heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values,
+ bool *isnull)
+{
+ /* detoast any external data before forming the tuple */
+ for (int i = 0; i < tupleDescriptor->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupleDescriptor, i);
+
+ if (isnull[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ continue;
+
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
+ }
+
+ return heap_form_tuple(tupleDescriptor, values, isnull);
+}
+
/*
* heap_modify_tuple
* form a new tuple from an old tuple and a set of replacement values.
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index e47ab1e..bff6c2b 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2841,11 +2841,11 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
HeapTuple tuple;
/* build tuple from evaluated field values */
- tuple = heap_form_tuple(op->d.row.tupdesc,
- op->d.row.elemvalues,
- op->d.row.elemnulls);
+ tuple = heap_form_flattened_tuple(op->d.row.tupdesc,
+ op->d.row.elemvalues,
+ op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3086,11 +3086,11 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
HeapTuple tuple;
/* argdesc should already be valid from the DeForm step */
- tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
- op->d.fieldstore.values,
- op->d.fieldstore.nulls);
+ tuple = heap_form_flattened_tuple(*op->d.fieldstore.argdesc,
+ op->d.fieldstore.values,
+ op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..dc3f9d4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,11 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ /* detoast any external data before forming the tuple */
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3770,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7ac3c23..be6641b 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -793,6 +793,8 @@ extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
+extern HeapTuple heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
TupleDesc tupleDesc,
Datum *replValues,
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..d4fcb70 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -404,7 +404,8 @@ static void exec_move_row_from_fields(PLpgSQL_execstate *estate,
static bool compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc);
static HeapTuple make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc);
+ TupleDesc tupdesc,
+ bool flatten);
static TupleDesc deconstruct_composite_datum(Datum value,
HeapTupleData *tmptup);
static void exec_move_row_from_datum(PLpgSQL_execstate *estate,
@@ -3418,7 +3419,7 @@ exec_stmt_return_next(PLpgSQL_execstate *estate,
/* Use eval_mcontext for tuple conversion work */
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tuple = make_tuple_from_row(estate, row, tupdesc);
+ tuple = make_tuple_from_row(estate, row, tupdesc, false);
if (tuple == NULL) /* should not happen */
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
@@ -5321,12 +5322,12 @@ exec_eval_datum(PLpgSQL_execstate *estate,
/* Make sure we have a valid type/typmod setting */
BlessTupleDesc(row->rowtupdesc);
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tup = make_tuple_from_row(estate, row, row->rowtupdesc);
+ tup = make_tuple_from_row(estate, row, row->rowtupdesc, true);
if (tup == NULL) /* should not happen */
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7265,12 +7266,15 @@ compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc)
*
* The result tuple is freshly palloc'd in caller's context. Some junk
* may be left behind in eval_mcontext, too.
+ *
+ * flatten - if this is passed true then any external data will be flattened.
* ----------
*/
static HeapTuple
make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc)
+ TupleDesc tupdesc,
+ bool flatten)
{
int natts = tupdesc->natts;
HeapTuple tuple;
@@ -7300,6 +7304,11 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+
+ /* detoast any external data if the caller has asked to do so */
+ if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v38-0004-Add-default_toast_compression-GUC.patchtext/x-patch; charset=US-ASCII; name=v38-0004-Add-default_toast_compression-GUC.patchDownload
From 2cce9331895c178c41398ddb57ac45e1491ece47 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 18 Mar 2021 12:05:54 +0530
Subject: [PATCH v38 4/6] Add default_toast_compression GUC
Justin Pryzby and Dilip Kumar
---
src/backend/access/common/toast_compression.c | 48 +++++++++++++++++++++++++++
src/backend/access/common/tupdesc.c | 2 +-
src/backend/bootstrap/bootstrap.c | 2 +-
src/backend/commands/tablecmds.c | 8 ++---
src/backend/utils/misc/guc.c | 12 +++++++
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/include/access/toast_compression.h | 24 ++++++++++++--
src/test/regress/expected/compression.out | 16 +++++++++
src/test/regress/expected/compression_1.out | 19 +++++++++++
src/test/regress/sql/compression.sql | 8 +++++
10 files changed, 132 insertions(+), 8 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 22a8d54..713c3fb 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -22,6 +22,9 @@
#include "fmgr.h"
#include "utils/builtins.h"
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
/*
* pglz_cmcompress - compression routine for pglz compression method
*
@@ -242,3 +245,48 @@ lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
return result;
#endif
}
+
+/*
+ * check_default_toast_compression - validate new default_toast_compression
+ */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index c612e3c..cb76465 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -666,7 +666,7 @@ TupleDescInitEntry(TupleDesc desc,
att->attcollation = typeForm->typcollation;
if (IsStorageCompressible(typeForm->typstorage))
- att->attcompression = DefaultCompressionMethod;
+ att->attcompression = GetDefaultToastCompression();
else
att->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index c57086c..99e5968 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -735,7 +735,7 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
if (IsStorageCompressible(attrtypes[attnum]->attstorage))
- attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
else
attrtypes[attnum]->attcompression = InvalidCompressionMethod;
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 08558f8..b9300b9 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -11931,7 +11931,7 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
if (!IsStorageCompressible(tform->typstorage))
attTup->attcompression = InvalidCompressionMethod;
else if (!CompressionMethodIsValid(attTup->attcompression))
- attTup->attcompression = DefaultCompressionMethod;
+ attTup->attcompression = GetDefaultToastCompression();
}
else
attTup->attcompression = InvalidCompressionMethod;
@@ -17745,9 +17745,9 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- return DefaultCompressionMethod;
-
- cmethod = CompressionNameToMethod(compression);
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
return cmethod;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 6b2d9d6..6abbd63 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3926,6 +3927,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 6647f8f..106016d 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -660,6 +660,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 9480100..fa4fa70 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,6 +13,14 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
+#include "utils/guc.h"
+
+/* GUCs */
+extern char *default_toast_compression;
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
@@ -42,8 +50,6 @@ typedef enum ToastCompressionId
errdetail("This functionality requires the server to be built with lz4 support."), \
errhint("You need to rebuild PostgreSQL using --with-lz4.")))
-/* use default compression method if it is not specified. */
-#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
@@ -88,6 +94,17 @@ CompressionNameToMethod(char *compression)
return InvalidCompressionMethod;
}
+/*
+ * GetDefaultToastCompression -- get the current toast compression
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
+
/* pglz compression/decompression routines */
extern struct varlena *pglz_cmcompress(const struct varlena *value);
extern struct varlena *pglz_cmdecompress(const struct varlena *value);
@@ -100,4 +117,7 @@ extern struct varlena *lz4_cmdecompress(const struct varlena *value);
extern struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
int32 slicelength);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
+
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3115829..ea4d76e 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -225,6 +225,22 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index e6bd377..f5896f8 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -218,6 +218,25 @@ CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
NOTICE: merging column "f1" with inherited definition
ERROR: column "f1" has a compression method conflict
DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index f56c24d..bc0c1bd 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -94,6 +94,14 @@ SELECT pg_column_compression(f1) FROM cmpart;
CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v38-0005-Alter-table-set-compression.patchtext/x-patch; charset=US-ASCII; name=v38-0005-Alter-table-set-compression.patchDownload
From 911f4a620ea569307bd999df257dfc232a10e55a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 17 Mar 2021 13:53:08 +0530
Subject: [PATCH v38 5/6] Alter table set compression
Add support for changing the compression method associated
with a column. There are only built-in methods so we don't
need to rewrite the table, only the new tuples will be
compressed with the new compression method.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Tomas Vondra, Alexander Korotkov and Justin Pryzby
---
doc/src/sgml/ref/alter_table.sgml | 16 +++
src/backend/access/heap/heapam_handler.c | 42 ++++++
src/backend/commands/tablecmds.c | 198 ++++++++++++++++++++++------
src/backend/parser/gram.y | 9 ++
src/bin/psql/tab-complete.c | 2 +-
src/include/nodes/parsenodes.h | 3 +-
src/test/regress/expected/compression.out | 47 ++++++-
src/test/regress/expected/compression_1.out | 48 ++++++-
src/test/regress/sql/compression.sql | 20 +++
9 files changed, 338 insertions(+), 47 deletions(-)
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index f82dce4..77adc9c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0..cbac0a1 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,46 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+
+ /*
+ * Since we are rewriting the table, use this opportunity to
+ * recompress any compressed attribute with current compression method
+ * of the attribute. Basically, if the compression method of the
+ * compressed varlena is not same as current compression method of the
+ * attribute then decompress it so that if it need to be compressed
+ * then it will be compressed with the current compression method of
+ * the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ ToastCompressionId cmid;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmid = toast_get_compression_id(new_value);
+
+ /* nothing to be done for uncompressed data */
+ if (cmid == TOAST_INVALID_COMPRESSION_ID)
+ continue;
+
+ /* convert compression id to compression method */
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ cmethod = TOAST_PGLZ_COMPRESSION;
+ break;
+ case TOAST_LZ4_COMPRESSION_ID:
+ cmethod = TOAST_LZ4_COMPRESSION;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+
+ /* if compression method doesn't match then detoast the value */
+ if (TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index b9300b9..0d76ae5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -528,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -3973,6 +3975,7 @@ AlterTableGetLockLevel(List *cmds)
*/
case AT_GenericOptions:
case AT_AlterColumnGenericOptions:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4500,7 +4503,8 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
case AT_DisableRowSecurity:
case AT_ForceRowSecurity:
case AT_NoForceRowSecurity:
- ATSimplePermissions(rel, ATT_TABLE);
+ case AT_SetCompression:
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
/* These commands never recurse */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
@@ -4908,6 +4912,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
Assert(rel->rd_rel->relkind == RELKIND_INDEX);
ATExecAlterCollationRefreshVersion(rel, cmd->object);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
default: /* oops */
elog(ERROR, "unrecognized alter table type: %d",
(int) cmd->subtype);
@@ -7773,6 +7781,67 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+ApplyChangesToIndexes(Relation rel, Relation attrelation, AttrNumber attnum,
+ char newcompression, char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7787,7 +7856,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7851,47 +7919,8 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ ApplyChangesToIndexes(rel, attrelation, attnum, InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -15017,6 +15046,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ ApplyChangesToIndexes(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 9d923b5..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -2308,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index d3fb734..5980641 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2116,7 +2116,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 19d2ba2..f9a87de 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -1901,7 +1901,8 @@ typedef enum AlterTableType
AT_AddIdentity, /* ADD IDENTITY */
AT_SetIdentity, /* SET identity column options */
AT_DropIdentity, /* DROP IDENTITY */
- AT_AlterCollationRefreshVersion /* ALTER COLLATION ... REFRESH VERSION */
+ AT_AlterCollationRefreshVersion, /* ALTER COLLATION ... REFRESH VERSION */
+ AT_SetCompression /* SET COMPRESSION */
} AlterTableType;
typedef struct ReplicaIdentityStmt
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index ea4d76e..995b5b5 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -241,12 +241,57 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+ pglz
+ lz4
+(4 rows)
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index f5896f8..f2280b3 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -237,12 +237,58 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
--------
10000
-(1 row)
+ 36036
+(2 rows)
SELECT length(f1) FROM cmdata1;
ERROR: relation "cmdata1" does not exist
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index bc0c1bd..2eb92ea 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -102,6 +102,26 @@ DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789',4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+
+SELECT pg_column_compression(f1) FROM cmpart;
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
v38-0003-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v38-0003-Built-in-compression-method.patchDownload
From 4e69e9fd59b95a0dcabb57e0feeea2f7a15bd98a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v38 3/6] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. No option for altering the
compression method for an existing column.
Dilip Kumar based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Justin Pryzby, Tomas Vondra and Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 170 +++++++++++++++
configure.ac | 20 ++
contrib/amcheck/verify_heapam.c | 2 +-
doc/src/sgml/catalogs.sgml | 10 +
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 99 ++++++---
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 244 ++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 75 ++++---
src/backend/access/common/tupdesc.c | 6 +
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 110 ++++++++++
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 26 ++-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/varlena.c | 49 +++++
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 ++++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/include/access/detoast.h | 18 +-
src/include/access/toast_compression.h | 103 ++++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 23 ++-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 2 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 46 ++++-
src/test/regress/expected/compression.out | 261 ++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 250 +++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 104 ++++++++++
src/tools/msvc/Solution.pm | 1 +
53 files changed, 1725 insertions(+), 108 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index 3fd4cec..8176e99 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,9 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -891,6 +895,8 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1596,6 +1603,8 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
@@ -8564,6 +8573,137 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -13381,6 +13521,36 @@ fi
fi
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 2f1585a..54efbb2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,21 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -1410,6 +1425,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index e614c12..6f972e6 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -1069,7 +1069,7 @@ check_tuple_attribute(HeapCheckContext *ctx)
*/
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ctx->attrsize = toast_pointer.va_extsize;
+ ctx->attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
ctx->endchunk = (ctx->attrsize - 1) / TOAST_MAX_CHUNK_SIZE;
ctx->totalchunks = ctx->endchunk + 1;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5c9f4af..de5f02a 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,16 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. Must be <literal>InvalidCompressionMethod</literal>
+ if and only if typstorage is 'plain' or 'external'.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index ff1b642..a731239 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..fc930b7 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -240,14 +240,15 @@ detoast_attr_slice(struct varlena *attr,
*/
if (slicelimit >= 0)
{
- int32 max_size;
+ int32 max_size = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
/*
* Determine maximum amount of compressed data needed for a prefix
* of a given length (after decompression).
*/
- max_size = pglz_maximum_compressed_size(slicelimit,
- toast_pointer.va_extsize);
+ if (VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer) ==
+ TOAST_PGLZ_COMPRESSION_ID)
+ max_size = pglz_maximum_compressed_size(slicelimit, max_size);
/*
* Fetch enough compressed slices (compressed marker will get set
@@ -347,7 +348,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
result = (struct varlena *) palloc(attrsize + VARHDRSZ);
@@ -408,7 +409,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) || 0 == sliceoffset);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
if (sliceoffset >= attrsize)
{
@@ -459,26 +460,60 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
/* ----------
* toast_decompress_datum -
*
+ * Returns compression method id stored in the compressed data. Otherwise,
+ * returns TOAST_INVALID_COMPRESSION_ID for the uncompressed data.
+ */
+ToastCompressionId
+toast_get_compression_id(struct varlena *attr)
+{
+ ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID;
+
+ /*
+ * Get the compression method id stored in compressed data. If it is
+ * stored externally then fetch it from the external toast pointer
+ * otherwise for inline compressed data fetch it from the compression
+ * header.
+ */
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ cmid = VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer);
+ }
+ else if (VARATT_IS_COMPRESSED(attr))
+ cmid = VARCOMPRESS_4B_C(attr);
+
+ return cmid;
+}
+/* ----------
+ * toast_decompress_datum -
+ *
* Decompress a compressed version of a varlena datum
*/
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ ToastCompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
-
- return result;
+ /*
+ * Fetch the compression method id stored in the compression header and
+ * decompress the data using the respective decompression routine.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return pglz_cmdecompress(attr);
+ case TOAST_LZ4_COMPRESSION_ID:
+ return lz4_cmdecompress(attr);
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
}
@@ -492,22 +527,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ ToastCompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * Fetch the compression method id stored in the compression header and
+ * decompress the data slice using the respective decompression routine.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return pglz_cmdecompress_slice(attr, slicelength);
+ case TOAST_LZ4_COMPRESSION_ID:
+ return lz4_cmdecompress_slice(attr, slicelength);
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
}
/* ----------
@@ -589,7 +626,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..22a8d54
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,244 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef USE_LZ4
+#include <lz4.h>
+#endif
+
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/*
+ * pglz_cmcompress - compression routine for pglz compression method
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_cmcompress(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_cmdecompress - decompression routine for pglz compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress - slice decompression routine for pglz compression method
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_cmcompress - compression routine for lz4 compression method
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+lz4_cmcompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "could not compress data with lz4");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_cmdecompress - decompression routine for lz4 compression method
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+lz4_cmdecompress(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_cmdecompress_slice - slice decompression routine for lz4 compression
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+lz4_cmdecompress_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* slice decompression not supported prior to 1.8.3 */
+ if (LZ4_versionNumber() < 10803)
+ return lz4_cmdecompress(value);
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..c84521e 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,48 +44,56 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
+ Assert(CompressionMethodIsValid(cmethod));
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
/*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
+ * Call respective compression routine for the compression method and also
+ * set the size and compression method id in the compressed data header.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ switch (cmethod)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ tmp = pglz_cmcompress((const struct varlena *) value);
+ if (tmp == NULL)
+ return PointerGetDatum(NULL);
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ TOAST_PGLZ_COMPRESSION_ID);
+ break;
+ case TOAST_LZ4_COMPRESSION:
+ tmp = lz4_cmcompress((const struct varlena *) value);
+ if (tmp == NULL)
+ return PointerGetDatum(NULL);
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize,
+ TOAST_LZ4_COMPRESSION_ID);
+ break;
+ default:
+ elog(ERROR, "invalid compression method %c", cmethod);
+ }
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
- {
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
+ if (VARSIZE(tmp) < valsize - 2)
/* successful compression */
return PointerGetDatum(tmp);
- }
else
{
/* incompressible data */
@@ -164,7 +172,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -172,7 +180,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+
+ /* set external size and compression method */
+ VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, data_todo,
+ VARCOMPRESS_4B_C(dval));
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -181,7 +192,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..c612e3c 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -664,6 +665,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = DefaultCompressionMethod;
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 41da0c5..c57086c 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -733,6 +734,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = DefaultCompressionMethod;
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d0ec44b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..933a073 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ffb1308..08558f8 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -558,6 +559,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +854,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2410,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression =
+ GetCompressionMethodName(attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2460,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2710,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -6340,6 +6388,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -11859,6 +11919,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = DefaultCompressionMethod;
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -17641,3 +17718,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ return DefaultCompressionMethod;
+
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index da91cbd..0d3b923 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2988,6 +2988,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6493a03..75343b8 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2876,6 +2876,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..9d923b5 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -3421,11 +3423,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3437,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3483,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3721,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15308,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15829,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index d56f81c..aa6c19a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1092,6 +1093,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 91600ac..c291b05 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4641,7 +4641,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..cbb3fd3 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5300,6 +5301,54 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ int typlen;
+ ToastCompressionId cmid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ /* get the compression method id stored in the compressed varlena */
+ cmid = toast_get_compression_id((struct varlena *)
+ DatumGetPointer(PG_GETARG_DATUM(0)));
+ if (cmid == TOAST_INVALID_COMPRESSION_ID)
+ PG_RETURN_NULL();
+
+ /* convert compression method id to compression method name */
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ PG_RETURN_TEXT_P(cstring_to_text("pglz"));
+ case TOAST_LZ4_COMPRESSION_ID:
+ PG_RETURN_TEXT_P(cstring_to_text("lz4"));
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..cef9bbd 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcmname,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression[j] &&
+ tbinfo->attcompression[j] != '\0')
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..453f946 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..b284113 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compresssion info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index daa5081..99a5947 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -372,6 +372,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..4bfff21 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -12,15 +12,7 @@
#ifndef DETOAST_H
#define DETOAST_H
-/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing extsize (the actual length of the external data) to rawsize
- * (the original uncompressed datum's size). The latter includes VARHDRSZ
- * overhead, the former doesn't. We never use compression unless it actually
- * saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
+#include "access/toast_compression.h"
/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
@@ -89,4 +81,12 @@ extern Size toast_raw_datum_size(Datum value);
*/
extern Size toast_datum_size(Datum value);
+/* ----------
+ * toast_get_compression_id -
+ *
+ * Return the compression method id from the compressed value
+ * ----------
+ */
+extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
+
#endif /* DETOAST_H */
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..9480100
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,103 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum ToastCompressionId
+{
+ TOAST_PGLZ_COMPRESSION_ID = 0,
+ TOAST_LZ4_COMPRESSION_ID = 1,
+ TOAST_INVALID_COMPRESSION_ID = 2
+} ToastCompressionId;
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define TOAST_PGLZ_COMPRESSION 'p'
+#define TOAST_LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+/* use default compression method if it is not specified. */
+#define DefaultCompressionMethod TOAST_PGLZ_COMPRESSION
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char *
+GetCompressionMethodName(char method)
+{
+ switch (method)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ return "pglz";
+ case TOAST_LZ4_COMPRESSION:
+ return "lz4";
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+static inline char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
+ {
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/* pglz compression/decompression routines */
+extern struct varlena *pglz_cmcompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress(const struct varlena *value);
+extern struct varlena *pglz_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* lz4 compression/decompression routines */
+extern struct varlena *lz4_cmcompress(const struct varlena *value);
+extern struct varlena *lz4_cmdecompress(const struct varlena *value);
+extern struct varlena *lz4_cmdecompress_slice(const struct varlena *value,
+ int32 slicelength);
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..b4d0684 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,26 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) \
+ (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = \
+ ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fc..e259531 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7103,6 +7103,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..19d2ba2 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 7a7cc21..0a6422d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -899,6 +899,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to build with LZ4 support (--with-lz4) */
+#undef USE_LZ4
+
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..9dfa85b9 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -67,7 +67,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * compression method */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,7 +146,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +276,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
@@ -323,6 +334,35 @@ typedef struct
(VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
#define VARATT_IS_EXTERNAL_NON_EXPANDED(PTR) \
(VARATT_IS_EXTERNAL(PTR) && !VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
+
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and compression method if external data is compressed.
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & VARLENA_RAWSIZE_MASK)
+
+#define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, len, cm) \
+ do { \
+ Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_pointer).va_extinfo = (len) | (cm) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
+
+#define VARATT_EXTERNAL_GET_COMPRESSION(PTR) \
+ ((toast_pointer).va_extinfo >> VARLENA_RAWSIZE_BITS)
+
+/*
+ * Testing whether an externally-stored value is compressed now requires
+ * comparing size stored in va_extinfo (the actual length of the external data)
+ * to rawsize (the original uncompressed datum's size). The latter includes
+ * VARHDRSZ overhead, the former doesn't. We never use compression unless it
+ * actually saves space, so we expect either equality or less-than.
+ */
+#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
+ (toast_pointer).va_rawsize - VARHDRSZ)
+
#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3115829
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,261 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ substr
+--------
+ 01234
+ 8f14e
+(2 rows)
+
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+ substr
+--------
+ 8f14e
+(1 row)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..e6bd377
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,250 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+ ^
+\d+ cmdata1
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ ^
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+ substr
+--------
+ 8f14e
+(1 row)
+
+DROP TABLE cmdata2;
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789',4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart;
+ERROR: relation "cmpart" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart;
+ ^
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e280198..70c3830 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -115,7 +115,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 6a57e88..d81d041 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -201,6 +201,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..f56c24d
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,104 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890',1000));
+\d+ cmdata
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890',1004));
+\d+ cmdata1
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- verify stored compression method
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890',1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+UPDATE cmmove2 SET f1 = cmdata.f1 FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+-- test external compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+DROP TABLE cmdata2;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789',1004));
+INSERT INTO cmpart VALUES (repeat('123456789',4004));
+SELECT pg_column_compression(f1) FROM cmpart;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index a4f5cc4..1460537 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -485,6 +485,7 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
+ USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
--
1.8.3.1
v38-0006-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v38-0006-default-to-with-lz4.patchDownload
From 021ca69fa07defd722b519531042b3fafedcbb37 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v38 6/6] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 8176e99..fd6d5b6 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 54efbb2..fe3bc98 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Thu, Mar 18, 2021 at 10:22 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I just realized that in the last patch (0003) I forgot to remove 2
unused functions, CompressionMethodToId and CompressionIdToMethod.
Removed in the latest patch.
I spent a little time polishing 0001 and here's what I came up with. I
adjusted some comments, added documentation, fixed up the commit
message, etc.
I still don't quite like the approach in 0002. I feel that the
function should not construct the tuple but modify the caller's arrays
as a side effect. And if we're absolutely committed to the design
where it does that, the comments need to call it out clearly, which
they don't.
Regarding 0003:
I think it might make sense to change the names of the compression and
decompression functions to match the names of the callers more
closely. Like, toast_decompress_datum() calls either
pglz_cmdecompress() or lz4_cmdecompress(). But, why not
pglz_decompress_datum() or lz4_decompress_datum()? The "cm" thing
doesn't really mean anything, and because the varlena is allocated by
that function itself rather than the caller, this can't be used for
anything other than TOAST.
In toast_compress_datum(), if (tmp == NULL) return
PointerGetDatum(NULL) is duplicated. It would be better to move it
after the switch.
Instead of "could not compress data with lz4" I suggest "lz4
compression failed".
In catalogs.sgml, you shouldn't mention InvalidCompressionMethod, but
you should explain what the actual possible values mean. Look at the
way attidentity and attgenerated are documented and do it like that.
In pg_column_compression() it might be a bit more elegant to add a
char *result variable or similar, and have the switch cases just set
it, and then do PG_RETURN_TEXT_P(cstring_to_text(result)) at the
bottom.
In getTableAttrs(), if the remoteVersion is new, the column gets a
different alias than if the column is old.
In dumpTableSchema(), the condition tbinfo->attcompression[j] means
exactly the thing as the condition tbinfo->attcompression[j] != '\0',
so it can't be right to test both. I think that there's some confusion
here about the data type of tbinfo->attcompression[j]. It seems to be
char *. Maybe you intended to test the first character in that second
test, but that's not what this does. But you don't need to test that
anyway because the switch already takes care of it. So I suggest (a)
removing tbinfo->attcompression[j] != '\0' from this if-statement and
(b) adding != NULL to the previous line for clarity. I would also
suggest concluding the switch with a break just for symmetry.
The patch removes 11 references to va_extsize and leaves behind 4.
None of those 4 look like things that should have been left.
The comment which says "When fetching a prefix of a compressed
external datum, account for the rawsize tracking amount of raw data,
which is stored at the beginning as an int32 value)" is no longer 100%
accurate. I suggest changing it to say something like "When fetching a
prefix of a compressed external datum, account for the space required
by va_tcinfo" and leave out the rest.
In describeOneTableDetails, the comment "compresssion info" needs to
be compressed by removing one "s".
It seems a little unfortunate that we need to include
access/toast_compression.h in detoast.h. It seems like the reason we
need to do that is because otherwise we won't have ToastCompressionId
defined and so we won't be able to prototype toast_get_compression_id.
But I think we should solve that problem by moving that file to
toast_compression.c. (I'm OK if you want to keep the files separate,
or if you want to reverse course and combine them I'm OK with that
too, but the extra header dependency is clearly a sign of a problem
with the split.)
Regarding 0005:
I think ApplyChangesToIndexes() should be renamed to something like
SetIndexStorageProperties(). It's too generic right now.
I think 0004 and 0005 should just be merged into 0003. I can't see
committing them separately. I know I was the one who made you split
the patch up in the first place, but those patches are quite small and
simple now, so it makes more sense to me to combine them.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patchapplication/octet-stream; name=v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patchDownload
From 7fc8a88b725090b8365e9df036e34c0a8689cde0 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Thu, 18 Mar 2021 12:20:00 -0400
Subject: [PATCH v39] Invent HeapTupleGetRawDatum and friends.
HeapTupleGetRawDatum, HeapTupleHeaderGetRawDatum, and
PG_RETURN_HEAPTUPLEHEADER_RAW, and heap_copy_tuple_as_raw_datum are
just like HeapTupleGetDatum, HeapTupleHeaderGetDatum,
PG_RETURN_HEAPTUPLEHEADER, and heap_copy_tuple_as_datum, except that
they don't check that the returned tuple is free of external datums.
Instead, the caller must guarantee that this is the case.
The overarching idea here is that it's usually more efficient for the
caller to make sure to build the tuple originally without any external
datums, rather than building a tuple that may contain external datums
and then invoking a function which will possibly deform that tuple,
detoast those fields, and build a replacement tuple. Fortunately, very
few places are relying on this behavior of the above functions, which
means that this patch can safely change practically all the calls to
the existing functions to use the "raw" variants.
In the places that continue to use the non-raw variants after this
patch, there is an opportunity to improve performance by fixing them
so that they can. That work is planned for a follow-up patch.
That said, the main motivation for this patch is actually to cut down
on the number of places that depend on HeapTupleHasExternal tests to
avoid performance problems, and to make it clear where the remaining
places are. New features - such as configurable TOAST compression -
will not have the luxury of consuming an infomask bit to indicate
whether or not they are in use somewhere inside a tuple.
Dilip Kumar, based on an idea by me. Documentation changes also by me.
Discussion: http://postgr.es/m/CAFiTN-u3jrRDiuyGxGvNSGDXLhG4=U2orHnjk4B6WBy9Eo9kMQ@mail.gmail.com
Discussion: http://postgr.es/m/CAFiTN-s8SsWs2zCZL3TFxXwi4VE35J%3D4aKpduKVrTaL%2B3wnB0A%40mail.gmail.com
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 +--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 ++---
contrib/pageinspect/ginfuncs.c | 6 ++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 +++---
contrib/pageinspect/heapfuncs.c | 6 ++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
.../pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 ++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
doc/src/sgml/xfunc.sgml | 8 ++++--
src/backend/access/common/heaptuple.c | 26 +++++++++++++++----
src/backend/access/transam/commit_ts.c | 4 +--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 ++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 +++++++----
src/backend/replication/slotfuncs.c | 8 +++---
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 +--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 +--
src/backend/utils/adt/misc.c | 4 +--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 +++--
src/backend/utils/adt/rowtypes.c | 4 +--
src/backend/utils/adt/tsvector_op.c | 4 +--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 +++---
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 8 ++++++
src/include/funcapi.h | 14 ++++++++++
src/pl/plperl/plperl.c | 4 +--
src/pl/tcl/pltcl.c | 4 +--
.../modules/test_predtest/test_predtest.c | 3 ++-
49 files changed, 144 insertions(+), 88 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa88e..5a41116e8e 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff844..cc7a8e0eef 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01cac..d466f6cc46 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df07177ed..5d517c8afc 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2deb66..626294685a 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b572f..f0028e4cc4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbcdb8..77dd559c7e 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f6303df..dbf34f87d4 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119474..957ee5bc14 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb31e..3a050ee88c 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21016..e7aab866b8 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579fcbb..1fd405e5df 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbfa44..7eb80d5b78 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124e62..43bb2ab852 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb892..333f7fd391 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193bb25..147c8d22eb 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb30f0..7d74c316c7 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff8af..70b8bf0855 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0bb98..53801e92cd 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 41bcc5b79d..7707400c42 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -2923,14 +2923,18 @@ HeapTuple BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values)
<para>
Once you have built a tuple to return from your function, it
- must be converted into a <type>Datum</type>. Use:
+ must be converted into a <type>Datum</type>. Use one of the following:
<programlisting>
HeapTupleGetDatum(HeapTuple tuple)
+HeapTupleGetRawDatum(HeapTuple tuple)
</programlisting>
to convert a <structname>HeapTuple</structname> into a valid Datum. This
<type>Datum</type> can be returned directly if you intend to return
just a single row, or it can be used as the current return value
- in a set-returning function.
+ in a set-returning function. <literal>HeapTupleGetRawDatum</literal>
+ is more efficient and is preferred for new code, but the caller must
+ ensure that the tuple contains no external TOAST pointers. If this
+ is not certain, use <literal>HeapTupleGetDatum</literal>.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0fa5a..06479f5f76 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,11 +991,29 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
+
+ /* The tuple should not contains any external TOAST pointers. */
+ Assert(!HeapTupleHasExternal(tuple));
/*
- * Fast path for easy case: just make a palloc'd copy and insert the
- * correct composite-Datum header fields (since those may not be set if
- * the given tuple came from disk, rather than from heap_form_tuple).
+ * Just make a palloc'd copy and insert the correct composite-Datum
+ * header fields (since those may not be set if the given tuple came
+ * from disk, rather than from heap_form_tuple).
*/
td = (HeapTupleHeader) palloc(tuple->t_len);
memcpy((char *) td, (char *) tuple->t_data, tuple->t_len);
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66286..015c84179e 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1fa1..b44066a4eb 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6023e7c16f..3f48597c2a 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index f363a4c639..f60d6f1496 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b690d8..7cd62e70b8 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9ccb..f566e1e0a5 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286d8c..e47ab1ec15 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to
+ * flatten any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44113..f92628bc59 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 8532296f26..f5425a385c 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1423,5 +1423,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1ba8..115131c707 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dced9..633c90d08d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e218..5871c0b404 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c55ea..ce11554b3b 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 7cf9a0efbe..711a094ddc 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265c12..081844939b 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574d7e..57e0ea09a3 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5db6..6915939f43 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5102227a60..b93ad73241 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1856,7 +1856,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[8] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2272,7 +2273,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6ae7..079a5af5fb 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebcc8f..6679493247 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index b263e3493b..6b2d9d6693 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9805,7 +9805,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a882..195c127739 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852e7f..7ac3c2382d 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c86e..89eec7275a 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -372,7 +372,15 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_TEXT_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
+
+/*
+ * A composite-type Datum must not contain external fields. To return a tuple
+ * that might, use PG_RETURN_HEAPTUPLEHEADER, which will build a replacement
+ * tuple without such fields if any are present. Otherwise, use
+ * PG_RETURN_HEAPTUPLEHEADER_RAW, which is slightly faster.
+ */
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2a1f..92cc7bdf04 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -207,6 +207,8 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*
* Macro declarations:
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same, but must have no external
+ * TOAST pointer
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -219,6 +221,7 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*/
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
@@ -231,6 +234,17 @@ extern AttInMetadata *TupleDescGetAttInMetadata(TupleDesc tupdesc);
extern HeapTuple BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values);
extern Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple);
+/*
+ * This is a faster version of HeapTupleHeaderGetDatum which can be used
+ * when it is known that no external TOAST pointers are present.
+ */
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
/*----------
* Support for Set Returning Functions (SRFs)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf71a..cdf79f78e8 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e11837559d..b4e710a22a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd61c..ec83645bbe 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
2.24.3 (Apple Git-128)
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 1:27 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Thu, Mar 18, 2021 at 10:22 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I just realized that in the last patch (0003) I forgot to remove 2
unused functions, CompressionMethodToId and CompressionIdToMethod.
Removed in the latest patch.I spent a little time polishing 0001 and here's what I came up with. I
adjusted some comments, added documentation, fixed up the commit
message, etc.
Thanks, the changes looks fine to me.
I still don't quite like the approach in 0002. I feel that the
function should not construct the tuple but modify the caller's arrays
as a side effect. And if we're absolutely committed to the design
where it does that, the comments need to call it out clearly, which
they don't.
Added comment for the same.
Regarding 0003:
I think it might make sense to change the names of the compression and
decompression functions to match the names of the callers more
closely. Like, toast_decompress_datum() calls either
pglz_cmdecompress() or lz4_cmdecompress(). But, why not
pglz_decompress_datum() or lz4_decompress_datum()? The "cm" thing
doesn't really mean anything, and because the varlena is allocated by
that function itself rather than the caller, this can't be used for
anything other than TOAST.
Done
In toast_compress_datum(), if (tmp == NULL) return
PointerGetDatum(NULL) is duplicated. It would be better to move it
after the switch.
Done
Instead of "could not compress data with lz4" I suggest "lz4
compression failed".
Done
In catalogs.sgml, you shouldn't mention InvalidCompressionMethod, but
you should explain what the actual possible values mean. Look at the
way attidentity and attgenerated are documented and do it like that.
Done
In pg_column_compression() it might be a bit more elegant to add a
char *result variable or similar, and have the switch cases just set
it, and then do PG_RETURN_TEXT_P(cstring_to_text(result)) at the
bottom.
Done
In getTableAttrs(), if the remoteVersion is new, the column gets a
different alias than if the column is old.
Fixed
In dumpTableSchema(), the condition tbinfo->attcompression[j] means
exactly the thing as the condition tbinfo->attcompression[j] != '\0',
so it can't be right to test both. I think that there's some confusion
here about the data type of tbinfo->attcompression[j]. It seems to be
char *. Maybe you intended to test the first character in that second
test, but that's not what this does. But you don't need to test that
anyway because the switch already takes care of it. So I suggest (a)
removing tbinfo->attcompression[j] != '\0' from this if-statement and
(b) adding != NULL to the previous line for clarity. I would also
suggest concluding the switch with a break just for symmetry.
Fixed
The patch removes 11 references to va_extsize and leaves behind 4.
None of those 4 look like things that should have been left.
Fixed
The comment which says "When fetching a prefix of a compressed
external datum, account for the rawsize tracking amount of raw data,
which is stored at the beginning as an int32 value)" is no longer 100%
accurate. I suggest changing it to say something like "When fetching a
prefix of a compressed external datum, account for the space required
by va_tcinfo" and leave out the rest.
Done
In describeOneTableDetails, the comment "compresssion info" needs to
be compressed by removing one "s".
Done
It seems a little unfortunate that we need to include
access/toast_compression.h in detoast.h. It seems like the reason we
need to do that is because otherwise we won't have ToastCompressionId
defined and so we won't be able to prototype toast_get_compression_id.
But I think we should solve that problem by moving that file to
toast_compression.c. (I'm OK if you want to keep the files separate,
or if you want to reverse course and combine them I'm OK with that
too, but the extra header dependency is clearly a sign of a problem
with the split.)
Moved to toast_compression.c
Regarding 0005:
I think ApplyChangesToIndexes() should be renamed to something like
SetIndexStorageProperties(). It's too generic right now.
Done
I think 0004 and 0005 should just be merged into 0003. I can't see
committing them separately. I know I was the one who made you split
the patch up in the first place, but those patches are quite small and
simple now, so it makes more sense to me to combine them.
Done
Also added a test case for vacuum full to recompress the data.
One question, like storage should we apply the alter set compression
changes recursively to the inherited children (I have attached a
separate patch for this )?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patchtext/x-patch; charset=US-ASCII; name=v39-0001-Invent-HeapTupleGetRawDatum-and-friends.patchDownload
From 8186bdc373418a407610959c16cc838a478feb5e Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Thu, 18 Mar 2021 12:20:00 -0400
Subject: [PATCH v39 1/4] Invent HeapTupleGetRawDatum and friends.
HeapTupleGetRawDatum, HeapTupleHeaderGetRawDatum, and
PG_RETURN_HEAPTUPLEHEADER_RAW, and heap_copy_tuple_as_raw_datum are
just like HeapTupleGetDatum, HeapTupleHeaderGetDatum,
PG_RETURN_HEAPTUPLEHEADER, and heap_copy_tuple_as_datum, except that
they don't check that the returned tuple is free of external datums.
Instead, the caller must guarantee that this is the case.
The overarching idea here is that it's usually more efficient for the
caller to make sure to build the tuple originally without any external
datums, rather than building a tuple that may contain external datums
and then invoking a function which will possibly deform that tuple,
detoast those fields, and build a replacement tuple. Fortunately, very
few places are relying on this behavior of the above functions, which
means that this patch can safely change practically all the calls to
the existing functions to use the "raw" variants.
In the places that continue to use the non-raw variants after this
patch, there is an opportunity to improve performance by fixing them
so that they can. That work is planned for a follow-up patch.
That said, the main motivation for this patch is actually to cut down
on the number of places that depend on HeapTupleHasExternal tests to
avoid performance problems, and to make it clear where the remaining
places are. New features - such as configurable TOAST compression -
will not have the luxury of consuming an infomask bit to indicate
whether or not they are in use somewhere inside a tuple.
Dilip Kumar, based on an idea by me. Documentation changes also by me.
Discussion: http://postgr.es/m/CAFiTN-u3jrRDiuyGxGvNSGDXLhG4=U2orHnjk4B6WBy9Eo9kMQ@mail.gmail.com
Discussion: http://postgr.es/m/CAFiTN-s8SsWs2zCZL3TFxXwi4VE35J%3D4aKpduKVrTaL%2B3wnB0A%40mail.gmail.com
---
contrib/dblink/dblink.c | 2 +-
contrib/hstore/hstore_io.c | 4 ++--
contrib/hstore/hstore_op.c | 2 +-
contrib/old_snapshot/time_mapping.c | 2 +-
contrib/pageinspect/brinfuncs.c | 2 +-
contrib/pageinspect/btreefuncs.c | 6 +++---
contrib/pageinspect/ginfuncs.c | 6 +++---
contrib/pageinspect/gistfuncs.c | 2 +-
contrib/pageinspect/hashfuncs.c | 8 ++++----
contrib/pageinspect/heapfuncs.c | 6 +++---
contrib/pageinspect/rawpage.c | 2 +-
contrib/pg_buffercache/pg_buffercache_pages.c | 2 +-
contrib/pg_stat_statements/pg_stat_statements.c | 3 ++-
contrib/pg_visibility/pg_visibility.c | 13 ++++++++-----
contrib/pgcrypto/pgp-pgsql.c | 2 +-
contrib/pgstattuple/pgstatapprox.c | 2 +-
contrib/pgstattuple/pgstatindex.c | 6 +++---
contrib/pgstattuple/pgstattuple.c | 2 +-
contrib/sslinfo/sslinfo.c | 2 +-
doc/src/sgml/xfunc.sgml | 8 ++++++--
src/backend/access/common/heaptuple.c | 26 ++++++++++++++++++++-----
src/backend/access/transam/commit_ts.c | 4 ++--
src/backend/access/transam/multixact.c | 2 +-
src/backend/access/transam/twophase.c | 2 +-
src/backend/access/transam/xlogfuncs.c | 2 +-
src/backend/catalog/objectaddress.c | 6 +++---
src/backend/commands/sequence.c | 2 +-
src/backend/executor/execExprInterp.c | 15 +++++++++-----
src/backend/replication/slotfuncs.c | 8 ++++----
src/backend/replication/walreceiver.c | 3 ++-
src/backend/statistics/mcv.c | 2 +-
src/backend/tsearch/wparser.c | 4 ++--
src/backend/utils/adt/acl.c | 2 +-
src/backend/utils/adt/datetime.c | 2 +-
src/backend/utils/adt/genfile.c | 2 +-
src/backend/utils/adt/lockfuncs.c | 4 ++--
src/backend/utils/adt/misc.c | 4 ++--
src/backend/utils/adt/partitionfuncs.c | 2 +-
src/backend/utils/adt/pgstatfuncs.c | 6 ++++--
src/backend/utils/adt/rowtypes.c | 4 ++--
src/backend/utils/adt/tsvector_op.c | 4 ++--
src/backend/utils/misc/guc.c | 2 +-
src/backend/utils/misc/pg_controldata.c | 8 ++++----
src/include/access/htup_details.h | 1 +
src/include/fmgr.h | 8 ++++++++
src/include/funcapi.h | 14 +++++++++++++
src/pl/plperl/plperl.c | 4 ++--
src/pl/tcl/pltcl.c | 4 ++--
src/test/modules/test_predtest/test_predtest.c | 3 ++-
49 files changed, 144 insertions(+), 88 deletions(-)
diff --git a/contrib/dblink/dblink.c b/contrib/dblink/dblink.c
index 3a0beaa..5a41116 100644
--- a/contrib/dblink/dblink.c
+++ b/contrib/dblink/dblink.c
@@ -1630,7 +1630,7 @@ dblink_get_pkey(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/hstore/hstore_io.c b/contrib/hstore/hstore_io.c
index b3304ff..cc7a8e0 100644
--- a/contrib/hstore/hstore_io.c
+++ b/contrib/hstore/hstore_io.c
@@ -1137,14 +1137,14 @@ hstore_populate_record(PG_FUNCTION_ARGS)
* check domain constraints before deciding we're done.
*/
if (argtype != tupdesc->tdtypeid)
- domain_check(HeapTupleGetDatum(rettuple), false,
+ domain_check(HeapTupleGetRawDatum(rettuple), false,
argtype,
&my_extra->domain_info,
fcinfo->flinfo->fn_mcxt);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_DATUM(HeapTupleGetDatum(rettuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(rettuple->t_data);
}
diff --git a/contrib/hstore/hstore_op.c b/contrib/hstore/hstore_op.c
index dd79d01..d466f6c 100644
--- a/contrib/hstore/hstore_op.c
+++ b/contrib/hstore/hstore_op.c
@@ -1067,7 +1067,7 @@ hstore_each(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, dvalues, nulls);
- res = HeapTupleGetDatum(tuple);
+ res = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, PointerGetDatum(res));
}
diff --git a/contrib/old_snapshot/time_mapping.c b/contrib/old_snapshot/time_mapping.c
index 3df0717..5d517c8 100644
--- a/contrib/old_snapshot/time_mapping.c
+++ b/contrib/old_snapshot/time_mapping.c
@@ -72,7 +72,7 @@ pg_old_snapshot_time_mapping(PG_FUNCTION_ARGS)
tuple = MakeOldSnapshotTimeMappingTuple(funcctx->tuple_desc, mapping);
++mapping->current_index;
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/contrib/pageinspect/brinfuncs.c b/contrib/pageinspect/brinfuncs.c
index 0e3c2de..6262946 100644
--- a/contrib/pageinspect/brinfuncs.c
+++ b/contrib/pageinspect/brinfuncs.c
@@ -366,7 +366,7 @@ brin_metapage_info(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/contrib/pageinspect/btreefuncs.c b/contrib/pageinspect/btreefuncs.c
index b7725b5..f0028e4 100644
--- a/contrib/pageinspect/btreefuncs.c
+++ b/contrib/pageinspect/btreefuncs.c
@@ -263,7 +263,7 @@ bt_page_stats_internal(PG_FUNCTION_ARGS, enum pageinspect_version ext_version)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -436,7 +436,7 @@ bt_page_print_tuples(struct user_args *uargs)
/* Build and return the result tuple */
tuple = heap_form_tuple(uargs->tupd, values, nulls);
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/*-------------------------------------------------------
@@ -766,7 +766,7 @@ bt_metap(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
UnlockReleaseBuffer(buffer);
relation_close(rel, AccessShareLock);
diff --git a/contrib/pageinspect/ginfuncs.c b/contrib/pageinspect/ginfuncs.c
index e425cbc..77dd559 100644
--- a/contrib/pageinspect/ginfuncs.c
+++ b/contrib/pageinspect/ginfuncs.c
@@ -82,7 +82,7 @@ gin_metapage_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
@@ -150,7 +150,7 @@ gin_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
typedef struct gin_leafpage_items_state
@@ -254,7 +254,7 @@ gin_leafpage_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->seg = GinNextPostingListSegment(cur);
diff --git a/contrib/pageinspect/gistfuncs.c b/contrib/pageinspect/gistfuncs.c
index eb9f630..dbf34f8 100644
--- a/contrib/pageinspect/gistfuncs.c
+++ b/contrib/pageinspect/gistfuncs.c
@@ -89,7 +89,7 @@ gist_page_opaque_info(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(resultTuple);
+ return HeapTupleGetRawDatum(resultTuple);
}
Datum
diff --git a/contrib/pageinspect/hashfuncs.c b/contrib/pageinspect/hashfuncs.c
index ff01119..957ee5b 100644
--- a/contrib/pageinspect/hashfuncs.c
+++ b/contrib/pageinspect/hashfuncs.c
@@ -273,7 +273,7 @@ hash_page_stats(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
@@ -368,7 +368,7 @@ hash_page_items(PG_FUNCTION_ARGS)
values[j] = Int64GetDatum((int64) hashkey);
tuple = heap_form_tuple(fctx->attinmeta->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
uargs->offset = uargs->offset + 1;
@@ -499,7 +499,7 @@ hash_bitmap_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* ------------------------------------------------
@@ -577,5 +577,5 @@ hash_metapage_info(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/heapfuncs.c b/contrib/pageinspect/heapfuncs.c
index f6760eb..3a050ee 100644
--- a/contrib/pageinspect/heapfuncs.c
+++ b/contrib/pageinspect/heapfuncs.c
@@ -282,7 +282,7 @@ heap_page_items(PG_FUNCTION_ARGS)
/* Build and return the result tuple. */
resultTuple = heap_form_tuple(inter_call_data->tupd, values, nulls);
- result = HeapTupleGetDatum(resultTuple);
+ result = HeapTupleGetRawDatum(resultTuple);
inter_call_data->offset++;
@@ -540,7 +540,7 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
values[0] = PointerGetDatum(construct_empty_array(TEXTOID));
values[1] = PointerGetDatum(construct_empty_array(TEXTOID));
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* build set of raw flags */
@@ -618,5 +618,5 @@ heap_tuple_infomask_flags(PG_FUNCTION_ARGS)
/* Returns the record as Datum */
tuple = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
diff --git a/contrib/pageinspect/rawpage.c b/contrib/pageinspect/rawpage.c
index 7272b21..e7aab86 100644
--- a/contrib/pageinspect/rawpage.c
+++ b/contrib/pageinspect/rawpage.c
@@ -328,7 +328,7 @@ page_header(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
diff --git a/contrib/pg_buffercache/pg_buffercache_pages.c b/contrib/pg_buffercache/pg_buffercache_pages.c
index 1bd579f..1fd405e 100644
--- a/contrib/pg_buffercache/pg_buffercache_pages.c
+++ b/contrib/pg_buffercache/pg_buffercache_pages.c
@@ -230,7 +230,7 @@ pg_buffercache_pages(PG_FUNCTION_ARGS)
/* Build and return the tuple. */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/contrib/pg_stat_statements/pg_stat_statements.c b/contrib/pg_stat_statements/pg_stat_statements.c
index 62cccbf..7eb80d5 100644
--- a/contrib/pg_stat_statements/pg_stat_statements.c
+++ b/contrib/pg_stat_statements/pg_stat_statements.c
@@ -1922,7 +1922,8 @@ pg_stat_statements_info(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(stats.dealloc);
values[1] = TimestampTzGetDatum(stats.stats_reset);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pg_visibility/pg_visibility.c b/contrib/pg_visibility/pg_visibility.c
index dd0c124..43bb2ab 100644
--- a/contrib/pg_visibility/pg_visibility.c
+++ b/contrib/pg_visibility/pg_visibility.c
@@ -97,7 +97,8 @@ pg_visibility_map(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -156,7 +157,8 @@ pg_visibility(PG_FUNCTION_ARGS)
relation_close(rel, AccessShareLock);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -197,7 +199,7 @@ pg_visibility_map_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -243,7 +245,7 @@ pg_visibility_rel(PG_FUNCTION_ARGS)
info->next++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -303,7 +305,8 @@ pg_visibility_map_summary(PG_FUNCTION_ARGS)
values[0] = Int64GetDatum(all_visible);
values[1] = Int64GetDatum(all_frozen);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
diff --git a/contrib/pgcrypto/pgp-pgsql.c b/contrib/pgcrypto/pgp-pgsql.c
index 0536bfb..333f7fd 100644
--- a/contrib/pgcrypto/pgp-pgsql.c
+++ b/contrib/pgcrypto/pgp-pgsql.c
@@ -973,7 +973,7 @@ pgp_armor_headers(PG_FUNCTION_ARGS)
/* build a tuple */
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
}
diff --git a/contrib/pgstattuple/pgstatapprox.c b/contrib/pgstattuple/pgstatapprox.c
index 1fe193b..147c8d2 100644
--- a/contrib/pgstattuple/pgstatapprox.c
+++ b/contrib/pgstattuple/pgstatapprox.c
@@ -314,5 +314,5 @@ pgstattuple_approx_internal(Oid relid, FunctionCallInfo fcinfo)
values[i++] = Float8GetDatum(stat.free_percent);
ret = heap_form_tuple(tupdesc, values, nulls);
- return HeapTupleGetDatum(ret);
+ return HeapTupleGetRawDatum(ret);
}
diff --git a/contrib/pgstattuple/pgstatindex.c b/contrib/pgstattuple/pgstatindex.c
index 5368bb3..7d74c31 100644
--- a/contrib/pgstattuple/pgstatindex.c
+++ b/contrib/pgstattuple/pgstatindex.c
@@ -362,7 +362,7 @@ pgstatindex_impl(Relation rel, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(TupleDescGetAttInMetadata(tupleDesc),
values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
}
return result;
@@ -571,7 +571,7 @@ pgstatginindex_internal(Oid relid, FunctionCallInfo fcinfo)
* Build and return the tuple
*/
tuple = heap_form_tuple(tupleDesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
return result;
}
@@ -727,7 +727,7 @@ pgstathashindex(PG_FUNCTION_ARGS)
values[7] = Float8GetDatum(free_percent);
tuple = heap_form_tuple(tupleDesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/* -------------------------------------------------
diff --git a/contrib/pgstattuple/pgstattuple.c b/contrib/pgstattuple/pgstattuple.c
index 21fdeff..70b8bf0 100644
--- a/contrib/pgstattuple/pgstattuple.c
+++ b/contrib/pgstattuple/pgstattuple.c
@@ -147,7 +147,7 @@ build_pgstattuple_type(pgstattuple_type *stat, FunctionCallInfo fcinfo)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- return HeapTupleGetDatum(tuple);
+ return HeapTupleGetRawDatum(tuple);
}
/* ----------
diff --git a/contrib/sslinfo/sslinfo.c b/contrib/sslinfo/sslinfo.c
index 30cae0b..53801e9 100644
--- a/contrib/sslinfo/sslinfo.c
+++ b/contrib/sslinfo/sslinfo.c
@@ -460,7 +460,7 @@ ssl_extension_info(PG_FUNCTION_ARGS)
/* Build tuple */
tuple = heap_form_tuple(fctx->tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
if (BIO_free(membuf) != 1)
elog(ERROR, "could not free OpenSSL BIO structure");
diff --git a/doc/src/sgml/xfunc.sgml b/doc/src/sgml/xfunc.sgml
index 41bcc5b..7707400 100644
--- a/doc/src/sgml/xfunc.sgml
+++ b/doc/src/sgml/xfunc.sgml
@@ -2923,14 +2923,18 @@ HeapTuple BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values)
<para>
Once you have built a tuple to return from your function, it
- must be converted into a <type>Datum</type>. Use:
+ must be converted into a <type>Datum</type>. Use one of the following:
<programlisting>
HeapTupleGetDatum(HeapTuple tuple)
+HeapTupleGetRawDatum(HeapTuple tuple)
</programlisting>
to convert a <structname>HeapTuple</structname> into a valid Datum. This
<type>Datum</type> can be returned directly if you intend to return
just a single row, or it can be used as the current return value
- in a set-returning function.
+ in a set-returning function. <literal>HeapTupleGetRawDatum</literal>
+ is more efficient and is preferred for new code, but the caller must
+ ensure that the tuple contains no external TOAST pointers. If this
+ is not certain, use <literal>HeapTupleGetDatum</literal>.
</para>
<para>
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 0b56b0f..06479f5 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -983,8 +983,6 @@ heap_expand_tuple(HeapTuple sourceTuple, TupleDesc tupleDesc)
Datum
heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
{
- HeapTupleHeader td;
-
/*
* If the tuple contains any external TOAST pointers, we have to inline
* those fields to meet the conventions for composite-type Datums.
@@ -993,11 +991,29 @@ heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc)
return toast_flatten_tuple_to_datum(tuple->t_data,
tuple->t_len,
tupleDesc);
+ else
+ return heap_copy_tuple_as_raw_datum(tuple, tupleDesc);
+}
+
+/* ----------------
+ * heap_copy_tuple_as_raw_datum
+ *
+ * copy a tuple as a composite-type Datum, but the input tuple should not
+ * contain any external data.
+ * ----------------
+ */
+Datum
+heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc)
+{
+ HeapTupleHeader td;
+
+ /* The tuple should not contains any external TOAST pointers. */
+ Assert(!HeapTupleHasExternal(tuple));
/*
- * Fast path for easy case: just make a palloc'd copy and insert the
- * correct composite-Datum header fields (since those may not be set if
- * the given tuple came from disk, rather than from heap_form_tuple).
+ * Just make a palloc'd copy and insert the correct composite-Datum
+ * header fields (since those may not be set if the given tuple came
+ * from disk, rather than from heap_form_tuple).
*/
td = (HeapTupleHeader) palloc(tuple->t_len);
memcpy((char *) td, (char *) tuple->t_data, tuple->t_len);
diff --git a/src/backend/access/transam/commit_ts.c b/src/backend/access/transam/commit_ts.c
index 48e8d66..015c841 100644
--- a/src/backend/access/transam/commit_ts.c
+++ b/src/backend/access/transam/commit_ts.c
@@ -469,7 +469,7 @@ pg_last_committed_xact(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -518,7 +518,7 @@ pg_xact_commit_timestamp_origin(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/access/transam/multixact.c b/src/backend/access/transam/multixact.c
index 1f9f1a1..b44066a 100644
--- a/src/backend/access/transam/multixact.c
+++ b/src/backend/access/transam/multixact.c
@@ -3399,7 +3399,7 @@ pg_get_multixact_members(PG_FUNCTION_ARGS)
multi->iter++;
pfree(values[0]);
- SRF_RETURN_NEXT(funccxt, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funccxt, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funccxt);
diff --git a/src/backend/access/transam/twophase.c b/src/backend/access/transam/twophase.c
index 6023e7c..3f48597 100644
--- a/src/backend/access/transam/twophase.c
+++ b/src/backend/access/transam/twophase.c
@@ -787,7 +787,7 @@ pg_prepared_xact(PG_FUNCTION_ARGS)
values[4] = ObjectIdGetDatum(proc->databaseId);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/access/transam/xlogfuncs.c b/src/backend/access/transam/xlogfuncs.c
index f363a4c..f60d6f1 100644
--- a/src/backend/access/transam/xlogfuncs.c
+++ b/src/backend/access/transam/xlogfuncs.c
@@ -487,7 +487,7 @@ pg_walfile_name_offset(PG_FUNCTION_ARGS)
*/
resultHeapTuple = heap_form_tuple(resultTupleDesc, values, isnull);
- result = HeapTupleGetDatum(resultHeapTuple);
+ result = HeapTupleGetRawDatum(resultHeapTuple);
PG_RETURN_DATUM(result);
}
diff --git a/src/backend/catalog/objectaddress.c b/src/backend/catalog/objectaddress.c
index 6d88b69..7cd62e7 100644
--- a/src/backend/catalog/objectaddress.c
+++ b/src/backend/catalog/objectaddress.c
@@ -2355,7 +2355,7 @@ pg_get_object_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4233,7 +4233,7 @@ pg_identify_object(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
@@ -4304,7 +4304,7 @@ pg_identify_object_as_address(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
/*
diff --git a/src/backend/commands/sequence.c b/src/backend/commands/sequence.c
index 0415df9..f566e1e 100644
--- a/src/backend/commands/sequence.c
+++ b/src/backend/commands/sequence.c
@@ -1834,7 +1834,7 @@ pg_sequence_parameters(PG_FUNCTION_ARGS)
ReleaseSysCache(pgstuple);
- return HeapTupleGetDatum(heap_form_tuple(tupdesc, values, isnull));
+ return HeapTupleGetRawDatum(heap_form_tuple(tupdesc, values, isnull));
}
/*
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index 6308286..e47ab1e 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -3169,8 +3169,13 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
{
/* Full conversion with attribute rearrangement needed */
result = execute_attr_map_tuple(&tmptup, op->d.convert_rowtype.map);
- /* Result already has appropriate composite-datum header fields */
- *op->resvalue = HeapTupleGetDatum(result);
+
+ /*
+ * Result already has appropriate composite-datum header fields. The
+ * input was a composite type so we aren't expecting to have to
+ * flatten any toasted fields so directly call HeapTupleGetRawDatum.
+ */
+ *op->resvalue = HeapTupleGetRawDatum(result);
}
else
{
@@ -3181,10 +3186,10 @@ ExecEvalConvertRowtype(ExprState *state, ExprEvalStep *op, ExprContext *econtext
* for this since it will both make the physical copy and insert the
* correct composite header fields. Note that we aren't expecting to
* have to flatten any toasted fields: the input was a composite
- * datum, so it shouldn't contain any. So heap_copy_tuple_as_datum()
- * is overkill here, but its check for external fields is cheap.
+ * datum, so it shouldn't contain any. So we can directly call
+ * heap_copy_tuple_as_raw_datum().
*/
- *op->resvalue = heap_copy_tuple_as_datum(&tmptup, outdesc);
+ *op->resvalue = heap_copy_tuple_as_raw_datum(&tmptup, outdesc);
}
}
diff --git a/src/backend/replication/slotfuncs.c b/src/backend/replication/slotfuncs.c
index 9817b44..f92628b 100644
--- a/src/backend/replication/slotfuncs.c
+++ b/src/backend/replication/slotfuncs.c
@@ -106,7 +106,7 @@ pg_create_physical_replication_slot(PG_FUNCTION_ARGS)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
@@ -205,7 +205,7 @@ pg_create_logical_replication_slot(PG_FUNCTION_ARGS)
memset(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
/* ok, slot is now fully created, mark it as persistent if needed */
if (!temporary)
@@ -689,7 +689,7 @@ pg_replication_slot_advance(PG_FUNCTION_ARGS)
nulls[1] = false;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
PG_RETURN_DATUM(result);
}
@@ -911,7 +911,7 @@ copy_replication_slot(FunctionCallInfo fcinfo, bool logical_slot)
nulls[1] = true;
tuple = heap_form_tuple(tupdesc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
ReplicationSlotRelease();
diff --git a/src/backend/replication/walreceiver.c b/src/backend/replication/walreceiver.c
index 8532296..f5425a3 100644
--- a/src/backend/replication/walreceiver.c
+++ b/src/backend/replication/walreceiver.c
@@ -1423,5 +1423,6 @@ pg_stat_get_wal_receiver(PG_FUNCTION_ARGS)
}
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
diff --git a/src/backend/statistics/mcv.c b/src/backend/statistics/mcv.c
index abbc1f1..115131c 100644
--- a/src/backend/statistics/mcv.c
+++ b/src/backend/statistics/mcv.c
@@ -1450,7 +1450,7 @@ pg_stats_ext_mcvlist_items(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->attinmeta->tupdesc, values, nulls);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/tsearch/wparser.c b/src/backend/tsearch/wparser.c
index 71882dc..633c90d 100644
--- a/src/backend/tsearch/wparser.c
+++ b/src/backend/tsearch/wparser.c
@@ -97,7 +97,7 @@ tt_process_call(FuncCallContext *funcctx)
values[2] = st->list[st->cur].descr;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
pfree(values[2]);
@@ -236,7 +236,7 @@ prs_process_call(FuncCallContext *funcctx)
sprintf(tid, "%d", st->list[st->cur].type);
values[1] = st->list[st->cur].lexeme;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[1]);
st->cur++;
diff --git a/src/backend/utils/adt/acl.c b/src/backend/utils/adt/acl.c
index c7f029e..5871c0b 100644
--- a/src/backend/utils/adt/acl.c
+++ b/src/backend/utils/adt/acl.c
@@ -1804,7 +1804,7 @@ aclexplode(PG_FUNCTION_ARGS)
MemSet(nulls, 0, sizeof(nulls));
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/datetime.c b/src/backend/utils/adt/datetime.c
index 350b0c5..ce11554 100644
--- a/src/backend/utils/adt/datetime.c
+++ b/src/backend/utils/adt/datetime.c
@@ -4776,7 +4776,7 @@ pg_timezone_abbrevs(PG_FUNCTION_ARGS)
(*pindex)++;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/genfile.c b/src/backend/utils/adt/genfile.c
index 169ddf8..d3d386b 100644
--- a/src/backend/utils/adt/genfile.c
+++ b/src/backend/utils/adt/genfile.c
@@ -454,7 +454,7 @@ pg_stat_file(PG_FUNCTION_ARGS)
pfree(filename);
- PG_RETURN_DATUM(HeapTupleGetDatum(tuple));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(tuple->t_data);
}
/*
diff --git a/src/backend/utils/adt/lockfuncs.c b/src/backend/utils/adt/lockfuncs.c
index 97f0265..0818449 100644
--- a/src/backend/utils/adt/lockfuncs.c
+++ b/src/backend/utils/adt/lockfuncs.c
@@ -344,7 +344,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
@@ -415,7 +415,7 @@ pg_lock_status(PG_FUNCTION_ARGS)
nulls[15] = true;
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/misc.c b/src/backend/utils/adt/misc.c
index 634f574..57e0ea0 100644
--- a/src/backend/utils/adt/misc.c
+++ b/src/backend/utils/adt/misc.c
@@ -484,7 +484,7 @@ pg_get_keywords(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
@@ -562,7 +562,7 @@ pg_get_catalog_foreign_keys(PG_FUNCTION_ARGS)
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
SRF_RETURN_DONE(funcctx);
diff --git a/src/backend/utils/adt/partitionfuncs.c b/src/backend/utils/adt/partitionfuncs.c
index 03660d5..6915939 100644
--- a/src/backend/utils/adt/partitionfuncs.c
+++ b/src/backend/utils/adt/partitionfuncs.c
@@ -159,7 +159,7 @@ pg_partition_tree(PG_FUNCTION_ARGS)
values[3] = Int32GetDatum(level);
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/adt/pgstatfuncs.c b/src/backend/utils/adt/pgstatfuncs.c
index 5102227..b93ad73 100644
--- a/src/backend/utils/adt/pgstatfuncs.c
+++ b/src/backend/utils/adt/pgstatfuncs.c
@@ -1856,7 +1856,8 @@ pg_stat_get_wal(PG_FUNCTION_ARGS)
values[8] = TimestampTzGetDatum(wal_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/*
@@ -2272,7 +2273,8 @@ pg_stat_get_archiver(PG_FUNCTION_ARGS)
values[6] = TimestampTzGetDatum(archiver_stats->stat_reset_timestamp);
/* Returns the record as Datum */
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
/* Get the statistics for the replication slots */
diff --git a/src/backend/utils/adt/rowtypes.c b/src/backend/utils/adt/rowtypes.c
index 23787a6..079a5af 100644
--- a/src/backend/utils/adt/rowtypes.c
+++ b/src/backend/utils/adt/rowtypes.c
@@ -293,7 +293,7 @@ record_in(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
@@ -660,7 +660,7 @@ record_recv(PG_FUNCTION_ARGS)
pfree(nulls);
ReleaseTupleDesc(tupdesc);
- PG_RETURN_HEAPTUPLEHEADER(result);
+ PG_RETURN_HEAPTUPLEHEADER_RAW(result);
}
/*
diff --git a/src/backend/utils/adt/tsvector_op.c b/src/backend/utils/adt/tsvector_op.c
index 9236ebc..6679493 100644
--- a/src/backend/utils/adt/tsvector_op.c
+++ b/src/backend/utils/adt/tsvector_op.c
@@ -707,7 +707,7 @@ tsvector_unnest(PG_FUNCTION_ARGS)
}
tuple = heap_form_tuple(funcctx->tuple_desc, values, nulls);
- SRF_RETURN_NEXT(funcctx, HeapTupleGetDatum(tuple));
+ SRF_RETURN_NEXT(funcctx, HeapTupleGetRawDatum(tuple));
}
else
{
@@ -2385,7 +2385,7 @@ ts_process_call(FuncCallContext *funcctx)
values[2] = nentry;
tuple = BuildTupleFromCStrings(funcctx->attinmeta, values);
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
pfree(values[0]);
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index b263e34..6b2d9d6 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -9805,7 +9805,7 @@ show_all_settings(PG_FUNCTION_ARGS)
tuple = BuildTupleFromCStrings(attinmeta, values);
/* make the tuple into a datum */
- result = HeapTupleGetDatum(tuple);
+ result = HeapTupleGetRawDatum(tuple);
SRF_RETURN_NEXT(funcctx, result);
}
diff --git a/src/backend/utils/misc/pg_controldata.c b/src/backend/utils/misc/pg_controldata.c
index 209a20a..195c127 100644
--- a/src/backend/utils/misc/pg_controldata.c
+++ b/src/backend/utils/misc/pg_controldata.c
@@ -73,7 +73,7 @@ pg_control_system(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -204,7 +204,7 @@ pg_control_checkpoint(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -257,7 +257,7 @@ pg_control_recovery(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
Datum
@@ -340,5 +340,5 @@ pg_control_init(PG_FUNCTION_ARGS)
htup = heap_form_tuple(tupdesc, values, nulls);
- PG_RETURN_DATUM(HeapTupleGetDatum(htup));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(htup->t_data);
}
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7c62852..7ac3c23 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -790,6 +790,7 @@ extern Datum getmissingattr(TupleDesc tupleDesc,
extern HeapTuple heap_copytuple(HeapTuple tuple);
extern void heap_copytuple_with_tuple(HeapTuple src, HeapTuple dest);
extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
+extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
diff --git a/src/include/fmgr.h b/src/include/fmgr.h
index ab7b85c..89eec72 100644
--- a/src/include/fmgr.h
+++ b/src/include/fmgr.h
@@ -372,7 +372,15 @@ extern struct varlena *pg_detoast_datum_packed(struct varlena *datum);
#define PG_RETURN_TEXT_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_BPCHAR_P(x) PG_RETURN_POINTER(x)
#define PG_RETURN_VARCHAR_P(x) PG_RETURN_POINTER(x)
+
+/*
+ * A composite-type Datum must not contain external fields. To return a tuple
+ * that might, use PG_RETURN_HEAPTUPLEHEADER, which will build a replacement
+ * tuple without such fields if any are present. Otherwise, use
+ * PG_RETURN_HEAPTUPLEHEADER_RAW, which is slightly faster.
+ */
#define PG_RETURN_HEAPTUPLEHEADER(x) return HeapTupleHeaderGetDatum(x)
+#define PG_RETURN_HEAPTUPLEHEADER_RAW(x) return HeapTupleHeaderGetRawDatum(x)
/*-------------------------------------------------------------------------
diff --git a/src/include/funcapi.h b/src/include/funcapi.h
index 83e6bc2..92cc7bd 100644
--- a/src/include/funcapi.h
+++ b/src/include/funcapi.h
@@ -207,6 +207,8 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*
* Macro declarations:
* HeapTupleGetDatum(HeapTuple tuple) - convert a HeapTuple to a Datum.
+ * HeapTupleGetRawDatum(HeapTuple tuple) - same, but must have no external
+ * TOAST pointer
*
* Obsolete routines and macros:
* TupleDesc RelationNameGetTupleDesc(const char *relname) - Use to get a
@@ -219,6 +221,7 @@ extern TupleDesc build_function_result_tupdesc_t(HeapTuple procTuple);
*/
#define HeapTupleGetDatum(tuple) HeapTupleHeaderGetDatum((tuple)->t_data)
+#define HeapTupleGetRawDatum(tuple) HeapTupleHeaderGetRawDatum((tuple)->t_data)
/* obsolete version of above */
#define TupleGetDatum(_slot, _tuple) HeapTupleGetDatum(_tuple)
@@ -231,6 +234,17 @@ extern AttInMetadata *TupleDescGetAttInMetadata(TupleDesc tupdesc);
extern HeapTuple BuildTupleFromCStrings(AttInMetadata *attinmeta, char **values);
extern Datum HeapTupleHeaderGetDatum(HeapTupleHeader tuple);
+/*
+ * This is a faster version of HeapTupleHeaderGetDatum which can be used
+ * when it is known that no external TOAST pointers are present.
+ */
+static inline Datum
+HeapTupleHeaderGetRawDatum(HeapTupleHeader tuple)
+{
+ Assert(!HeapTupleHeaderHasExternal(tuple));
+ return PointerGetDatum(tuple);
+}
+
/*----------
* Support for Set Returning Functions (SRFs)
diff --git a/src/pl/plperl/plperl.c b/src/pl/plperl/plperl.c
index 6299adf..cdf79f7 100644
--- a/src/pl/plperl/plperl.c
+++ b/src/pl/plperl/plperl.c
@@ -1127,7 +1127,7 @@ plperl_hash_to_datum(SV *src, TupleDesc td)
{
HeapTuple tup = plperl_build_tuple_result((HV *) SvRV(src), td);
- return HeapTupleGetDatum(tup);
+ return HeapTupleGetRawDatum(tup);
}
/*
@@ -3334,7 +3334,7 @@ plperl_return_next_internal(SV *sv)
current_call_data->ret_tdesc);
if (OidIsValid(current_call_data->cdomain_oid))
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
current_call_data->cdomain_oid,
¤t_call_data->cdomain_info,
rsi->econtext->ecxt_per_query_memory);
diff --git a/src/pl/tcl/pltcl.c b/src/pl/tcl/pltcl.c
index e118375..b4e710a 100644
--- a/src/pl/tcl/pltcl.c
+++ b/src/pl/tcl/pltcl.c
@@ -1024,7 +1024,7 @@ pltcl_func_handler(PG_FUNCTION_ARGS, pltcl_call_state *call_state,
tup = pltcl_build_tuple_result(interp, resultObjv, resultObjc,
call_state);
- retval = HeapTupleGetDatum(tup);
+ retval = HeapTupleGetRawDatum(tup);
}
else
retval = InputFunctionCall(&prodesc->result_in_func,
@@ -3235,7 +3235,7 @@ pltcl_build_tuple_result(Tcl_Interp *interp, Tcl_Obj **kvObjv, int kvObjc,
/* if result type is domain-over-composite, check domain constraints */
if (call_state->prodesc->fn_retisdomain)
- domain_check(HeapTupleGetDatum(tuple), false,
+ domain_check(HeapTupleGetRawDatum(tuple), false,
call_state->prodesc->result_typid,
&call_state->prodesc->domain_info,
call_state->prodesc->fn_cxt);
diff --git a/src/test/modules/test_predtest/test_predtest.c b/src/test/modules/test_predtest/test_predtest.c
index 9c0aadd..ec83645 100644
--- a/src/test/modules/test_predtest/test_predtest.c
+++ b/src/test/modules/test_predtest/test_predtest.c
@@ -214,5 +214,6 @@ test_predtest(PG_FUNCTION_ARGS)
values[6] = BoolGetDatum(s_r_holds);
values[7] = BoolGetDatum(w_r_holds);
- PG_RETURN_DATUM(HeapTupleGetDatum(heap_form_tuple(tupdesc, values, nulls)));
+ PG_RETURN_HEAPTUPLEHEADER_RAW(
+ heap_form_tuple(tupdesc, values, nulls)->t_data);
}
--
1.8.3.1
v39-0002-Expand-the-external-data-before-forming-the-tupl.patchtext/x-patch; charset=US-ASCII; name=v39-0002-Expand-the-external-data-before-forming-the-tupl.patchDownload
From e228f0876c14ded2ed46337f2f57c0704d6c491e Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Thu, 4 Mar 2021 14:55:56 +0530
Subject: [PATCH v39 2/4] Expand the external data before forming the tuple
All the callers of HeapTupleGetDatum and HeapTupleHeaderGetDatum, who might
contain the external varlena in their tuple, try to flatten the external
varlena before forming the tuple wherever it is possible.
Dilip Kumar, based on idea by Robert Haas
---
src/backend/access/common/heaptuple.c | 32 ++++++++++++++++++++++++++++++++
src/backend/executor/execExprInterp.c | 16 ++++++++--------
src/backend/utils/adt/jsonfuncs.c | 9 +++++++--
src/include/access/htup_details.h | 2 ++
src/pl/plpgsql/src/pl_exec.c | 19 ++++++++++++++-----
5 files changed, 63 insertions(+), 15 deletions(-)
diff --git a/src/backend/access/common/heaptuple.c b/src/backend/access/common/heaptuple.c
index 06479f5..99cc138 100644
--- a/src/backend/access/common/heaptuple.c
+++ b/src/backend/access/common/heaptuple.c
@@ -61,6 +61,7 @@
#include "access/sysattr.h"
#include "access/tupdesc_details.h"
#include "executor/tuptable.h"
+#include "fmgr.h"
#include "utils/expandeddatum.h"
@@ -1114,6 +1115,37 @@ heap_form_tuple(TupleDesc tupleDescriptor,
return tuple;
}
+
+/*
+ * heap_form_flattened_tuple
+ * wrapper over heap_form_tuple which flatten any external toast pointers
+ * before forming the tuple.
+ *
+ * As a side effect, any datum in the values array, which is stored as a
+ * external toast pointer will be flattened and replaced with a new value.
+ *
+ * The result is allocated in the current memory context.
+ */
+HeapTuple
+heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values,
+ bool *isnull)
+{
+ /* detoast any external data before forming the tuple */
+ for (int i = 0; i < tupleDescriptor->natts; i++)
+ {
+ Form_pg_attribute attr = TupleDescAttr(tupleDescriptor, i);
+
+ if (isnull[i] || attr->attlen != -1 ||
+ !VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ continue;
+
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
+ }
+
+ return heap_form_tuple(tupleDescriptor, values, isnull);
+}
+
/*
* heap_modify_tuple
* form a new tuple from an old tuple and a set of replacement values.
diff --git a/src/backend/executor/execExprInterp.c b/src/backend/executor/execExprInterp.c
index e47ab1e..bff6c2b 100644
--- a/src/backend/executor/execExprInterp.c
+++ b/src/backend/executor/execExprInterp.c
@@ -2841,11 +2841,11 @@ ExecEvalRow(ExprState *state, ExprEvalStep *op)
HeapTuple tuple;
/* build tuple from evaluated field values */
- tuple = heap_form_tuple(op->d.row.tupdesc,
- op->d.row.elemvalues,
- op->d.row.elemnulls);
+ tuple = heap_form_flattened_tuple(op->d.row.tupdesc,
+ op->d.row.elemvalues,
+ op->d.row.elemnulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
@@ -3086,11 +3086,11 @@ ExecEvalFieldStoreForm(ExprState *state, ExprEvalStep *op, ExprContext *econtext
HeapTuple tuple;
/* argdesc should already be valid from the DeForm step */
- tuple = heap_form_tuple(*op->d.fieldstore.argdesc,
- op->d.fieldstore.values,
- op->d.fieldstore.nulls);
+ tuple = heap_form_flattened_tuple(*op->d.fieldstore.argdesc,
+ op->d.fieldstore.values,
+ op->d.fieldstore.nulls);
- *op->resvalue = HeapTupleGetDatum(tuple);
+ *op->resvalue = HeapTupleGetRawDatum(tuple);
*op->resnull = false;
}
diff --git a/src/backend/utils/adt/jsonfuncs.c b/src/backend/utils/adt/jsonfuncs.c
index 5114672..dc3f9d4 100644
--- a/src/backend/utils/adt/jsonfuncs.c
+++ b/src/backend/utils/adt/jsonfuncs.c
@@ -2968,7 +2968,7 @@ populate_composite(CompositeIOData *io,
/* populate resulting record tuple */
tuple = populate_record(io->tupdesc, &io->record_io,
defaultval, mcxt, &jso);
- result = HeapTupleHeaderGetDatum(tuple);
+ result = HeapTupleHeaderGetRawDatum(tuple);
JsObjectFree(&jso);
}
@@ -3387,6 +3387,11 @@ populate_record(TupleDesc tupdesc,
nulls[i] ? (Datum) 0 : values[i],
&field,
&nulls[i]);
+
+ /* detoast any external data before forming the tuple */
+ if (!nulls[i] && att->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(values[i])))
+ values[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(values[i]));
}
res = heap_form_tuple(tupdesc, values, nulls);
@@ -3765,7 +3770,7 @@ populate_recordset_record(PopulateRecordsetState *state, JsObject *obj)
/* if it's domain over composite, check domain constraints */
if (cache->c.typcat == TYPECAT_COMPOSITE_DOMAIN)
- domain_check(HeapTupleHeaderGetDatum(tuphead), false,
+ domain_check(HeapTupleHeaderGetRawDatum(tuphead), false,
cache->argtype,
&cache->c.io.composite.domain_info,
cache->fn_mcxt);
diff --git a/src/include/access/htup_details.h b/src/include/access/htup_details.h
index 7ac3c23..be6641b 100644
--- a/src/include/access/htup_details.h
+++ b/src/include/access/htup_details.h
@@ -793,6 +793,8 @@ extern Datum heap_copy_tuple_as_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern Datum heap_copy_tuple_as_raw_datum(HeapTuple tuple, TupleDesc tupleDesc);
extern HeapTuple heap_form_tuple(TupleDesc tupleDescriptor,
Datum *values, bool *isnull);
+extern HeapTuple heap_form_flattened_tuple(TupleDesc tupleDescriptor,
+ Datum *values, bool *isnull);
extern HeapTuple heap_modify_tuple(HeapTuple tuple,
TupleDesc tupleDesc,
Datum *replValues,
diff --git a/src/pl/plpgsql/src/pl_exec.c b/src/pl/plpgsql/src/pl_exec.c
index b4c70aa..d4fcb70 100644
--- a/src/pl/plpgsql/src/pl_exec.c
+++ b/src/pl/plpgsql/src/pl_exec.c
@@ -404,7 +404,8 @@ static void exec_move_row_from_fields(PLpgSQL_execstate *estate,
static bool compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc);
static HeapTuple make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc);
+ TupleDesc tupdesc,
+ bool flatten);
static TupleDesc deconstruct_composite_datum(Datum value,
HeapTupleData *tmptup);
static void exec_move_row_from_datum(PLpgSQL_execstate *estate,
@@ -3418,7 +3419,7 @@ exec_stmt_return_next(PLpgSQL_execstate *estate,
/* Use eval_mcontext for tuple conversion work */
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tuple = make_tuple_from_row(estate, row, tupdesc);
+ tuple = make_tuple_from_row(estate, row, tupdesc, false);
if (tuple == NULL) /* should not happen */
ereport(ERROR,
(errcode(ERRCODE_DATATYPE_MISMATCH),
@@ -5321,12 +5322,12 @@ exec_eval_datum(PLpgSQL_execstate *estate,
/* Make sure we have a valid type/typmod setting */
BlessTupleDesc(row->rowtupdesc);
oldcontext = MemoryContextSwitchTo(get_eval_mcontext(estate));
- tup = make_tuple_from_row(estate, row, row->rowtupdesc);
+ tup = make_tuple_from_row(estate, row, row->rowtupdesc, true);
if (tup == NULL) /* should not happen */
elog(ERROR, "row not compatible with its own tupdesc");
*typeid = row->rowtupdesc->tdtypeid;
*typetypmod = row->rowtupdesc->tdtypmod;
- *value = HeapTupleGetDatum(tup);
+ *value = HeapTupleGetRawDatum(tup);
*isnull = false;
MemoryContextSwitchTo(oldcontext);
break;
@@ -7265,12 +7266,15 @@ compatible_tupdescs(TupleDesc src_tupdesc, TupleDesc dst_tupdesc)
*
* The result tuple is freshly palloc'd in caller's context. Some junk
* may be left behind in eval_mcontext, too.
+ *
+ * flatten - if this is passed true then any external data will be flattened.
* ----------
*/
static HeapTuple
make_tuple_from_row(PLpgSQL_execstate *estate,
PLpgSQL_row *row,
- TupleDesc tupdesc)
+ TupleDesc tupdesc,
+ bool flatten)
{
int natts = tupdesc->natts;
HeapTuple tuple;
@@ -7300,6 +7304,11 @@ make_tuple_from_row(PLpgSQL_execstate *estate,
&dvalues[i], &nulls[i]);
if (fieldtypeid != TupleDescAttr(tupdesc, i)->atttypid)
return NULL;
+
+ /* detoast any external data if the caller has asked to do so */
+ if (flatten && !nulls[i] && TupleDescAttr(tupdesc, i)->attlen == -1 &&
+ VARATT_IS_EXTERNAL(DatumGetPointer(dvalues[i])))
+ dvalues[i] = PointerGetDatum(PG_DETOAST_DATUM_PACKED(dvalues[i]));
/* XXX should we insist on typmod match, too? */
}
--
1.8.3.1
v39-0004-default-to-with-lz4.patchtext/x-patch; charset=US-ASCII; name=v39-0004-default-to-with-lz4.patchDownload
From 9ce979cb9afc6d8266cdded9c879db4e846743d4 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 10 Mar 2021 14:44:42 +0530
Subject: [PATCH v39 4/4] default to --with-lz4
this is meant to excercize the new feature in the CIs, and not meant to be merged
---
configure | 6 ++++--
configure.ac | 4 ++--
2 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/configure b/configure
index 8176e99..fd6d5b6 100755
--- a/configure
+++ b/configure
@@ -1575,7 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
- --with-lz4 build with LZ4 support
+ --without-lz4 build without LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -8598,7 +8598,9 @@ $as_echo "#define USE_LZ4 1" >>confdefs.h
esac
else
- with_lz4=no
+ with_lz4=yes
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
fi
diff --git a/configure.ac b/configure.ac
index 54efbb2..fe3bc98 100644
--- a/configure.ac
+++ b/configure.ac
@@ -990,8 +990,8 @@ AC_SUBST(with_zlib)
# LZ4
#
AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+PGAC_ARG_BOOL(with, lz4, yes, [build without LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build without LZ4 support. (--without-lz4)])])
AC_MSG_RESULT([$with_lz4])
AC_SUBST(with_lz4)
--
1.8.3.1
v39-0003-Built-in-compression-method.patchtext/x-patch; charset=US-ASCII; name=v39-0003-Built-in-compression-method.patchDownload
From b23338502a568cf4c7185ccd69d527f62492e511 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Fri, 5 Mar 2021 09:33:08 +0530
Subject: [PATCH v39 3/4] Built-in compression method
Add syntax allowing a compression method to be specified.
As of now there is only 2 option for build-in compression
method (pglz, lz4) which can be set while creating a table
or adding a new column. Also, add support for changing the
compression method associated with a column using alter
column set compression. There are only built-in methods so
we don't rewrite the table while altering the compression
method, so only the new tuples will be compressed with the
new compression method. VACUUM FULL/CLUSTER can be used
for recompressing the table data according to the current
compression method of the attribute. The default compression
method is pglz and there is a support for GUC to change the
default compression method.
Dilip Kumar, Justin Pryzby based on the patches from Ildus Kurbangaliev.
Design input from Robert Haas and Tomas Vondra.
Reviewed by Robert Haas, Justin Pryzby, Tomas Vondra and Alexander Korotkov
Discussions:
https://www.postgresql.org/message-id/20171213151818.75a20259@postgrespro.ru
https://www.postgresql.org/message-id/CA%2BTgmoaKDW1Oi9V%3Djc9hOGyf77NbkNEABuqgHD1Cq%3D%3D1QsOcxg%40mail.gmail.com
https://www.postgresql.org/message-id/CA%2BTgmobSDVgUage9qQ5P_%3DF_9jaMkCgyKxUQGtFQU7oN4kX-AA%40mail.gmail.com
https://www.postgresql.org/message-id/20201005160355.byp74sh3ejsv7wrj%40development
https://www.postgresql.org/message-id/CAFiTN-tzTTT2oqWdRGLv1dvvS5MC1W%2BLE%2B3bqWPJUZj4GnHOJg%40mail.gmail.com
---
configure | 170 ++++++++++++
configure.ac | 20 ++
contrib/amcheck/verify_heapam.c | 2 +-
doc/src/sgml/catalogs.sgml | 12 +
doc/src/sgml/ref/alter_table.sgml | 16 ++
doc/src/sgml/ref/create_table.sgml | 33 ++-
doc/src/sgml/ref/psql-ref.sgml | 11 +
src/backend/access/brin/brin_tuple.c | 5 +-
src/backend/access/common/Makefile | 1 +
src/backend/access/common/detoast.c | 76 +++---
src/backend/access/common/indextuple.c | 3 +-
src/backend/access/common/toast_compression.c | 324 ++++++++++++++++++++++
src/backend/access/common/toast_internals.c | 78 +++---
src/backend/access/common/tupdesc.c | 6 +
src/backend/access/heap/heapam_handler.c | 41 +++
src/backend/access/table/toast_helper.c | 5 +-
src/backend/bootstrap/bootstrap.c | 5 +
src/backend/catalog/genbki.pl | 3 +
src/backend/catalog/heap.c | 4 +
src/backend/catalog/index.c | 1 +
src/backend/catalog/toasting.c | 6 +
src/backend/commands/tablecmds.c | 313 ++++++++++++++++++---
src/backend/nodes/copyfuncs.c | 1 +
src/backend/nodes/equalfuncs.c | 1 +
src/backend/nodes/nodeFuncs.c | 2 +
src/backend/nodes/outfuncs.c | 1 +
src/backend/parser/gram.y | 35 ++-
src/backend/parser/parse_utilcmd.c | 9 +
src/backend/replication/logical/reorderbuffer.c | 2 +-
src/backend/utils/adt/varlena.c | 54 ++++
src/backend/utils/misc/guc.c | 12 +
src/backend/utils/misc/postgresql.conf.sample | 1 +
src/bin/pg_amcheck/t/004_verify_heapam.pl | 4 +-
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_dump.c | 39 +++
src/bin/pg_dump/pg_dump.h | 1 +
src/bin/pg_dump/t/002_pg_dump.pl | 12 +-
src/bin/psql/describe.c | 31 ++-
src/bin/psql/help.c | 2 +
src/bin/psql/settings.h | 1 +
src/bin/psql/startup.c | 10 +
src/bin/psql/tab-complete.c | 2 +-
src/include/access/detoast.h | 10 -
src/include/access/toast_compression.h | 123 +++++++++
src/include/access/toast_helper.h | 1 +
src/include/access/toast_internals.h | 23 +-
src/include/catalog/pg_attribute.h | 8 +-
src/include/catalog/pg_proc.dat | 4 +
src/include/nodes/parsenodes.h | 3 +
src/include/parser/kwlist.h | 1 +
src/include/pg_config.h.in | 3 +
src/include/postgres.h | 50 +++-
src/test/regress/expected/compression.out | 347 ++++++++++++++++++++++++
src/test/regress/expected/compression_1.out | 340 +++++++++++++++++++++++
src/test/regress/parallel_schedule | 2 +-
src/test/regress/pg_regress_main.c | 4 +-
src/test/regress/serial_schedule | 1 +
src/test/regress/sql/compression.sql | 136 ++++++++++
src/tools/msvc/Solution.pm | 1 +
59 files changed, 2256 insertions(+), 157 deletions(-)
create mode 100644 src/backend/access/common/toast_compression.c
create mode 100644 src/include/access/toast_compression.h
create mode 100644 src/test/regress/expected/compression.out
create mode 100644 src/test/regress/expected/compression_1.out
create mode 100644 src/test/regress/sql/compression.sql
diff --git a/configure b/configure
index 3fd4cec..8176e99 100755
--- a/configure
+++ b/configure
@@ -699,6 +699,9 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
+LZ4_LIBS
+LZ4_CFLAGS
+with_lz4
with_zlib
with_system_tzdata
with_libxslt
@@ -864,6 +867,7 @@ with_libxml
with_libxslt
with_system_tzdata
with_zlib
+with_lz4
with_gnu_ld
with_ssl
with_openssl
@@ -891,6 +895,8 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
+LZ4_CFLAGS
+LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
@@ -1569,6 +1575,7 @@ Optional Packages:
--with-system-tzdata=DIR
use system time zone data in DIR
--without-zlib do not use Zlib
+ --with-lz4 build with LZ4 support
--with-gnu-ld assume the C compiler uses GNU ld [default=no]
--with-ssl=LIB use LIB for SSL/TLS support (openssl)
--with-openssl obsolete spelling of --with-ssl=openssl
@@ -1596,6 +1603,8 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
+ LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
+ LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
@@ -8564,6 +8573,137 @@ fi
#
+# LZ4
+#
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
+$as_echo_n "checking whether to build with LZ4 support... " >&6; }
+
+
+
+# Check whether --with-lz4 was given.
+if test "${with_lz4+set}" = set; then :
+ withval=$with_lz4;
+ case $withval in
+ yes)
+
+$as_echo "#define USE_LZ4 1" >>confdefs.h
+
+ ;;
+ no)
+ :
+ ;;
+ *)
+ as_fn_error $? "no argument expected for --with-lz4 option" "$LINENO" 5
+ ;;
+ esac
+
+else
+ with_lz4=no
+
+fi
+
+
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
+$as_echo "$with_lz4" >&6; }
+
+
+if test "$with_lz4" = yes; then
+
+pkg_failed=no
+{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
+$as_echo_n "checking for liblz4... " >&6; }
+
+if test -n "$LZ4_CFLAGS"; then
+ pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+if test -n "$LZ4_LIBS"; then
+ pkg_cv_LZ4_LIBS="$LZ4_LIBS"
+ elif test -n "$PKG_CONFIG"; then
+ if test -n "$PKG_CONFIG" && \
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
+ ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
+ ac_status=$?
+ $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
+ test $ac_status = 0; }; then
+ pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
+ test "x$?" != "x0" && pkg_failed=yes
+else
+ pkg_failed=yes
+fi
+ else
+ pkg_failed=untried
+fi
+
+
+
+if test $pkg_failed = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+
+if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
+ _pkg_short_errors_supported=yes
+else
+ _pkg_short_errors_supported=no
+fi
+ if test $_pkg_short_errors_supported = yes; then
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
+ else
+ LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
+ fi
+ # Put the nasty error message in config.log where it belongs
+ echo "$LZ4_PKG_ERRORS" >&5
+
+ as_fn_error $? "Package requirements (liblz4) were not met:
+
+$LZ4_PKG_ERRORS
+
+Consider adjusting the PKG_CONFIG_PATH environment variable if you
+installed software in a non-standard prefix.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details." "$LINENO" 5
+elif test $pkg_failed = untried; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
+$as_echo "no" >&6; }
+ { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
+$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
+as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
+is in your PATH or set the PKG_CONFIG environment variable to the full
+path to pkg-config.
+
+Alternatively, you may set the environment variables LZ4_CFLAGS
+and LZ4_LIBS to avoid the need to call pkg-config.
+See the pkg-config man page for more details.
+
+To get pkg-config, see <http://pkg-config.freedesktop.org/>.
+See \`config.log' for more details" "$LINENO" 5; }
+else
+ LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
+ LZ4_LIBS=$pkg_cv_LZ4_LIBS
+ { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
+$as_echo "yes" >&6; }
+
+fi
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -13381,6 +13521,36 @@ fi
fi
+if test "$with_lz4" = yes; then
+ for ac_header in lz4/lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_LZ4_H 1
+_ACEOF
+
+else
+ for ac_header in lz4.h
+do :
+ ac_fn_c_check_header_mongrel "$LINENO" "lz4.h" "ac_cv_header_lz4_h" "$ac_includes_default"
+if test "x$ac_cv_header_lz4_h" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LZ4_H 1
+_ACEOF
+
+else
+ as_fn_error $? "lz4.h header file is required for LZ4" "$LINENO" 5
+fi
+
+done
+
+fi
+
+done
+
+fi
+
if test "$with_gssapi" = yes ; then
for ac_header in gssapi/gssapi.h
do :
diff --git a/configure.ac b/configure.ac
index 2f1585a..54efbb2 100644
--- a/configure.ac
+++ b/configure.ac
@@ -987,6 +987,21 @@ PGAC_ARG_BOOL(with, zlib, yes,
AC_SUBST(with_zlib)
#
+# LZ4
+#
+AC_MSG_CHECKING([whether to build with LZ4 support])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
+ [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
+AC_MSG_RESULT([$with_lz4])
+AC_SUBST(with_lz4)
+
+if test "$with_lz4" = yes; then
+ PKG_CHECK_MODULES(LZ4, liblz4)
+ LIBS="$LZ4_LIBS $LIBS"
+ CFLAGS="$LZ4_CFLAGS $CFLAGS"
+fi
+
+#
# Assignments
#
@@ -1410,6 +1425,11 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+fi
+
if test "$with_gssapi" = yes ; then
AC_CHECK_HEADERS(gssapi/gssapi.h, [],
[AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
diff --git a/contrib/amcheck/verify_heapam.c b/contrib/amcheck/verify_heapam.c
index e614c12..6f972e6 100644
--- a/contrib/amcheck/verify_heapam.c
+++ b/contrib/amcheck/verify_heapam.c
@@ -1069,7 +1069,7 @@ check_tuple_attribute(HeapCheckContext *ctx)
*/
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- ctx->attrsize = toast_pointer.va_extsize;
+ ctx->attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
ctx->endchunk = (ctx->attrsize - 1) / TOAST_MAX_CHUNK_SIZE;
ctx->totalchunks = ctx->endchunk + 1;
diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml
index 5c9f4af..68d1960 100644
--- a/doc/src/sgml/catalogs.sgml
+++ b/doc/src/sgml/catalogs.sgml
@@ -1357,6 +1357,18 @@
<row>
<entry role="catalog_table_entry"><para role="column_definition">
+ <structfield>attcompression</structfield> <type>char</type>
+ </para>
+ <para>
+ The current compression method of the column. If it is an invalid
+ compression method (<literal>'\0'</literal>) then column data will not
+ be compressed. Otherwise, <literal>'p'</literal> = pglz compression or
+ <literal>'l'</literal> = lz4 compression.
+ </para></entry>
+ </row>
+
+ <row>
+ <entry role="catalog_table_entry"><para role="column_definition">
<structfield>attacl</structfield> <type>aclitem[]</type>
</para>
<para>
diff --git a/doc/src/sgml/ref/alter_table.sgml b/doc/src/sgml/ref/alter_table.sgml
index f82dce4..77adc9c 100644
--- a/doc/src/sgml/ref/alter_table.sgml
+++ b/doc/src/sgml/ref/alter_table.sgml
@@ -54,6 +54,7 @@ ALTER TABLE [ IF EXISTS ] <replaceable class="parameter">name</replaceable>
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET ( <replaceable class="parameter">attribute_option</replaceable> = <replaceable class="parameter">value</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> RESET ( <replaceable class="parameter">attribute_option</replaceable> [, ... ] )
ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET STORAGE { PLAIN | EXTERNAL | EXTENDED | MAIN }
+ ALTER [ COLUMN ] <replaceable class="parameter">column_name</replaceable> SET COMPRESSION <replaceable class="parameter">compression_method</replaceable>
ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]
ADD <replaceable class="parameter">table_constraint_using_index</replaceable>
ALTER CONSTRAINT <replaceable class="parameter">constraint_name</replaceable> [ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -103,6 +104,7 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
GENERATED { ALWAYS | BY DEFAULT } AS IDENTITY [ ( <replaceable>sequence_options</replaceable> ) ] |
UNIQUE <replaceable class="parameter">index_parameters</replaceable> |
PRIMARY KEY <replaceable class="parameter">index_parameters</replaceable> |
+ COMPRESSION <replaceable class="parameter">compression_method</replaceable> |
REFERENCES <replaceable class="parameter">reftable</replaceable> [ ( <replaceable class="parameter">refcolumn</replaceable> ) ] [ MATCH FULL | MATCH PARTIAL | MATCH SIMPLE ]
[ ON DELETE <replaceable class="parameter">referential_action</replaceable> ] [ ON UPDATE <replaceable class="parameter">referential_action</replaceable> ] }
[ DEFERRABLE | NOT DEFERRABLE ] [ INITIALLY DEFERRED | INITIALLY IMMEDIATE ]
@@ -384,6 +386,20 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term>
+ <literal>SET COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal>
+ </term>
+ <listitem>
+ <para>
+ This sets the compression method for a column. The supported compression
+ methods are <literal>pglz</literal> and <literal>lz4</literal>.
+ <literal>lz4</literal> is available only if <literal>--with-lz4</literal>
+ was used when building <productname>PostgreSQL</productname>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>ADD <replaceable class="parameter">table_constraint</replaceable> [ NOT VALID ]</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/create_table.sgml b/doc/src/sgml/ref/create_table.sgml
index ff1b642..a731239 100644
--- a/doc/src/sgml/ref/create_table.sgml
+++ b/doc/src/sgml/ref/create_table.sgml
@@ -22,7 +22,7 @@ PostgreSQL documentation
<refsynopsisdiv>
<synopsis>
CREATE [ [ GLOBAL | LOCAL ] { TEMPORARY | TEMP } | UNLOGGED ] TABLE [ IF NOT EXISTS ] <replaceable class="parameter">table_name</replaceable> ( [
- { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
+ { <replaceable class="parameter">column_name</replaceable> <replaceable class="parameter">data_type</replaceable> [ COLLATE <replaceable>collation</replaceable> ] [ COMPRESSION <replaceable>compression_method</replaceable> ] [ <replaceable class="parameter">column_constraint</replaceable> [ ... ] ]
| <replaceable>table_constraint</replaceable>
| LIKE <replaceable>source_table</replaceable> [ <replaceable>like_option</replaceable> ... ] }
[, ... ]
@@ -289,6 +289,26 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>COMPRESSION <replaceable class="parameter">compression_method</replaceable></literal></term>
+ <listitem>
+ <para>
+ The <literal>COMPRESSION</literal> clause sets the compression method
+ for a column. Compression is supported only for variable-width data
+ types, and is used only for columns whose storage type is main or
+ extended. (See <xref linkend="sql-altertable"/> for information on
+ column storage types.) Setting this property for a partitioned table
+ has no direct effect, because such tables have no storage of their own,
+ but the configured value is inherited by newly-created partitions.
+ The supported compression methods are <literal>pglz</literal> and
+ <literal>lz4</literal>. <literal>lz4</literal> is available only if
+ <literal>--with-lz4</literal> was used when building
+ <productname>PostgreSQL</productname>. The default
+ is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INHERITS ( <replaceable>parent_table</replaceable> [, ... ] )</literal></term>
<listitem>
<para>
@@ -606,6 +626,17 @@ WITH ( MODULUS <replaceable class="parameter">numeric_literal</replaceable>, REM
</varlistentry>
<varlistentry>
+ <term><literal>INCLUDING COMPRESSION</literal></term>
+ <listitem>
+ <para>
+ Compression method of the columns will be copied. The default
+ behavior is to exclude compression methods, resulting in columns
+ having the default compression method.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><literal>INCLUDING CONSTRAINTS</literal></term>
<listitem>
<para>
diff --git a/doc/src/sgml/ref/psql-ref.sgml b/doc/src/sgml/ref/psql-ref.sgml
index 13c1edf..01ec9b8 100644
--- a/doc/src/sgml/ref/psql-ref.sgml
+++ b/doc/src/sgml/ref/psql-ref.sgml
@@ -3864,6 +3864,17 @@ bar
</varlistentry>
<varlistentry>
+ <term><varname>HIDE_TOAST_COMPRESSION</varname></term>
+ <listitem>
+ <para>
+ If this variable is set to <literal>true</literal>, column
+ compression method details are not displayed. This is mainly
+ useful for regression tests.
+ </para>
+ </listitem>
+ </varlistentry>
+
+ <varlistentry>
<term><varname>HIDE_TABLEAM</varname></term>
<listitem>
<para>
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index a7eb1c9..0ab5712 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,7 +213,10 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(value);
+ Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
+ keyno);
+ Datum cvalue = toast_compress_datum(value,
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/Makefile b/src/backend/access/common/Makefile
index 5a007d6..b9aff0c 100644
--- a/src/backend/access/common/Makefile
+++ b/src/backend/access/common/Makefile
@@ -25,6 +25,7 @@ OBJS = \
scankey.o \
session.o \
syncscan.o \
+ toast_compression.o \
toast_internals.o \
tupconvert.o \
tupdesc.o
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index d1cdbaf..ef2a378 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -240,14 +240,19 @@ detoast_attr_slice(struct varlena *attr,
*/
if (slicelimit >= 0)
{
- int32 max_size;
+ int32 max_size = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
/*
* Determine maximum amount of compressed data needed for a prefix
* of a given length (after decompression).
+ *
+ * XXX as of now, for lz4, fetch complete compressed data as we
+ * don't have mechanism to compute the size of the max compressed
+ * data for decompressing the slice.
*/
- max_size = pglz_maximum_compressed_size(slicelimit,
- toast_pointer.va_extsize);
+ if (VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer) ==
+ TOAST_PGLZ_COMPRESSION_ID)
+ max_size = pglz_maximum_compressed_size(slicelimit, max_size);
/*
* Fetch enough compressed slices (compressed marker will get set
@@ -347,7 +352,7 @@ toast_fetch_datum(struct varlena *attr)
/* Must copy to access aligned fields */
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
result = (struct varlena *) palloc(attrsize + VARHDRSZ);
@@ -408,7 +413,7 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
*/
Assert(!VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) || 0 == sliceoffset);
- attrsize = toast_pointer.va_extsize;
+ attrsize = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
if (sliceoffset >= attrsize)
{
@@ -418,8 +423,8 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
/*
* When fetching a prefix of a compressed external datum, account for the
- * rawsize tracking amount of raw data, which is stored at the beginning
- * as an int32 value).
+ * space required by va_tcinfo, which is stored at the beginning as an
+ * int32 value.
*/
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) && slicelength > 0)
slicelength = slicelength + sizeof(int32);
@@ -464,21 +469,24 @@ toast_fetch_datum_slice(struct varlena *attr, int32 sliceoffset,
static struct varlena *
toast_decompress_datum(struct varlena *attr)
{
- struct varlena *result;
+ ToastCompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *)
- palloc(TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
- SET_VARSIZE(result, TOAST_COMPRESS_RAWSIZE(attr) + VARHDRSZ);
-
- if (pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- TOAST_COMPRESS_SIZE(attr),
- VARDATA(result),
- TOAST_COMPRESS_RAWSIZE(attr), true) < 0)
- elog(ERROR, "compressed data is corrupted");
-
- return result;
+ /*
+ * Fetch the compression method id stored in the compression header and
+ * decompress the data using the respective decompression routine.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return pglz_decompress_datum(attr);
+ case TOAST_LZ4_COMPRESSION_ID:
+ return lz4_decompress_datum(attr);
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
}
@@ -492,22 +500,24 @@ toast_decompress_datum(struct varlena *attr)
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
{
- struct varlena *result;
- int32 rawsize;
+ ToastCompressionId cmid;
Assert(VARATT_IS_COMPRESSED(attr));
- result = (struct varlena *) palloc(slicelength + VARHDRSZ);
-
- rawsize = pglz_decompress(TOAST_COMPRESS_RAWDATA(attr),
- VARSIZE(attr) - TOAST_COMPRESS_HDRSZ,
- VARDATA(result),
- slicelength, false);
- if (rawsize < 0)
- elog(ERROR, "compressed data is corrupted");
-
- SET_VARSIZE(result, rawsize + VARHDRSZ);
- return result;
+ /*
+ * Fetch the compression method id stored in the compression header and
+ * decompress the data slice using the respective decompression routine.
+ */
+ cmid = TOAST_COMPRESS_METHOD(attr);
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ return pglz_decompress_datum_slice(attr, slicelength);
+ case TOAST_LZ4_COMPRESSION_ID:
+ return lz4_decompress_datum_slice(attr, slicelength);
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
}
/* ----------
@@ -589,7 +599,7 @@ toast_datum_size(Datum value)
struct varatt_external toast_pointer;
VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
- result = toast_pointer.va_extsize;
+ result = VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer);
}
else if (VARATT_IS_EXTERNAL_INDIRECT(attr))
{
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index b72a138..1f6b7b7 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,7 +103,8 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i]);
+ Datum cvalue = toast_compress_datum(untoasted_values[i],
+ att->attcompression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
new file mode 100644
index 0000000..6933812
--- /dev/null
+++ b/src/backend/access/common/toast_compression.c
@@ -0,0 +1,324 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.c
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ *
+ * IDENTIFICATION
+ * src/backend/access/common/toast_compression.c
+ *
+ *-------------------------------------------------------------------------
+ */
+#include "postgres.h"
+
+#ifdef USE_LZ4
+#include <lz4.h>
+#endif
+
+#include "access/detoast.h"
+#include "access/toast_compression.h"
+#include "common/pg_lzcompress.h"
+#include "fmgr.h"
+#include "utils/builtins.h"
+
+/* Compile-time default */
+char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+
+/*
+ * pglz_compress_datum - compression routine for pglz.
+ *
+ * Compresses source into dest using the default strategy. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+pglz_compress_datum(const struct varlena *value)
+{
+ int32 valsize,
+ len;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
+ /*
+ * No point in wasting a palloc cycle if value size is outside the allowed
+ * range for compression.
+ */
+ if (valsize < PGLZ_strategy_default->min_input_size ||
+ valsize > PGLZ_strategy_default->max_input_size)
+ return NULL;
+
+ /*
+ * Figure out the maximum possible size of the pglz output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
+ VARHDRSZ_COMPRESS);
+
+ len = pglz_compress(VARDATA_ANY(value),
+ valsize,
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ NULL);
+ if (len < 0)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+}
+
+/*
+ * pglz_decompress_datum - decompression routine for pglz.
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_decompress_datum(const struct varlena *value)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARRAWSIZE_4B_C(value), true);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * pglz_decompress_datum_slice - slice decompression routine for pglz.
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+pglz_decompress_datum_slice(const struct varlena *value,
+ int32 slicelength)
+{
+ struct varlena *result;
+ int32 rawsize;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = pglz_decompress((char *) value + VARHDRSZ_COMPRESS,
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ slicelength, false);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed pglz data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+}
+
+/*
+ * lz4_compress_datum - compression routine for lz4.
+ *
+ * Compresses source into dest using the LZ4 defaults. Returns the
+ * compressed varlena, or NULL if compression fails.
+ */
+struct varlena *
+lz4_compress_datum(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 valsize;
+ int32 len;
+ int32 max_size;
+ struct varlena *tmp = NULL;
+
+ valsize = VARSIZE_ANY_EXHDR(value);
+
+ /*
+ * Figure out the maximum possible size of the LZ4 output, add the bytes
+ * that will be needed for varlena overhead, and allocate that amount.
+ */
+ max_size = LZ4_compressBound(valsize);
+ tmp = (struct varlena *) palloc(max_size + VARHDRSZ_COMPRESS);
+
+ len = LZ4_compress_default(VARDATA_ANY(value),
+ (char *) tmp + VARHDRSZ_COMPRESS,
+ valsize, max_size);
+ if (len <= 0)
+ elog(ERROR, "lz4 compression failed");
+
+ /* data is incompressible so just free the memory and return NULL */
+ if (len > valsize)
+ {
+ pfree(tmp);
+ return NULL;
+ }
+
+ SET_VARSIZE_COMPRESSED(tmp, len + VARHDRSZ_COMPRESS);
+
+ return tmp;
+#endif
+}
+
+/*
+ * lz4_decompress_datum - decompression routine for lz4.
+ *
+ * Returns the decompressed varlena.
+ */
+struct varlena *
+lz4_decompress_datum(const struct varlena *value)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(VARRAWSIZE_4B_C(value) + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ VARRAWSIZE_4B_C(value));
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/*
+ * lz4_decompress_datum_slice - slice decompression routine for lz4
+ *
+ * Decompresses part of the data. Returns the decompressed varlena.
+ */
+struct varlena *
+lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
+{
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#else
+ int32 rawsize;
+ struct varlena *result;
+
+ /* slice decompression not supported prior to 1.8.3 */
+ if (LZ4_versionNumber() < 10803)
+ return lz4_decompress_datum(value);
+
+ /* allocate memory for the uncompressed data */
+ result = (struct varlena *) palloc(slicelength + VARHDRSZ);
+
+ /* decompress the data */
+ rawsize = LZ4_decompress_safe_partial((char *) value + VARHDRSZ_COMPRESS,
+ VARDATA(result),
+ VARSIZE(value) - VARHDRSZ_COMPRESS,
+ slicelength,
+ slicelength);
+ if (rawsize < 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATA_CORRUPTED),
+ errmsg_internal("compressed lz4 data is corrupt")));
+
+ SET_VARSIZE(result, rawsize + VARHDRSZ);
+
+ return result;
+#endif
+}
+
+/* ----------
+ * toast_get_compression_id - get compression id from compressed data
+ *
+ * Returns compression method id stored in the compressed data. Otherwise,
+ * returns TOAST_INVALID_COMPRESSION_ID for the uncompressed data.
+ */
+ToastCompressionId
+toast_get_compression_id(struct varlena *attr)
+{
+ ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID;
+
+ /*
+ * If it is stored externally then fetch the compression method id from the
+ * external toast pointer. Otherwise, for the inline compressed data fetch
+ * it from the toast compression header.
+ */
+ if (VARATT_IS_EXTERNAL_ONDISK(attr))
+ {
+ struct varatt_external toast_pointer;
+
+ VARATT_EXTERNAL_GET_POINTER(toast_pointer, attr);
+
+ if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
+ cmid = VARATT_EXTERNAL_GET_COMPRESSION(toast_pointer);
+ }
+ else if (VARATT_IS_COMPRESSED(attr))
+ cmid = VARCOMPRESS_4B_C(attr);
+
+ return cmid;
+}
+
+/*
+ * check_default_toast_compression - validate new default_toast_compression
+ */
+bool
+check_default_toast_compression(char **newval, void **extra, GucSource source)
+{
+ if (**newval == '\0')
+ {
+ GUC_check_errdetail("%s cannot be empty.",
+ "default_toast_compression");
+ return false;
+ }
+
+ if (strlen(*newval) >= NAMEDATALEN)
+ {
+ GUC_check_errdetail("%s is too long (maximum %d characters).",
+ "default_toast_compression", NAMEDATALEN - 1);
+ return false;
+ }
+
+ if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+ {
+ /*
+ * When source == PGC_S_TEST, don't throw a hard error for a
+ * nonexistent compression method, only a NOTICE. See comments in
+ * guc.h.
+ */
+ if (source == PGC_S_TEST)
+ {
+ ereport(NOTICE,
+ (errcode(ERRCODE_UNDEFINED_OBJECT),
+ errmsg("compression method \"%s\" does not exist",
+ *newval)));
+ }
+ else
+ {
+ GUC_check_errdetail("Compression method \"%s\" does not exist.",
+ *newval);
+ return false;
+ }
+ }
+
+ return true;
+}
\ No newline at end of file
diff --git a/src/backend/access/common/toast_internals.c b/src/backend/access/common/toast_internals.c
index 9b9da0f..32c31d5 100644
--- a/src/backend/access/common/toast_internals.c
+++ b/src/backend/access/common/toast_internals.c
@@ -44,46 +44,55 @@ static bool toastid_valueid_exists(Oid toastrelid, Oid valueid);
* ----------
*/
Datum
-toast_compress_datum(Datum value)
+toast_compress_datum(Datum value, char cmethod)
{
- struct varlena *tmp;
- int32 valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
- int32 len;
+ struct varlena *tmp = NULL;
+ int32 valsize;
+ ToastCompressionId cmid = TOAST_INVALID_COMPRESSION_ID;
Assert(!VARATT_IS_EXTERNAL(DatumGetPointer(value)));
Assert(!VARATT_IS_COMPRESSED(DatumGetPointer(value)));
+ Assert(CompressionMethodIsValid(cmethod));
+
+ valsize = VARSIZE_ANY_EXHDR(DatumGetPointer(value));
+
/*
- * No point in wasting a palloc cycle if value size is out of the allowed
- * range for compression
+ * Call respective compression routine for the compression method and also
+ * set the size and compression method id in the compressed data header.
*/
- if (valsize < PGLZ_strategy_default->min_input_size ||
- valsize > PGLZ_strategy_default->max_input_size)
- return PointerGetDatum(NULL);
+ switch (cmethod)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ tmp = pglz_compress_datum((const struct varlena *) value);
+ cmid = TOAST_PGLZ_COMPRESSION_ID;
+ break;
+ case TOAST_LZ4_COMPRESSION:
+ tmp = lz4_compress_datum((const struct varlena *) value);
+ cmid = TOAST_LZ4_COMPRESSION_ID;
+ break;
+ default:
+ elog(ERROR, "invalid compression method %c", cmethod);
+ }
- tmp = (struct varlena *) palloc(PGLZ_MAX_OUTPUT(valsize) +
- TOAST_COMPRESS_HDRSZ);
+ if (tmp == NULL)
+ return PointerGetDatum(NULL);
/*
- * We recheck the actual size even if pglz_compress() reports success,
- * because it might be satisfied with having saved as little as one byte
- * in the compressed data --- which could turn into a net loss once you
- * consider header and alignment padding. Worst case, the compressed
- * format might require three padding bytes (plus header, which is
- * included in VARSIZE(tmp)), whereas the uncompressed format would take
- * only one header byte and no padding if the value is short enough. So
- * we insist on a savings of more than 2 bytes to ensure we have a gain.
+ * We recheck the actual size even if compression reports success, because
+ * it might be satisfied with having saved as little as one byte in the
+ * compressed data --- which could turn into a net loss once you consider
+ * header and alignment padding. Worst case, the compressed format might
+ * require three padding bytes (plus header, which is included in
+ * VARSIZE(tmp)), whereas the uncompressed format would take only one
+ * header byte and no padding if the value is short enough. So we insist
+ * on a savings of more than 2 bytes to ensure we have a gain.
*/
- len = pglz_compress(VARDATA_ANY(DatumGetPointer(value)),
- valsize,
- TOAST_COMPRESS_RAWDATA(tmp),
- PGLZ_strategy_default);
- if (len >= 0 &&
- len + TOAST_COMPRESS_HDRSZ < valsize - 2)
+ if (VARSIZE(tmp) < valsize - 2)
{
- TOAST_COMPRESS_SET_RAWSIZE(tmp, valsize);
- SET_VARSIZE_COMPRESSED(tmp, len + TOAST_COMPRESS_HDRSZ);
/* successful compression */
+ Assert(cmid != TOAST_INVALID_COMPRESSION_ID);
+ TOAST_COMPRESS_SET_SIZE_AND_METHOD(tmp, valsize, cmid);
return PointerGetDatum(tmp);
}
else
@@ -152,19 +161,21 @@ toast_save_datum(Relation rel, Datum value,
&num_indexes);
/*
- * Get the data pointer and length, and compute va_rawsize and va_extsize.
+ * Get the data pointer and length, and compute va_rawsize and va_extinfo.
*
* va_rawsize is the size of the equivalent fully uncompressed datum, so
* we have to adjust for short headers.
*
- * va_extsize is the actual size of the data payload in the toast records.
+ * va_extinfo stored the actual size of the data payload in the toast
+ * records and the compression method in first 2 bits if data is
+ * compressed.
*/
if (VARATT_IS_SHORT(dval))
{
data_p = VARDATA_SHORT(dval);
data_todo = VARSIZE_SHORT(dval) - VARHDRSZ_SHORT;
toast_pointer.va_rawsize = data_todo + VARHDRSZ; /* as if not short */
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
else if (VARATT_IS_COMPRESSED(dval))
{
@@ -172,7 +183,10 @@ toast_save_datum(Relation rel, Datum value,
data_todo = VARSIZE(dval) - VARHDRSZ;
/* rawsize in a compressed datum is just the size of the payload */
toast_pointer.va_rawsize = VARRAWSIZE_4B_C(dval) + VARHDRSZ;
- toast_pointer.va_extsize = data_todo;
+
+ /* set external size and compression method */
+ VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, data_todo,
+ VARCOMPRESS_4B_C(dval));
/* Assert that the numbers look like it's compressed */
Assert(VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer));
}
@@ -181,7 +195,7 @@ toast_save_datum(Relation rel, Datum value,
data_p = VARDATA(dval);
data_todo = VARSIZE(dval) - VARHDRSZ;
toast_pointer.va_rawsize = VARSIZE(dval);
- toast_pointer.va_extsize = data_todo;
+ toast_pointer.va_extinfo = data_todo;
}
/*
diff --git a/src/backend/access/common/tupdesc.c b/src/backend/access/common/tupdesc.c
index 902f594..cb76465 100644
--- a/src/backend/access/common/tupdesc.c
+++ b/src/backend/access/common/tupdesc.c
@@ -20,6 +20,7 @@
#include "postgres.h"
#include "access/htup_details.h"
+#include "access/toast_compression.h"
#include "access/tupdesc_details.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
@@ -664,6 +665,11 @@ TupleDescInitEntry(TupleDesc desc,
att->attstorage = typeForm->typstorage;
att->attcollation = typeForm->typcollation;
+ if (IsStorageCompressible(typeForm->typstorage))
+ att->attcompression = GetDefaultToastCompression();
+ else
+ att->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
}
diff --git a/src/backend/access/heap/heapam_handler.c b/src/backend/access/heap/heapam_handler.c
index bd5faf0..6bbe18e 100644
--- a/src/backend/access/heap/heapam_handler.c
+++ b/src/backend/access/heap/heapam_handler.c
@@ -19,6 +19,7 @@
*/
#include "postgres.h"
+#include "access/detoast.h"
#include "access/genam.h"
#include "access/heapam.h"
#include "access/heaptoast.h"
@@ -26,6 +27,7 @@
#include "access/rewriteheap.h"
#include "access/syncscan.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/tsmapi.h"
#include "access/xact.h"
#include "catalog/catalog.h"
@@ -2469,6 +2471,45 @@ reform_and_rewrite_tuple(HeapTuple tuple,
{
if (TupleDescAttr(newTupDesc, i)->attisdropped)
isnull[i] = true;
+
+ /*
+ * Since we are rewriting the table, use this opportunity to recompress
+ * any compressed data with current compression method of the
+ * attribute. Basically, if the compression method of the compressed
+ * varlena is not same as current compression method of the attribute
+ * then decompress it so that if it need to be compressed then it will
+ * be compressed with the current compression method of the attribute.
+ */
+ else if (!isnull[i] && TupleDescAttr(newTupDesc, i)->attlen == -1)
+ {
+ struct varlena *new_value;
+ ToastCompressionId cmid;
+ char cmethod;
+
+ new_value = (struct varlena *) DatumGetPointer(values[i]);
+ cmid = toast_get_compression_id(new_value);
+
+ /* nothing to be done for uncompressed data */
+ if (cmid == TOAST_INVALID_COMPRESSION_ID)
+ continue;
+
+ /* convert compression id to compression method */
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ cmethod = TOAST_PGLZ_COMPRESSION;
+ break;
+ case TOAST_LZ4_COMPRESSION_ID:
+ cmethod = TOAST_LZ4_COMPRESSION;
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+
+ /* if compression method doesn't match then detoast the value */
+ if (TupleDescAttr(newTupDesc, i)->attcompression != cmethod)
+ values[i] = PointerGetDatum(detoast_attr(new_value));
+ }
}
copiedTuple = heap_form_tuple(newTupDesc, values, isnull);
diff --git a/src/backend/access/table/toast_helper.c b/src/backend/access/table/toast_helper.c
index fb36151..53f78f9 100644
--- a/src/backend/access/table/toast_helper.c
+++ b/src/backend/access/table/toast_helper.c
@@ -54,6 +54,7 @@ toast_tuple_init(ToastTupleContext *ttc)
ttc->ttc_attr[i].tai_colflags = 0;
ttc->ttc_attr[i].tai_oldexternal = NULL;
+ ttc->ttc_attr[i].tai_compression = att->attcompression;
if (ttc->ttc_oldvalues != NULL)
{
@@ -226,9 +227,11 @@ void
toast_tuple_try_compression(ToastTupleContext *ttc, int attribute)
{
Datum *value = &ttc->ttc_values[attribute];
- Datum new_value = toast_compress_datum(*value);
+ Datum new_value;
ToastAttrInfo *attr = &ttc->ttc_attr[attribute];
+ new_value = toast_compress_datum(*value, attr->tai_compression);
+
if (DatumGetPointer(new_value) != NULL)
{
/* successful compression */
diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c
index 41da0c5..99e5968 100644
--- a/src/backend/bootstrap/bootstrap.c
+++ b/src/backend/bootstrap/bootstrap.c
@@ -21,6 +21,7 @@
#include "access/heapam.h"
#include "access/htup_details.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog_internal.h"
#include "bootstrap/bootstrap.h"
@@ -733,6 +734,10 @@ DefineAttr(char *name, char *type, int attnum, int nullness)
attrtypes[attnum]->attcacheoff = -1;
attrtypes[attnum]->atttypmod = -1;
attrtypes[attnum]->attislocal = true;
+ if (IsStorageCompressible(attrtypes[attnum]->attstorage))
+ attrtypes[attnum]->attcompression = GetDefaultToastCompression();
+ else
+ attrtypes[attnum]->attcompression = InvalidCompressionMethod;
if (nullness == BOOTCOL_NULL_FORCE_NOT_NULL)
{
diff --git a/src/backend/catalog/genbki.pl b/src/backend/catalog/genbki.pl
index b159958..9586c29 100644
--- a/src/backend/catalog/genbki.pl
+++ b/src/backend/catalog/genbki.pl
@@ -906,6 +906,9 @@ sub morph_row_for_pgattr
$row->{attcollation} =
$type->{typcollation} ne '0' ? $C_COLLATION_OID : 0;
+ $row->{attcompression} =
+ $type->{typstorage} ne 'p' && $type->{typstorage} ne 'e' ? 'p' : '\0';
+
if (defined $attr->{forcenotnull})
{
$row->{attnotnull} = 't';
diff --git a/src/backend/catalog/heap.c b/src/backend/catalog/heap.c
index 9abc4a1..d0ec44b 100644
--- a/src/backend/catalog/heap.c
+++ b/src/backend/catalog/heap.c
@@ -36,6 +36,7 @@
#include "access/sysattr.h"
#include "access/table.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/xact.h"
#include "access/xlog.h"
@@ -789,6 +790,7 @@ InsertPgAttributeTuples(Relation pg_attribute_rel,
slot[slotCount]->tts_values[Anum_pg_attribute_attislocal - 1] = BoolGetDatum(attrs->attislocal);
slot[slotCount]->tts_values[Anum_pg_attribute_attinhcount - 1] = Int32GetDatum(attrs->attinhcount);
slot[slotCount]->tts_values[Anum_pg_attribute_attcollation - 1] = ObjectIdGetDatum(attrs->attcollation);
+ slot[slotCount]->tts_values[Anum_pg_attribute_attcompression - 1] = CharGetDatum(attrs->attcompression);
if (attoptions && attoptions[natts] != (Datum) 0)
slot[slotCount]->tts_values[Anum_pg_attribute_attoptions - 1] = attoptions[natts];
else
@@ -1715,6 +1717,8 @@ RemoveAttributeById(Oid relid, AttrNumber attnum)
/* Unset this so no one tries to look up the generation expression */
attStruct->attgenerated = '\0';
+ attStruct->attcompression = InvalidCompressionMethod;
+
/*
* Change the column name to something that isn't likely to conflict
*/
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 4ef61b5..397d70d 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -348,6 +348,7 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attbyval = from->attbyval;
to->attstorage = from->attstorage;
to->attalign = from->attalign;
+ to->attcompression = from->attcompression;
}
else
{
diff --git a/src/backend/catalog/toasting.c b/src/backend/catalog/toasting.c
index d7b8060..933a073 100644
--- a/src/backend/catalog/toasting.c
+++ b/src/backend/catalog/toasting.c
@@ -15,6 +15,7 @@
#include "postgres.h"
#include "access/heapam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "catalog/binary_upgrade.h"
#include "catalog/catalog.h"
@@ -220,6 +221,11 @@ create_toast_table(Relation rel, Oid toastOid, Oid toastIndexOid,
TupleDescAttr(tupdesc, 1)->attstorage = TYPSTORAGE_PLAIN;
TupleDescAttr(tupdesc, 2)->attstorage = TYPSTORAGE_PLAIN;
+ /* Toast field should not be compressed */
+ TupleDescAttr(tupdesc, 0)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 1)->attcompression = InvalidCompressionMethod;
+ TupleDescAttr(tupdesc, 2)->attcompression = InvalidCompressionMethod;
+
/*
* Toast tables for regular relations go in pg_toast; those for temp
* relations go into the per-backend temp-toast-table namespace.
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ffb1308..ab89935 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -23,6 +23,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/xact.h"
#include "access/xlog.h"
#include "catalog/catalog.h"
@@ -527,6 +528,8 @@ static void ATExecReplicaIdentity(Relation rel, ReplicaIdentityStmt *stmt, LOCKM
static void ATExecGenericOptions(Relation rel, List *options);
static void ATExecSetRowSecurity(Relation rel, bool rls);
static void ATExecForceNoForceRowSecurity(Relation rel, bool force_rls);
+static ObjectAddress ATExecSetCompression(AlteredTableInfo *tab, Relation rel,
+ const char *column, Node *newValue, LOCKMODE lockmode);
static void index_copy_data(Relation rel, RelFileNode newrnode);
static const char *storage_name(char c);
@@ -558,6 +561,7 @@ static void refuseDupeIndexAttach(Relation parentIdx, Relation partIdx,
static List *GetParentedForeignKeyRefs(Relation partition);
static void ATDetachCheckNoForeignKeyRefs(Relation partition);
static void ATExecAlterCollationRefreshVersion(Relation rel, List *coll);
+static char GetAttributeCompression(Form_pg_attribute att, char *compression);
/* ----------------------------------------------------------------
@@ -852,6 +856,18 @@ DefineRelation(CreateStmt *stmt, char relkind, Oid ownerId,
if (colDef->generated)
attr->attgenerated = colDef->generated;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (relkind == RELKIND_RELATION ||
+ relkind == RELKIND_PARTITIONED_TABLE ||
+ relkind == RELKIND_MATVIEW)
+ attr->attcompression =
+ GetAttributeCompression(attr, colDef->compression);
+ else
+ attr->attcompression = InvalidCompressionMethod;
}
/*
@@ -2396,6 +2412,22 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(attribute->attstorage))));
+ /* Copy/check compression parameter */
+ if (CompressionMethodIsValid(attribute->attcompression))
+ {
+ const char *compression =
+ GetCompressionMethodName(attribute->attcompression);
+
+ if (def->compression == NULL)
+ def->compression = pstrdup(compression);
+ else if (strcmp(def->compression, compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, compression)));
+ }
+
def->inhcount++;
/* Merge of NOT NULL constraints = OR 'em together */
def->is_not_null |= attribute->attnotnull;
@@ -2430,6 +2462,11 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
def->collOid = attribute->attcollation;
def->constraints = NIL;
def->location = -1;
+ if (CompressionMethodIsValid(attribute->attcompression))
+ def->compression = pstrdup(GetCompressionMethodName(
+ attribute->attcompression));
+ else
+ def->compression = NULL;
inhSchema = lappend(inhSchema, def);
newattmap->attnums[parent_attno - 1] = ++child_attno;
}
@@ -2675,6 +2712,19 @@ MergeAttributes(List *schema, List *supers, char relpersistence,
storage_name(def->storage),
storage_name(newdef->storage))));
+ /* Copy compression parameter */
+ if (def->compression == NULL)
+ def->compression = newdef->compression;
+ else if (newdef->compression != NULL)
+ {
+ if (strcmp(def->compression, newdef->compression) != 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_DATATYPE_MISMATCH),
+ errmsg("column \"%s\" has a compression method conflict",
+ attributeName),
+ errdetail("%s versus %s", def->compression, newdef->compression)));
+ }
+
/* Mark the column as locally defined */
def->is_local = true;
/* Merge of NOT NULL constraints = OR 'em together */
@@ -3961,6 +4011,7 @@ AlterTableGetLockLevel(List *cmds)
case AT_DropIdentity:
case AT_SetIdentity:
case AT_DropExpression:
+ case AT_SetCompression:
cmd_lockmode = AccessExclusiveLock;
break;
@@ -4283,6 +4334,12 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
+ case AT_SetCompression: /* ALTER COLUMN SET COMPRESSION */
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
+ /* This command never recurses */
+ /* No command-specific prep needed */
+ pass = AT_PASS_MISC;
+ break;
case AT_DropColumn: /* DROP COLUMN */
ATSimplePermissions(rel,
ATT_TABLE | ATT_COMPOSITE_TYPE | ATT_FOREIGN_TABLE);
@@ -4626,6 +4683,10 @@ ATExecCmd(List **wqueue, AlteredTableInfo *tab, Relation rel,
case AT_SetStorage: /* ALTER COLUMN SET STORAGE */
address = ATExecSetStorage(rel, cmd->name, cmd->def, lockmode);
break;
+ case AT_SetCompression:
+ address = ATExecSetCompression(tab, rel, cmd->name, cmd->def,
+ lockmode);
+ break;
case AT_DropColumn: /* DROP COLUMN */
address = ATExecDropColumn(wqueue, rel, cmd->name,
cmd->behavior, false, false,
@@ -6340,6 +6401,18 @@ ATExecAddColumn(List **wqueue, AlteredTableInfo *tab, Relation rel,
attribute.attislocal = colDef->is_local;
attribute.attinhcount = colDef->inhcount;
attribute.attcollation = collOid;
+
+ /*
+ * lookup attribute's compression method and store it in the
+ * attr->attcompression.
+ */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ attribute.attcompression = GetAttributeCompression(&attribute,
+ colDef->compression);
+ else
+ attribute.attcompression = InvalidCompressionMethod;
+
/* attribute.attacl is handled by InsertPgAttributeTuples() */
ReleaseSysCache(typeTuple);
@@ -7713,6 +7786,68 @@ ATExecSetOptions(Relation rel, const char *colName, Node *options,
}
/*
+ * Helper function for ATExecSetStorage and ATExecSetCompression
+ *
+ * Set the attcompression and/or attstorage for the respective index attribute
+ * if the respective input values are valid.
+ */
+static void
+SetIndexStorageProperties(Relation rel, Relation attrelation,
+ AttrNumber attnum, char newcompression,
+ char newstorage, LOCKMODE lockmode)
+{
+ HeapTuple tuple;
+ ListCell *lc;
+ Form_pg_attribute attrtuple;
+
+ foreach(lc, RelationGetIndexList(rel))
+ {
+ Oid indexoid = lfirst_oid(lc);
+ Relation indrel;
+ AttrNumber indattnum = 0;
+
+ indrel = index_open(indexoid, lockmode);
+
+ for (int i = 0; i < indrel->rd_index->indnatts; i++)
+ {
+ if (indrel->rd_index->indkey.values[i] == attnum)
+ {
+ indattnum = i + 1;
+ break;
+ }
+ }
+
+ if (indattnum == 0)
+ {
+ index_close(indrel, lockmode);
+ continue;
+ }
+
+ tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
+
+ if (HeapTupleIsValid(tuple))
+ {
+ attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
+
+ if (CompressionMethodIsValid(newcompression))
+ attrtuple->attcompression = newcompression;
+
+ if (newstorage != '\0')
+ attrtuple->attstorage = newstorage;
+
+ CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ attrtuple->attnum);
+
+ heap_freetuple(tuple);
+ }
+
+ index_close(indrel, lockmode);
+ }
+}
+/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
* Return value is the address of the modified column
@@ -7727,7 +7862,6 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
Form_pg_attribute attrtuple;
AttrNumber attnum;
ObjectAddress address;
- ListCell *lc;
Assert(IsA(newValue, String));
storagemode = strVal(newValue);
@@ -7791,47 +7925,9 @@ ATExecSetStorage(Relation rel, const char *colName, Node *newValue, LOCKMODE loc
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- foreach(lc, RelationGetIndexList(rel))
- {
- Oid indexoid = lfirst_oid(lc);
- Relation indrel;
- AttrNumber indattnum = 0;
-
- indrel = index_open(indexoid, lockmode);
-
- for (int i = 0; i < indrel->rd_index->indnatts; i++)
- {
- if (indrel->rd_index->indkey.values[i] == attnum)
- {
- indattnum = i + 1;
- break;
- }
- }
-
- if (indattnum == 0)
- {
- index_close(indrel, lockmode);
- continue;
- }
-
- tuple = SearchSysCacheCopyAttNum(RelationGetRelid(indrel), indattnum);
-
- if (HeapTupleIsValid(tuple))
- {
- attrtuple = (Form_pg_attribute) GETSTRUCT(tuple);
- attrtuple->attstorage = newstorage;
-
- CatalogTupleUpdate(attrelation, &tuple->t_self, tuple);
-
- InvokeObjectPostAlterHook(RelationRelationId,
- RelationGetRelid(rel),
- attrtuple->attnum);
-
- heap_freetuple(tuple);
- }
-
- index_close(indrel, lockmode);
- }
+ SetIndexStorageProperties(rel, attrelation, attnum,
+ InvalidCompressionMethod,
+ newstorage, lockmode);
table_close(attrelation, RowExclusiveLock);
@@ -11859,6 +11955,23 @@ ATExecAlterColumnType(AlteredTableInfo *tab, Relation rel,
ReleaseSysCache(typeTuple);
+ /* Setup attribute compression */
+ if (rel->rd_rel->relkind == RELKIND_RELATION ||
+ rel->rd_rel->relkind == RELKIND_PARTITIONED_TABLE)
+ {
+ /*
+ * No compression for plain/external storage, otherwise, default
+ * compression method if it is not already set, refer comments atop
+ * attcompression parameter in pg_attribute.h.
+ */
+ if (!IsStorageCompressible(tform->typstorage))
+ attTup->attcompression = InvalidCompressionMethod;
+ else if (!CompressionMethodIsValid(attTup->attcompression))
+ attTup->attcompression = GetDefaultToastCompression();
+ }
+ else
+ attTup->attcompression = InvalidCompressionMethod;
+
CatalogTupleUpdate(attrelation, &heapTup->t_self, heapTup);
table_close(attrelation, RowExclusiveLock);
@@ -14940,6 +15053,89 @@ ATExecGenericOptions(Relation rel, List *options)
}
/*
+ * ALTER TABLE ALTER COLUMN SET COMPRESSION
+ *
+ * Return value is the address of the modified column
+ */
+static ObjectAddress
+ATExecSetCompression(AlteredTableInfo *tab,
+ Relation rel,
+ const char *column,
+ Node *newValue,
+ LOCKMODE lockmode)
+{
+ Relation attrel;
+ HeapTuple tuple;
+ Form_pg_attribute atttableform;
+ AttrNumber attnum;
+ char *compression;
+ char typstorage;
+ Oid cmoid;
+ Datum values[Natts_pg_attribute];
+ bool nulls[Natts_pg_attribute];
+ bool replace[Natts_pg_attribute];
+ ObjectAddress address;
+
+ Assert(IsA(newValue, String));
+ compression = strVal(newValue);
+
+ attrel = table_open(AttributeRelationId, RowExclusiveLock);
+
+ tuple = SearchSysCacheAttName(RelationGetRelid(rel), column);
+ if (!HeapTupleIsValid(tuple))
+ ereport(ERROR,
+ (errcode(ERRCODE_UNDEFINED_COLUMN),
+ errmsg("column \"%s\" of relation \"%s\" does not exist",
+ column, RelationGetRelationName(rel))));
+
+ /* prevent them from altering a system attribute */
+ atttableform = (Form_pg_attribute) GETSTRUCT(tuple);
+ attnum = atttableform->attnum;
+ if (attnum <= 0)
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("cannot alter system column \"%s\"", column)));
+
+ typstorage = get_typstorage(atttableform->atttypid);
+
+ /* prevent from setting compression methods for uncompressible type */
+ if (!IsStorageCompressible(typstorage))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(atttableform->atttypid))));
+
+ /* initialize buffers for new tuple values */
+ memset(values, 0, sizeof(values));
+ memset(nulls, false, sizeof(nulls));
+ memset(replace, false, sizeof(replace));
+
+ /* get the attribute compression method. */
+ cmoid = GetAttributeCompression(atttableform, compression);
+
+ atttableform->attcompression = cmoid;
+ CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
+
+ InvokeObjectPostAlterHook(RelationRelationId,
+ RelationGetRelid(rel),
+ atttableform->attnum);
+
+ ReleaseSysCache(tuple);
+
+ /* apply changes to the index column as well */
+ SetIndexStorageProperties(rel, attrel, attnum, cmoid, '\0', lockmode);
+ table_close(attrel, RowExclusiveLock);
+
+ /* make changes visible */
+ CommandCounterIncrement();
+
+ ObjectAddressSubSet(address, RelationRelationId,
+ RelationGetRelid(rel), atttableform->attnum);
+ return address;
+}
+
+
+/*
* Preparation phase for SET LOGGED/UNLOGGED
*
* This verifies that we're not trying to change a temp table. Also,
@@ -17641,3 +17837,36 @@ ATExecAlterCollationRefreshVersion(Relation rel, List *coll)
index_update_collation_versions(rel->rd_id, get_collation_oid(coll, false));
CacheInvalidateRelcache(rel);
}
+
+/*
+ * resolve column compression specification to compression method.
+ */
+static char
+GetAttributeCompression(Form_pg_attribute att, char *compression)
+{
+ char typstorage = get_typstorage(att->atttypid);
+ char cmethod;
+
+ /*
+ * No compression for plain/external storage, refer comments atop
+ * attcompression parameter in pg_attribute.h
+ */
+ if (!IsStorageCompressible(typstorage))
+ {
+ if (compression == NULL)
+ return InvalidCompressionMethod;
+
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("column data type %s does not support compression",
+ format_type_be(att->atttypid))));
+ }
+
+ /* fallback to default compression if it's not specified */
+ if (compression == NULL)
+ cmethod = GetDefaultToastCompression();
+ else
+ cmethod = CompressionNameToMethod(compression);
+
+ return cmethod;
+}
diff --git a/src/backend/nodes/copyfuncs.c b/src/backend/nodes/copyfuncs.c
index da91cbd..0d3b923 100644
--- a/src/backend/nodes/copyfuncs.c
+++ b/src/backend/nodes/copyfuncs.c
@@ -2988,6 +2988,7 @@ _copyColumnDef(const ColumnDef *from)
COPY_STRING_FIELD(colname);
COPY_NODE_FIELD(typeName);
+ COPY_STRING_FIELD(compression);
COPY_SCALAR_FIELD(inhcount);
COPY_SCALAR_FIELD(is_local);
COPY_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/equalfuncs.c b/src/backend/nodes/equalfuncs.c
index c2d7362..f359200 100644
--- a/src/backend/nodes/equalfuncs.c
+++ b/src/backend/nodes/equalfuncs.c
@@ -2599,6 +2599,7 @@ _equalColumnDef(const ColumnDef *a, const ColumnDef *b)
{
COMPARE_STRING_FIELD(colname);
COMPARE_NODE_FIELD(typeName);
+ COMPARE_STRING_FIELD(compression);
COMPARE_SCALAR_FIELD(inhcount);
COMPARE_SCALAR_FIELD(is_local);
COMPARE_SCALAR_FIELD(is_not_null);
diff --git a/src/backend/nodes/nodeFuncs.c b/src/backend/nodes/nodeFuncs.c
index 49357ac..3822653 100644
--- a/src/backend/nodes/nodeFuncs.c
+++ b/src/backend/nodes/nodeFuncs.c
@@ -3897,6 +3897,8 @@ raw_expression_tree_walker(Node *node,
if (walker(coldef->typeName, context))
return true;
+ if (walker(coldef->compression, context))
+ return true;
if (walker(coldef->raw_default, context))
return true;
if (walker(coldef->collClause, context))
diff --git a/src/backend/nodes/outfuncs.c b/src/backend/nodes/outfuncs.c
index 6493a03..75343b8 100644
--- a/src/backend/nodes/outfuncs.c
+++ b/src/backend/nodes/outfuncs.c
@@ -2876,6 +2876,7 @@ _outColumnDef(StringInfo str, const ColumnDef *node)
WRITE_STRING_FIELD(colname);
WRITE_NODE_FIELD(typeName);
+ WRITE_STRING_FIELD(compression);
WRITE_INT_FIELD(inhcount);
WRITE_BOOL_FIELD(is_local);
WRITE_BOOL_FIELD(is_not_null);
diff --git a/src/backend/parser/gram.y b/src/backend/parser/gram.y
index 652be0b..5c4e779 100644
--- a/src/backend/parser/gram.y
+++ b/src/backend/parser/gram.y
@@ -596,6 +596,8 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
%type <list> hash_partbound
%type <defelt> hash_partbound_elem
+%type <str> optColumnCompression
+
/*
* Non-keyword token types. These are hard-wired into the "flex" lexer.
* They must be listed first so that their numeric codes do not depend on
@@ -631,9 +633,9 @@ static Node *makeRecursiveViewSelect(char *relname, List *aliases, Node *query);
CACHE CALL CALLED CASCADE CASCADED CASE CAST CATALOG_P CHAIN CHAR_P
CHARACTER CHARACTERISTICS CHECK CHECKPOINT CLASS CLOSE
CLUSTER COALESCE COLLATE COLLATION COLUMN COLUMNS COMMENT COMMENTS COMMIT
- COMMITTED CONCURRENTLY CONFIGURATION CONFLICT CONNECTION CONSTRAINT
- CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY COST CREATE
- CROSS CSV CUBE CURRENT_P
+ COMMITTED COMPRESSION CONCURRENTLY CONFIGURATION CONFLICT
+ CONNECTION CONSTRAINT CONSTRAINTS CONTENT_P CONTINUE_P CONVERSION_P COPY
+ COST CREATE CROSS CSV CUBE CURRENT_P
CURRENT_CATALOG CURRENT_DATE CURRENT_ROLE CURRENT_SCHEMA
CURRENT_TIME CURRENT_TIMESTAMP CURRENT_USER CURSOR CYCLE
@@ -2306,6 +2308,15 @@ alter_table_cmd:
n->missing_ok = true;
$$ = (Node *)n;
}
+ /* ALTER TABLE <name> ALTER [COLUMN] <colname> SET (COMPRESSION <cm>) */
+ | ALTER opt_column ColId SET optColumnCompression
+ {
+ AlterTableCmd *n = makeNode(AlterTableCmd);
+ n->subtype = AT_SetCompression;
+ n->name = $3;
+ n->def = (Node *) makeString($5);
+ $$ = (Node *)n;
+ }
/* ALTER TABLE <name> DROP [COLUMN] IF EXISTS <colname> [RESTRICT|CASCADE] */
| DROP opt_column IF_P EXISTS ColId opt_drop_behavior
{
@@ -3421,11 +3432,12 @@ TypedTableElement:
| TableConstraint { $$ = $1; }
;
-columnDef: ColId Typename create_generic_options ColQualList
+columnDef: ColId Typename optColumnCompression create_generic_options ColQualList
{
ColumnDef *n = makeNode(ColumnDef);
n->colname = $1;
n->typeName = $2;
+ n->compression = $3;
n->inhcount = 0;
n->is_local = true;
n->is_not_null = false;
@@ -3434,8 +3446,8 @@ columnDef: ColId Typename create_generic_options ColQualList
n->raw_default = NULL;
n->cooked_default = NULL;
n->collOid = InvalidOid;
- n->fdwoptions = $3;
- SplitColQualList($4, &n->constraints, &n->collClause,
+ n->fdwoptions = $4;
+ SplitColQualList($5, &n->constraints, &n->collClause,
yyscanner);
n->location = @1;
$$ = (Node *)n;
@@ -3480,6 +3492,14 @@ columnOptions: ColId ColQualList
}
;
+optColumnCompression:
+ COMPRESSION name
+ {
+ $$ = $2;
+ }
+ | /*EMPTY*/ { $$ = NULL; }
+ ;
+
ColQualList:
ColQualList ColConstraint { $$ = lappend($1, $2); }
| /*EMPTY*/ { $$ = NIL; }
@@ -3710,6 +3730,7 @@ TableLikeOption:
| INDEXES { $$ = CREATE_TABLE_LIKE_INDEXES; }
| STATISTICS { $$ = CREATE_TABLE_LIKE_STATISTICS; }
| STORAGE { $$ = CREATE_TABLE_LIKE_STORAGE; }
+ | COMPRESSION { $$ = CREATE_TABLE_LIKE_COMPRESSION; }
| ALL { $$ = CREATE_TABLE_LIKE_ALL; }
;
@@ -15296,6 +15317,7 @@ unreserved_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONFIGURATION
| CONFLICT
| CONNECTION
@@ -15816,6 +15838,7 @@ bare_label_keyword:
| COMMENTS
| COMMIT
| COMMITTED
+ | COMPRESSION
| CONCURRENTLY
| CONFIGURATION
| CONFLICT
diff --git a/src/backend/parser/parse_utilcmd.c b/src/backend/parser/parse_utilcmd.c
index d56f81c..aa6c19a 100644
--- a/src/backend/parser/parse_utilcmd.c
+++ b/src/backend/parser/parse_utilcmd.c
@@ -31,6 +31,7 @@
#include "access/relation.h"
#include "access/reloptions.h"
#include "access/table.h"
+#include "access/toast_compression.h"
#include "catalog/dependency.h"
#include "catalog/heap.h"
#include "catalog/index.h"
@@ -1092,6 +1093,14 @@ transformTableLikeClause(CreateStmtContext *cxt, TableLikeClause *table_like_cla
else
def->storage = 0;
+ /* Likewise, copy compression if requested */
+ if ((table_like_clause->options & CREATE_TABLE_LIKE_COMPRESSION) != 0
+ && CompressionMethodIsValid(attribute->attcompression))
+ def->compression =
+ pstrdup(GetCompressionMethodName(attribute->attcompression));
+ else
+ def->compression = NULL;
+
/* Likewise, copy comment if requested */
if ((table_like_clause->options & CREATE_TABLE_LIKE_COMMENTS) &&
(comment = GetComment(attribute->attrelid,
diff --git a/src/backend/replication/logical/reorderbuffer.c b/src/backend/replication/logical/reorderbuffer.c
index 91600ac..c291b05 100644
--- a/src/backend/replication/logical/reorderbuffer.c
+++ b/src/backend/replication/logical/reorderbuffer.c
@@ -4641,7 +4641,7 @@ ReorderBufferToastReplace(ReorderBuffer *rb, ReorderBufferTXN *txn,
VARSIZE(chunk) - VARHDRSZ);
data_done += VARSIZE(chunk) - VARHDRSZ;
}
- Assert(data_done == toast_pointer.va_extsize);
+ Assert(data_done == VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer));
/* make sure its marked as compressed or not */
if (VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer))
diff --git a/src/backend/utils/adt/varlena.c b/src/backend/utils/adt/varlena.c
index 479ed9a..0bc345a 100644
--- a/src/backend/utils/adt/varlena.c
+++ b/src/backend/utils/adt/varlena.c
@@ -18,6 +18,7 @@
#include <limits.h>
#include "access/detoast.h"
+#include "access/toast_compression.h"
#include "catalog/pg_collation.h"
#include "catalog/pg_type.h"
#include "common/hashfn.h"
@@ -5300,6 +5301,59 @@ pg_column_size(PG_FUNCTION_ARGS)
}
/*
+ * Return the compression method stored in the compressed attribute. Return
+ * NULL for non varlena type or uncompressed data.
+ */
+Datum
+pg_column_compression(PG_FUNCTION_ARGS)
+{
+ int typlen;
+ char *result;
+ ToastCompressionId cmid;
+
+ /* On first call, get the input type's typlen, and save at *fn_extra */
+ if (fcinfo->flinfo->fn_extra == NULL)
+ {
+ /* Lookup the datatype of the supplied argument */
+ Oid argtypeid = get_fn_expr_argtype(fcinfo->flinfo, 0);
+
+ typlen = get_typlen(argtypeid);
+ if (typlen == 0) /* should not happen */
+ elog(ERROR, "cache lookup failed for type %u", argtypeid);
+
+ fcinfo->flinfo->fn_extra = MemoryContextAlloc(fcinfo->flinfo->fn_mcxt,
+ sizeof(int));
+ *((int *) fcinfo->flinfo->fn_extra) = typlen;
+ }
+ else
+ typlen = *((int *) fcinfo->flinfo->fn_extra);
+
+ if (typlen != -1)
+ PG_RETURN_NULL();
+
+ /* get the compression method id stored in the compressed varlena */
+ cmid = toast_get_compression_id((struct varlena *)
+ DatumGetPointer(PG_GETARG_DATUM(0)));
+ if (cmid == TOAST_INVALID_COMPRESSION_ID)
+ PG_RETURN_NULL();
+
+ /* convert compression method id to compression method name */
+ switch (cmid)
+ {
+ case TOAST_PGLZ_COMPRESSION_ID:
+ result = "pglz";
+ break;
+ case TOAST_LZ4_COMPRESSION_ID:
+ result = "lz4";
+ break;
+ default:
+ elog(ERROR, "invalid compression method id %d", cmid);
+ }
+
+ PG_RETURN_TEXT_P(cstring_to_text(result));
+}
+
+/*
* string_agg - Concatenates values and returns string.
*
* Syntax: string_agg(value text, delimiter text) RETURNS text
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 6b2d9d6..6abbd63 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -33,6 +33,7 @@
#include "access/gin.h"
#include "access/rmgr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/twophase.h"
#include "access/xact.h"
@@ -3926,6 +3927,17 @@ static struct config_string ConfigureNamesString[] =
},
{
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ DEFAULT_TOAST_COMPRESSION,
+ check_default_toast_compression, NULL, NULL
+ },
+
+ {
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
gettext_noop("An empty string selects the database's default tablespace."),
diff --git a/src/backend/utils/misc/postgresql.conf.sample b/src/backend/utils/misc/postgresql.conf.sample
index 6647f8f..106016d 100644
--- a/src/backend/utils/misc/postgresql.conf.sample
+++ b/src/backend/utils/misc/postgresql.conf.sample
@@ -660,6 +660,7 @@
#temp_tablespaces = '' # a list of tablespace names, '' uses
# only default tablespace
#default_table_access_method = 'heap'
+#default_toast_compression = 'pglz' # 'pglz' or 'lz4'
#check_function_bodies = on
#default_transaction_isolation = 'read committed'
#default_transaction_read_only = off
diff --git a/src/bin/pg_amcheck/t/004_verify_heapam.pl b/src/bin/pg_amcheck/t/004_verify_heapam.pl
index 16574cb..3660759 100644
--- a/src/bin/pg_amcheck/t/004_verify_heapam.pl
+++ b/src/bin/pg_amcheck/t/004_verify_heapam.pl
@@ -124,7 +124,7 @@ sub read_tuple
c_va_header => shift,
c_va_vartag => shift,
c_va_rawsize => shift,
- c_va_extsize => shift,
+ c_va_extinfo => shift,
c_va_valueid => shift,
c_va_toastrelid => shift);
# Stitch together the text for column 'b'
@@ -169,7 +169,7 @@ sub write_tuple
$tup->{c_va_header},
$tup->{c_va_vartag},
$tup->{c_va_rawsize},
- $tup->{c_va_extsize},
+ $tup->{c_va_extinfo},
$tup->{c_va_valueid},
$tup->{c_va_toastrelid});
seek($fh, $offset, 0)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index eea9f30..0296b9b 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -160,6 +160,7 @@ typedef struct _dumpOptions
int no_subscriptions;
int no_synchronized_snapshots;
int no_unlogged_table_data;
+ int no_toast_compression;
int serializable_deferrable;
int disable_triggers;
int outputNoTablespaces;
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index eb988d7..f8bec3f 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -387,6 +387,7 @@ main(int argc, char **argv)
{"no-synchronized-snapshots", no_argument, &dopt.no_synchronized_snapshots, 1},
{"no-unlogged-table-data", no_argument, &dopt.no_unlogged_table_data, 1},
{"no-subscriptions", no_argument, &dopt.no_subscriptions, 1},
+ {"no-toast-compression", no_argument, &dopt.no_toast_compression, 1},
{"no-sync", no_argument, NULL, 7},
{"on-conflict-do-nothing", no_argument, &dopt.do_nothing, 1},
{"rows-per-insert", required_argument, NULL, 10},
@@ -1047,6 +1048,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -8617,6 +8619,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
{
DumpOptions *dopt = fout->dopt;
PQExpBuffer q = createPQExpBuffer();
+ bool createWithCompression;
for (int i = 0; i < numTables; i++)
{
@@ -8702,6 +8705,15 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
appendPQExpBufferStr(q,
"'' AS attidentity,\n");
+ createWithCompression = (fout->remoteVersion >= 140000);
+
+ if (createWithCompression)
+ appendPQExpBuffer(q,
+ "a.attcompression AS attcompression,\n");
+ else
+ appendPQExpBuffer(q,
+ "NULL AS attcompression,\n");
+
if (fout->remoteVersion >= 110000)
appendPQExpBufferStr(q,
"CASE WHEN a.atthasmissing AND NOT a.attisdropped "
@@ -8747,6 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -8775,6 +8788,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation[j] = atooid(PQgetvalue(res, j, PQfnumber(res, "attcollation")));
tbinfo->attfdwoptions[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attfdwoptions")));
tbinfo->attmissingval[j] = pg_strdup(PQgetvalue(res, j, PQfnumber(res, "attmissingval")));
+ tbinfo->attcompression[j] = *(PQgetvalue(res, j, PQfnumber(res, "attcompression")));
tbinfo->attrdefs[j] = NULL; /* fix below */
if (PQgetvalue(res, j, PQfnumber(res, "atthasdef"))[0] == 't')
hasdefaults = true;
@@ -15891,6 +15905,31 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
+ /*
+ * Attribute compression
+ */
+ if (!dopt->no_toast_compression &&
+ tbinfo->attcompression != NULL)
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ break;
+ }
+
+ if (cmname != NULL)
+ appendPQExpBuffer(q, " COMPRESSION %s", cmname);
+ }
+
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
diff --git a/src/bin/pg_dump/pg_dump.h b/src/bin/pg_dump/pg_dump.h
index 0a2213f..453f946 100644
--- a/src/bin/pg_dump/pg_dump.h
+++ b/src/bin/pg_dump/pg_dump.h
@@ -326,6 +326,7 @@ typedef struct _tableInfo
char *partbound; /* partition bound definition */
bool needs_override; /* has GENERATED ALWAYS AS IDENTITY */
char *amname; /* relation access method */
+ char *attcompression; /* per-attribute current compression method */
/*
* Stuff computed only for dumpable tables.
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index 737e464..bc91bb1 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text,\E\n
- \s+\Qcol3 text,\E\n
- \s+\Qcol4 text,\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*,\n
+ \s+\Qcol3 text COMPRESSION\E\D*,\n
+ \s+\Qcol4 text COMPRESSION\E\D*,\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text\E
+ \n\s+\Qcol2 text COMPRESSION\E\D*
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5),\E
+ \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text\E\n
+ \s+\Qcol2 text COMPRESSION\E\D*\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index 20af5a9..eeac0ef 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1459,7 +1459,7 @@ describeOneTableDetails(const char *schemaname,
bool printTableInitialized = false;
int i;
char *view_def = NULL;
- char *headers[11];
+ char *headers[12];
PQExpBufferData title;
PQExpBufferData tmpbuf;
int cols;
@@ -1475,7 +1475,8 @@ describeOneTableDetails(const char *schemaname,
fdwopts_col = -1,
attstorage_col = -1,
attstattarget_col = -1,
- attdescr_col = -1;
+ attdescr_col = -1,
+ attcompression_col = -1;
int numrows;
struct
{
@@ -1892,6 +1893,17 @@ describeOneTableDetails(const char *schemaname,
appendPQExpBufferStr(&buf, ",\n a.attstorage");
attstorage_col = cols++;
+ /* compression info */
+ if (pset.sversion >= 140000 &&
+ !pset.hide_compression &&
+ (tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
+ tableinfo.relkind == RELKIND_MATVIEW))
+ {
+ appendPQExpBufferStr(&buf, ",\n a.attcompression AS attcompression");
+ attcompression_col = cols++;
+ }
+
/* stats target, if relevant to relkind */
if (tableinfo.relkind == RELKIND_RELATION ||
tableinfo.relkind == RELKIND_INDEX ||
@@ -2018,6 +2030,8 @@ describeOneTableDetails(const char *schemaname,
headers[cols++] = gettext_noop("FDW options");
if (attstorage_col >= 0)
headers[cols++] = gettext_noop("Storage");
+ if (attcompression_col >= 0)
+ headers[cols++] = gettext_noop("Compression");
if (attstattarget_col >= 0)
headers[cols++] = gettext_noop("Stats target");
if (attdescr_col >= 0)
@@ -2097,6 +2111,19 @@ describeOneTableDetails(const char *schemaname,
false, false);
}
+ /* Column compression. */
+ if (attcompression_col >= 0)
+ {
+ char *compression = PQgetvalue(res, i, attcompression_col);
+
+ /* these strings are literal in our syntax, so not translated. */
+ printTableAddCell(&cont, (compression[0] == 'p' ? "pglz" :
+ (compression[0] == 'l' ? "lz4" :
+ (compression[0] == '\0' ? "" :
+ "???"))),
+ false, false);
+ }
+
/* Statistics target, if the relkind supports this feature */
if (attstattarget_col >= 0)
printTableAddCell(&cont, PQgetvalue(res, i, attstattarget_col),
diff --git a/src/bin/psql/help.c b/src/bin/psql/help.c
index daa5081..99a5947 100644
--- a/src/bin/psql/help.c
+++ b/src/bin/psql/help.c
@@ -372,6 +372,8 @@ helpVariables(unsigned short int pager)
" true if last query failed, else false\n"));
fprintf(output, _(" FETCH_COUNT\n"
" the number of result rows to fetch and display at a time (0 = unlimited)\n"));
+ fprintf(output, _(" HIDE_TOAST_COMPRESSION\n"
+ " if set, compression methods are not displayed\n"));
fprintf(output, _(" HIDE_TABLEAM\n"
" if set, table access methods are not displayed\n"));
fprintf(output, _(" HISTCONTROL\n"
diff --git a/src/bin/psql/settings.h b/src/bin/psql/settings.h
index d659900..83f2e6f 100644
--- a/src/bin/psql/settings.h
+++ b/src/bin/psql/settings.h
@@ -134,6 +134,7 @@ typedef struct _psqlSettings
bool quiet;
bool singleline;
bool singlestep;
+ bool hide_compression;
bool hide_tableam;
int fetch_count;
int histsize;
diff --git a/src/bin/psql/startup.c b/src/bin/psql/startup.c
index 780479c..110906a 100644
--- a/src/bin/psql/startup.c
+++ b/src/bin/psql/startup.c
@@ -1160,6 +1160,13 @@ show_context_hook(const char *newval)
}
static bool
+hide_compression_hook(const char *newval)
+{
+ return ParseVariableBool(newval, "HIDE_TOAST_COMPRESSION",
+ &pset.hide_compression);
+}
+
+static bool
hide_tableam_hook(const char *newval)
{
return ParseVariableBool(newval, "HIDE_TABLEAM", &pset.hide_tableam);
@@ -1227,6 +1234,9 @@ EstablishVariableSpace(void)
SetVariableHooks(pset.vars, "SHOW_CONTEXT",
show_context_substitute_hook,
show_context_hook);
+ SetVariableHooks(pset.vars, "HIDE_TOAST_COMPRESSION",
+ bool_substitute_hook,
+ hide_compression_hook);
SetVariableHooks(pset.vars, "HIDE_TABLEAM",
bool_substitute_hook,
hide_tableam_hook);
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index d3fb734..5980641 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -2116,7 +2116,7 @@ psql_completion(const char *text, int start, int end)
/* ALTER TABLE ALTER [COLUMN] <foo> SET */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET"))
- COMPLETE_WITH("(", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
+ COMPLETE_WITH("(", "COMPRESSION", "DEFAULT", "NOT NULL", "STATISTICS", "STORAGE");
/* ALTER TABLE ALTER [COLUMN] <foo> SET ( */
else if (Matches("ALTER", "TABLE", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "(") ||
Matches("ALTER", "TABLE", MatchAny, "ALTER", MatchAny, "SET", "("))
diff --git a/src/include/access/detoast.h b/src/include/access/detoast.h
index 0adf53c..773a02f 100644
--- a/src/include/access/detoast.h
+++ b/src/include/access/detoast.h
@@ -13,16 +13,6 @@
#define DETOAST_H
/*
- * Testing whether an externally-stored value is compressed now requires
- * comparing extsize (the actual length of the external data) to rawsize
- * (the original uncompressed datum's size). The latter includes VARHDRSZ
- * overhead, the former doesn't. We never use compression unless it actually
- * saves space, so we expect either equality or less-than.
- */
-#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
- ((toast_pointer).va_extsize < (toast_pointer).va_rawsize - VARHDRSZ)
-
-/*
* Macro to fetch the possibly-unaligned contents of an EXTERNAL datum
* into a local "struct varatt_external" toast pointer. This should be
* just a memcpy, but some versions of gcc seem to produce broken code
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
new file mode 100644
index 0000000..514df0b
--- /dev/null
+++ b/src/include/access/toast_compression.h
@@ -0,0 +1,123 @@
+/*-------------------------------------------------------------------------
+ *
+ * toast_compression.h
+ * Functions for toast compression.
+ *
+ * Copyright (c) 2021, PostgreSQL Global Development Group
+ *
+ * src/include/access/toast_compression.h
+ *
+ *-------------------------------------------------------------------------
+ */
+
+#ifndef TOAST_COMPRESSION_H
+#define TOAST_COMPRESSION_H
+
+#include "utils/guc.h"
+
+/* GUCs */
+extern char *default_toast_compression;
+
+/* default compression method if not specified. */
+#define DEFAULT_TOAST_COMPRESSION "pglz"
+
+/*
+ * Built-in compression method-id. The toast compression header will store
+ * this in the first 2 bits of the raw length. These built-in compression
+ * method-id are directly mapped to the built-in compression methods.
+ */
+typedef enum ToastCompressionId
+{
+ TOAST_PGLZ_COMPRESSION_ID = 0,
+ TOAST_LZ4_COMPRESSION_ID = 1,
+ TOAST_INVALID_COMPRESSION_ID = 2
+} ToastCompressionId;
+
+/*
+ * Built-in compression methods. pg_attribute will store this in the
+ * attcompression column.
+ */
+#define TOAST_PGLZ_COMPRESSION 'p'
+#define TOAST_LZ4_COMPRESSION 'l'
+
+#define InvalidCompressionMethod '\0'
+#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+
+#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
+
+#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
+ (storage) != TYPSTORAGE_EXTERNAL)
+
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+static inline const char *
+GetCompressionMethodName(char method)
+{
+ switch (method)
+ {
+ case TOAST_PGLZ_COMPRESSION:
+ return "pglz";
+ case TOAST_LZ4_COMPRESSION:
+ return "lz4";
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ }
+}
+
+/*
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
+ */
+static inline char
+CompressionNameToMethod(char *compression)
+{
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
+ {
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
+ }
+
+ return InvalidCompressionMethod;
+}
+
+/*
+ * GetDefaultToastCompression -- get the default toast compression method
+ *
+ * This exists to hide the use of the default_toast_compression GUC variable.
+ */
+static inline char
+GetDefaultToastCompression(void)
+{
+ return CompressionNameToMethod(default_toast_compression);
+}
+
+/* pglz compression/decompression routines */
+extern struct varlena *pglz_compress_datum(const struct varlena *value);
+extern struct varlena *pglz_decompress_datum(const struct varlena *value);
+extern struct varlena *pglz_decompress_datum_slice(const struct varlena *value,
+ int32 slicelength);
+
+/* lz4 compression/decompression routines */
+extern struct varlena *lz4_compress_datum(const struct varlena *value);
+extern struct varlena *lz4_decompress_datum(const struct varlena *value);
+extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value,
+ int32 slicelength);
+extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
+extern bool check_default_toast_compression(char **newval, void **extra,
+ GucSource source);
+
+#endif /* TOAST_COMPRESSION_H */
diff --git a/src/include/access/toast_helper.h b/src/include/access/toast_helper.h
index a9a6d64..05104ce 100644
--- a/src/include/access/toast_helper.h
+++ b/src/include/access/toast_helper.h
@@ -32,6 +32,7 @@ typedef struct
struct varlena *tai_oldexternal;
int32 tai_size;
uint8 tai_colflags;
+ char tai_compression;
} ToastAttrInfo;
/*
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index cedfb89..b4d0684 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -12,6 +12,7 @@
#ifndef TOAST_INTERNALS_H
#define TOAST_INTERNALS_H
+#include "access/toast_compression.h"
#include "storage/lockdefs.h"
#include "utils/relcache.h"
#include "utils/snapshot.h"
@@ -22,22 +23,26 @@
typedef struct toast_compress_header
{
int32 vl_len_; /* varlena header (do not touch directly!) */
- int32 rawsize;
+ uint32 tcinfo; /* 2 bits for compression method and 30 bits
+ * rawsize */
} toast_compress_header;
/*
* Utilities for manipulation of header information for compressed
* toast entries.
*/
-#define TOAST_COMPRESS_HDRSZ ((int32) sizeof(toast_compress_header))
-#define TOAST_COMPRESS_RAWSIZE(ptr) (((toast_compress_header *) (ptr))->rawsize)
-#define TOAST_COMPRESS_SIZE(ptr) ((int32) VARSIZE_ANY(ptr) - TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_RAWDATA(ptr) \
- (((char *) (ptr)) + TOAST_COMPRESS_HDRSZ)
-#define TOAST_COMPRESS_SET_RAWSIZE(ptr, len) \
- (((toast_compress_header *) (ptr))->rawsize = (len))
+#define TOAST_COMPRESS_METHOD(ptr) \
+ (((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
+#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
+ do { \
+ Assert((len) > 0 && (len) <= VARLENA_RAWSIZE_MASK); \
+ Assert((cm_method) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm_method) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_compress_header *) (ptr))->tcinfo = \
+ ((len) | (cm_method) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
-extern Datum toast_compress_datum(Datum value);
+extern Datum toast_compress_datum(Datum value, char cmethod);
extern Oid toast_get_valid_index(Oid toastoid, LOCKMODE lock);
extern void toast_delete_datum(Relation rel, Datum value, bool is_speculative);
diff --git a/src/include/catalog/pg_attribute.h b/src/include/catalog/pg_attribute.h
index 3db42ab..560f8f0 100644
--- a/src/include/catalog/pg_attribute.h
+++ b/src/include/catalog/pg_attribute.h
@@ -160,6 +160,12 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
/* attribute's collation, if any */
Oid attcollation BKI_LOOKUP_OPT(pg_collation);
+ /*
+ * compression method. Must be InvalidCompressionMethod if and only if
+ * typstorage is 'plain' or 'external'.
+ */
+ char attcompression BKI_DEFAULT('\0');
+
#ifdef CATALOG_VARLEN /* variable-length fields start here */
/* NOTE: The following fields are not present in tuple descriptors. */
@@ -187,7 +193,7 @@ CATALOG(pg_attribute,1249,AttributeRelationId) BKI_BOOTSTRAP BKI_ROWTYPE_OID(75,
* can access fields beyond attcollation except in a real tuple!
*/
#define ATTRIBUTE_FIXED_PART_SIZE \
- (offsetof(FormData_pg_attribute,attcollation) + sizeof(Oid))
+ (offsetof(FormData_pg_attribute,attcompression) + sizeof(char))
/* ----------------
* Form_pg_attribute corresponds to a pointer to a tuple with
diff --git a/src/include/catalog/pg_proc.dat b/src/include/catalog/pg_proc.dat
index 93393fc..e259531 100644
--- a/src/include/catalog/pg_proc.dat
+++ b/src/include/catalog/pg_proc.dat
@@ -7103,6 +7103,10 @@
descr => 'bytes required to store the value, perhaps with compression',
proname => 'pg_column_size', provolatile => 's', prorettype => 'int4',
proargtypes => 'any', prosrc => 'pg_column_size' },
+{ oid => '2121',
+ descr => 'compression method for the compressed datum',
+ proname => 'pg_column_compression', provolatile => 's', prorettype => 'text',
+ proargtypes => 'any', prosrc => 'pg_column_compression' },
{ oid => '2322',
descr => 'total disk space usage for the specified tablespace',
proname => 'pg_tablespace_size', provolatile => 'v', prorettype => 'int8',
diff --git a/src/include/nodes/parsenodes.h b/src/include/nodes/parsenodes.h
index 236832a..5c97e61 100644
--- a/src/include/nodes/parsenodes.h
+++ b/src/include/nodes/parsenodes.h
@@ -646,6 +646,7 @@ typedef struct ColumnDef
NodeTag type;
char *colname; /* name of column */
TypeName *typeName; /* type of column */
+ char *compression; /* compression method for column */
int inhcount; /* number of times column is inherited */
bool is_local; /* column has local (non-inherited) def'n */
bool is_not_null; /* NOT NULL constraint specified? */
@@ -685,6 +686,7 @@ typedef enum TableLikeOption
CREATE_TABLE_LIKE_INDEXES = 1 << 5,
CREATE_TABLE_LIKE_STATISTICS = 1 << 6,
CREATE_TABLE_LIKE_STORAGE = 1 << 7,
+ CREATE_TABLE_LIKE_COMPRESSION = 1 << 8,
CREATE_TABLE_LIKE_ALL = PG_INT32_MAX
} TableLikeOption;
@@ -1845,6 +1847,7 @@ typedef enum AlterTableType
AT_SetOptions, /* alter column set ( options ) */
AT_ResetOptions, /* alter column reset ( options ) */
AT_SetStorage, /* alter column set storage */
+ AT_SetCompression, /* alter column set compression */
AT_DropColumn, /* drop column */
AT_DropColumnRecurse, /* internal to commands/tablecmds.c */
AT_AddIndex, /* add index */
diff --git a/src/include/parser/kwlist.h b/src/include/parser/kwlist.h
index 28083aa..ca1f950 100644
--- a/src/include/parser/kwlist.h
+++ b/src/include/parser/kwlist.h
@@ -88,6 +88,7 @@ PG_KEYWORD("comment", COMMENT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("comments", COMMENTS, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("commit", COMMIT, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("committed", COMMITTED, UNRESERVED_KEYWORD, BARE_LABEL)
+PG_KEYWORD("compression", COMPRESSION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("concurrently", CONCURRENTLY, TYPE_FUNC_NAME_KEYWORD, BARE_LABEL)
PG_KEYWORD("configuration", CONFIGURATION, UNRESERVED_KEYWORD, BARE_LABEL)
PG_KEYWORD("conflict", CONFLICT, UNRESERVED_KEYWORD, BARE_LABEL)
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 7a7cc21..0a6422d 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -899,6 +899,9 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
+/* Define to 1 to build with LZ4 support (--with-lz4) */
+#undef USE_LZ4
+
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/include/postgres.h b/src/include/postgres.h
index 2ed5720..2ccbea8 100644
--- a/src/include/postgres.h
+++ b/src/include/postgres.h
@@ -55,7 +55,9 @@
/*
* struct varatt_external is a traditional "TOAST pointer", that is, the
* information needed to fetch a Datum stored out-of-line in a TOAST table.
- * The data is compressed if and only if va_extsize < va_rawsize - VARHDRSZ.
+ * The data is compressed if and only if the size stored in va_extinfo <
+ * va_rawsize - VARHDRSZ.
+ *
* This struct must not contain any padding, because we sometimes compare
* these pointers using memcmp.
*
@@ -67,7 +69,8 @@
typedef struct varatt_external
{
int32 va_rawsize; /* Original data size (includes header) */
- int32 va_extsize; /* External saved size (doesn't) */
+ uint32 va_extinfo; /* External saved size (without header) and
+ * compression method */
Oid va_valueid; /* Unique ID of value within TOAST table */
Oid va_toastrelid; /* RelID of TOAST table containing it */
} varatt_external;
@@ -145,7 +148,8 @@ typedef union
struct /* Compressed-in-line format */
{
uint32 va_header;
- uint32 va_rawsize; /* Original data size (excludes header) */
+ uint32 va_tcinfo; /* Original data size (excludes header) and
+ * compression method */
char va_data[FLEXIBLE_ARRAY_MEMBER]; /* Compressed data */
} va_compressed;
} varattrib_4b;
@@ -274,14 +278,23 @@ typedef struct
(VARSIZE(PTR) - VARHDRSZ + VARHDRSZ_SHORT)
#define VARHDRSZ_EXTERNAL offsetof(varattrib_1b_e, va_data)
+#define VARHDRSZ_COMPRESS offsetof(varattrib_4b, va_compressed.va_data)
#define VARDATA_4B(PTR) (((varattrib_4b *) (PTR))->va_4byte.va_data)
#define VARDATA_4B_C(PTR) (((varattrib_4b *) (PTR))->va_compressed.va_data)
#define VARDATA_1B(PTR) (((varattrib_1b *) (PTR))->va_data)
#define VARDATA_1B_E(PTR) (((varattrib_1b_e *) (PTR))->va_data)
+#define VARLENA_RAWSIZE_BITS 30
+#define VARLENA_RAWSIZE_MASK ((1U << VARLENA_RAWSIZE_BITS) - 1)
+
+/*
+ * va_tcinfo in va_compress contains raw size of datum and compression method.
+ */
#define VARRAWSIZE_4B_C(PTR) \
- (((varattrib_4b *) (PTR))->va_compressed.va_rawsize)
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo & VARLENA_RAWSIZE_MASK)
+#define VARCOMPRESS_4B_C(PTR) \
+ (((varattrib_4b *) (PTR))->va_compressed.va_tcinfo >> VARLENA_RAWSIZE_BITS)
/* Externally visible macros */
@@ -323,6 +336,35 @@ typedef struct
(VARATT_IS_EXTERNAL(PTR) && VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
#define VARATT_IS_EXTERNAL_NON_EXPANDED(PTR) \
(VARATT_IS_EXTERNAL(PTR) && !VARTAG_IS_EXPANDED(VARTAG_EXTERNAL(PTR)))
+
+/*
+ * va_extinfo in varatt_external contains actual length of the external data
+ * and compression method if external data is compressed.
+ */
+#define VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) \
+ ((toast_pointer).va_extinfo & VARLENA_RAWSIZE_MASK)
+
+#define VARATT_EXTERNAL_SET_SIZE_AND_COMPRESSION(toast_pointer, len, cm) \
+ do { \
+ Assert((cm) == TOAST_PGLZ_COMPRESSION_ID || \
+ (cm) == TOAST_LZ4_COMPRESSION_ID); \
+ ((toast_pointer).va_extinfo = (len) | (cm) << VARLENA_RAWSIZE_BITS); \
+ } while (0)
+
+#define VARATT_EXTERNAL_GET_COMPRESSION(PTR) \
+ ((toast_pointer).va_extinfo >> VARLENA_RAWSIZE_BITS)
+
+/*
+ * Testing whether an externally-stored value is compressed now requires
+ * comparing size stored in va_extinfo (the actual length of the external data)
+ * to rawsize (the original uncompressed datum's size). The latter includes
+ * VARHDRSZ overhead, the former doesn't. We never use compression unless it
+ * actually saves space, so we expect either equality or less-than.
+ */
+#define VARATT_EXTERNAL_IS_COMPRESSED(toast_pointer) \
+ (VARATT_EXTERNAL_GET_EXTSIZE(toast_pointer) < \
+ (toast_pointer).va_rawsize - VARHDRSZ)
+
#define VARATT_IS_SHORT(PTR) VARATT_IS_1B(PTR)
#define VARATT_IS_EXTENDED(PTR) (!VARATT_IS_4B_U(PTR))
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
new file mode 100644
index 0000000..3de2886
--- /dev/null
+++ b/src/test/regress/expected/compression.out
@@ -0,0 +1,347 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890', 1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890', 1004));
+\d+ cmdata1
+ Table "public.cmdata1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- verify stored compression method in the data
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ substr
+----------------------------------------------------
+ 01234567890123456789012345678901234567890123456789
+(1 row)
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+\d+ cmmove1
+ Table "public.cmmove1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+DROP TABLE cmdata2;
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890', 1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+-- test externally stored compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ substr
+--------
+ 01234
+ 8f14e
+(2 rows)
+
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+ substr
+--------
+ 8f14e
+(1 row)
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | pglz | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+SELECT pg_column_compression(x) FROM mv;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+SELECT pg_column_compression(f1) FROM cmpart1;
+ pg_column_compression
+-----------------------
+ lz4
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmpart2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+NOTICE: merging multiple inherited definitions of column "f1"
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789', 4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | lz4 | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+ Materialized view "public.mv"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ x | text | | | | extended | lz4 | |
+View definition:
+ SELECT cmdata1.f1 AS x
+ FROM cmdata1;
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+SELECT pg_column_compression(f1) FROM cmpart1;
+ pg_column_compression
+-----------------------
+ lz4
+ pglz
+(2 rows)
+
+SELECT pg_column_compression(f1) FROM cmpart2;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+--vacuum full to recompress the data
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ lz4
+(2 rows)
+
+VACUUM FULL cmdata;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ lz4
+ lz4
+(2 rows)
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+ 36036
+(2 rows)
+
+SELECT length(f1) FROM cmdata1;
+ length
+--------
+ 10040
+ 12449
+(2 rows)
+
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+ 10040
+(2 rows)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
new file mode 100644
index 0000000..40aad81
--- /dev/null
+++ b/src/test/regress/expected/compression_1.out
@@ -0,0 +1,340 @@
+\set HIDE_TOAST_COMPRESSION false
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890', 1000));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata1 VALUES(repeat('1234567890', 1004));
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 VALUES(repeat('1234567890', 1004));
+ ^
+\d+ cmdata1
+-- verify stored compression method in the data
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+ substr
+--------
+ 01234
+(1 row)
+
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+ ^
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+\d+ cmmove1
+ Table "public.cmmove1"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+SELECT pg_column_compression(f1) FROM cmmove1;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmmove3 SELECT * FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove3;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+ ^
+\d+ cmdata2
+DROP TABLE cmdata2;
+ERROR: table "cmdata2" does not exist
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+ERROR: column data type integer does not support compression
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890', 1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+ ^
+SELECT pg_column_compression(f1) FROM cmmove2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+-- test externally stored compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+ pglz
+(1 row)
+
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+ERROR: relation "cmdata1" does not exist
+LINE 1: INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+ ^
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+ ^
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+ substr
+--------
+ 8f14e
+(1 row)
+
+DROP TABLE cmdata2;
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | integer | | | | plain | | |
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | character varying | | | | extended | pglz | |
+
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+-------------------+-----------+----------+---------+---------+-------------+--------------+-------------
+ f1 | character varying | | | | plain | pglz | |
+
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+ pg_column_compression
+-----------------------
+
+(1 row)
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+ ^
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmdata1;
+ ^
+SELECT pg_column_compression(x) FROM mv;
+ERROR: relation "mv" does not exist
+LINE 1: SELECT pg_column_compression(x) FROM mv;
+ ^
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+ERROR: relation "cmpart" does not exist
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+ERROR: relation "cmpart" does not exist
+INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart1;
+ERROR: relation "cmpart1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart1;
+ ^
+SELECT pg_column_compression(f1) FROM cmpart2;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+ERROR: relation "cmdata1" does not exist
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+NOTICE: merging column "f1" with inherited definition
+ERROR: column "f1" has a compression method conflict
+DETAIL: pglz versus lz4
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+ERROR: invalid value for parameter "default_toast_compression": ""
+DETAIL: default_toast_compression cannot be empty.
+SET default_toast_compression = 'I do not exist compression';
+ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
+DETAIL: Compression method "I do not exist compression" does not exist.
+SET default_toast_compression = 'lz4';
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+ Table "public.cmdata2"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+INSERT INTO cmdata VALUES (repeat('123456789', 4004));
+\d+ cmdata
+ Table "public.cmdata"
+ Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description
+--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
+ f1 | text | | | | extended | pglz | |
+Indexes:
+ "idx" btree (f1)
+
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+ERROR: relation "mv" does not exist
+\d+ mv
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ERROR: relation "cmpart1" does not exist
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+ ^
+INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+ERROR: relation "cmpart" does not exist
+LINE 1: INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+ ^
+SELECT pg_column_compression(f1) FROM cmpart1;
+ERROR: relation "cmpart1" does not exist
+LINE 1: SELECT pg_column_compression(f1) FROM cmpart1;
+ ^
+SELECT pg_column_compression(f1) FROM cmpart2;
+ pg_column_compression
+-----------------------
+(0 rows)
+
+--vacuum full to recompress the data
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+VACUUM FULL cmdata;
+SELECT pg_column_compression(f1) FROM cmdata;
+ pg_column_compression
+-----------------------
+ pglz
+ pglz
+(2 rows)
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+ length
+--------
+ 10000
+ 36036
+(2 rows)
+
+SELECT length(f1) FROM cmdata1;
+ERROR: relation "cmdata1" does not exist
+LINE 1: SELECT length(f1) FROM cmdata1;
+ ^
+SELECT length(f1) FROM cmmove1;
+ length
+--------
+ 10000
+(1 row)
+
+SELECT length(f1) FROM cmmove2;
+ length
+--------
+ 10040
+(1 row)
+
+SELECT length(f1) FROM cmmove3;
+ length
+--------
+ 10000
+(1 row)
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/parallel_schedule b/src/test/regress/parallel_schedule
index e280198..70c3830 100644
--- a/src/test/regress/parallel_schedule
+++ b/src/test/regress/parallel_schedule
@@ -115,7 +115,7 @@ test: plancache limit plpgsql copy2 temp domain rangefuncs prepare conversion tr
# ----------
# Another group of parallel tests
# ----------
-test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain
+test: partition_join partition_prune reloptions hash_part indexing partition_aggregate partition_info tuplesort explain compression
# event triggers cannot run concurrently with any test that runs DDL
# oidjoins is read-only, though, and should run late for best coverage
diff --git a/src/test/regress/pg_regress_main.c b/src/test/regress/pg_regress_main.c
index 8dc4941..1524676 100644
--- a/src/test/regress/pg_regress_main.c
+++ b/src/test/regress/pg_regress_main.c
@@ -78,11 +78,11 @@ psql_start_test(const char *testname,
* against different AMs without unnecessary differences.
*/
offset += snprintf(psql_cmd + offset, sizeof(psql_cmd) - offset,
- "\"%s%spsql\" -X -a -q -d \"%s\" -v %s < \"%s\" > \"%s\" 2>&1",
+ "\"%s%spsql\" -X -a -q -d \"%s\" %s < \"%s\" > \"%s\" 2>&1",
bindir ? bindir : "",
bindir ? "/" : "",
dblist->str,
- "HIDE_TABLEAM=\"on\"",
+ "-v HIDE_TABLEAM=on -v HIDE_TOAST_COMPRESSION=on",
infile,
outfile);
if (offset >= sizeof(psql_cmd))
diff --git a/src/test/regress/serial_schedule b/src/test/regress/serial_schedule
index 6a57e88..d81d041 100644
--- a/src/test/regress/serial_schedule
+++ b/src/test/regress/serial_schedule
@@ -201,6 +201,7 @@ test: partition_aggregate
test: partition_info
test: tuplesort
test: explain
+test: compression
test: event_trigger
test: oidjoins
test: fast_default
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
new file mode 100644
index 0000000..d97e26b
--- /dev/null
+++ b/src/test/regress/sql/compression.sql
@@ -0,0 +1,136 @@
+\set HIDE_TOAST_COMPRESSION false
+
+-- test creating table with compression method
+CREATE TABLE cmdata(f1 text COMPRESSION pglz);
+CREATE INDEX idx ON cmdata(f1);
+INSERT INTO cmdata VALUES(repeat('1234567890', 1000));
+\d+ cmdata
+CREATE TABLE cmdata1(f1 TEXT COMPRESSION lz4);
+INSERT INTO cmdata1 VALUES(repeat('1234567890', 1004));
+\d+ cmdata1
+
+-- verify stored compression method in the data
+SELECT pg_column_compression(f1) FROM cmdata;
+SELECT pg_column_compression(f1) FROM cmdata1;
+
+-- decompress data slice
+SELECT SUBSTR(f1, 200, 5) FROM cmdata;
+SELECT SUBSTR(f1, 2000, 50) FROM cmdata1;
+
+-- copy with table creation
+SELECT * INTO cmmove1 FROM cmdata;
+\d+ cmmove1
+SELECT pg_column_compression(f1) FROM cmmove1;
+
+-- copy to existing table
+CREATE TABLE cmmove3(f1 text COMPRESSION pglz);
+INSERT INTO cmmove3 SELECT * FROM cmdata;
+INSERT INTO cmmove3 SELECT * FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove3;
+
+-- test LIKE INCLUDING COMPRESSION
+CREATE TABLE cmdata2 (LIKE cmdata1 INCLUDING COMPRESSION);
+\d+ cmdata2
+DROP TABLE cmdata2;
+
+-- try setting compression for incompressible data type
+CREATE TABLE cmdata2 (f1 int COMPRESSION pglz);
+
+-- update using datum from different table
+CREATE TABLE cmmove2(f1 text COMPRESSION pglz);
+INSERT INTO cmmove2 VALUES (repeat('1234567890', 1004));
+SELECT pg_column_compression(f1) FROM cmmove2;
+UPDATE cmmove2 SET f1 = cmdata1.f1 FROM cmdata1;
+SELECT pg_column_compression(f1) FROM cmmove2;
+
+-- test externally stored compressed data
+CREATE OR REPLACE FUNCTION large_val() RETURNS TEXT LANGUAGE SQL AS
+'select array_agg(md5(g::text))::text from generate_series(1, 256) g';
+CREATE TABLE cmdata2 (f1 text COMPRESSION pglz);
+INSERT INTO cmdata2 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata2;
+INSERT INTO cmdata1 SELECT large_val() || repeat('a', 4000);
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT SUBSTR(f1, 200, 5) FROM cmdata1;
+SELECT SUBSTR(f1, 200, 5) FROM cmdata2;
+DROP TABLE cmdata2;
+
+--test column type update varlena/non-varlena
+CREATE TABLE cmdata2 (f1 int);
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE int USING f1::integer;
+\d+ cmdata2
+
+--changing column storage should not impact the compression method
+--but the data should not be compressed
+ALTER TABLE cmdata2 ALTER COLUMN f1 TYPE varchar;
+\d+ cmdata2
+ALTER TABLE cmdata2 ALTER COLUMN f1 SET STORAGE plain;
+\d+ cmdata2
+INSERT INTO cmdata2 VALUES (repeat('123456789', 800));
+SELECT pg_column_compression(f1) FROM cmdata2;
+
+-- test compression with materialized view
+CREATE MATERIALIZED VIEW mv(x) AS SELECT * FROM cmdata1;
+\d+ mv
+SELECT pg_column_compression(f1) FROM cmdata1;
+SELECT pg_column_compression(x) FROM mv;
+
+-- test compression with partition
+CREATE TABLE cmpart(f1 text COMPRESSION lz4) PARTITION BY HASH(f1);
+CREATE TABLE cmpart1 PARTITION OF cmpart FOR VALUES WITH (MODULUS 2, REMAINDER 0);
+CREATE TABLE cmpart2(f1 text COMPRESSION pglz);
+
+ALTER TABLE cmpart ATTACH PARTITION cmpart2 FOR VALUES WITH (MODULUS 2, REMAINDER 1);
+INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+SELECT pg_column_compression(f1) FROM cmpart1;
+SELECT pg_column_compression(f1) FROM cmpart2;
+
+-- test compression with inheritence, error
+CREATE TABLE cminh() INHERITS(cmdata, cmdata1);
+CREATE TABLE cminh(f1 TEXT COMPRESSION lz4) INHERITS(cmdata);
+
+-- test default_toast_compression GUC
+SET default_toast_compression = '';
+SET default_toast_compression = 'I do not exist compression';
+SET default_toast_compression = 'lz4';
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 text);
+\d+ cmdata2
+
+-- test alter compression method
+ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
+INSERT INTO cmdata VALUES (repeat('123456789', 4004));
+\d+ cmdata
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- test alter compression method for the materialized view
+ALTER MATERIALIZED VIEW mv ALTER COLUMN x SET COMPRESSION lz4;
+\d+ mv
+
+-- test alter compression method for the partitioned table
+ALTER TABLE cmpart1 ALTER COLUMN f1 SET COMPRESSION pglz;
+ALTER TABLE cmpart2 ALTER COLUMN f1 SET COMPRESSION lz4;
+
+-- new data should be compressed with the current compression method
+INSERT INTO cmpart VALUES (repeat('123456789', 1004));
+INSERT INTO cmpart VALUES (repeat('123456789', 4004));
+SELECT pg_column_compression(f1) FROM cmpart1;
+SELECT pg_column_compression(f1) FROM cmpart2;
+
+--vacuum full to recompress the data
+SELECT pg_column_compression(f1) FROM cmdata;
+VACUUM FULL cmdata;
+SELECT pg_column_compression(f1) FROM cmdata;
+
+-- check data is ok
+SELECT length(f1) FROM cmdata;
+SELECT length(f1) FROM cmdata1;
+SELECT length(f1) FROM cmmove1;
+SELECT length(f1) FROM cmmove2;
+SELECT length(f1) FROM cmmove3;
+
+\set HIDE_TOAST_COMPRESSION true
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index a4f5cc4..1460537 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -485,6 +485,7 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
+ USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
--
1.8.3.1
recursive_set_compression.patchtext/x-patch; charset=US-ASCII; name=recursive_set_compression.patchDownload
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index ab89935..9649a45 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -4336,7 +4336,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
break;
case AT_SetCompression: /* ALTER COLUMN SET COMPRESSION */
ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
- /* This command never recurses */
+ ATSimpleRecursion(wqueue, rel, cmd, recurse, lockmode, context);
/* No command-specific prep needed */
pass = AT_PASS_MISC;
break;
Re: [HACKERS] Custom compression methods
I sent offlist a couple of times but notice that the latest patch is missing
this bit around AC_CHECK_HEADERS, which apparently can sometimes cause
warnings on mac.
ac_save_CPPFLAGS=$CPPFLAGS
CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS"
AC_CHECK_HEADERS(lz4/lz4.h, [],
[AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
CPPFLAGS=$ac_save_CPPFLAGS
Show quoted text
diff --git a/configure.ac b/configure.ac index 2f1585a..54efbb2 100644 --- a/configure.ac +++ b/configure.ac @@ -1410,6 +1425,11 @@ failure. It is possible the compiler isn't looking in the proper directory. Use --without-zlib to disable zlib support.])]) fi+if test "$with_lz4" = yes; then + AC_CHECK_HEADERS(lz4/lz4.h, [], + [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])]) +fi + if test "$with_gssapi" = yes ; then AC_CHECK_HEADERS(gssapi/gssapi.h, [], [AC_CHECK_HEADERS(gssapi.h, [], [AC_MSG_ERROR([gssapi.h header file is required for GSSAPI])])])
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 12:35 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I sent offlist a couple of times but notice that the latest patch is missing
this bit around AC_CHECK_HEADERS, which apparently can sometimes cause
warnings on mac.ac_save_CPPFLAGS=$CPPFLAGS
CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS"
AC_CHECK_HEADERS(lz4/lz4.h, [],
[AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
CPPFLAGS=$ac_save_CPPFLAGS
Hmm, it's working for me on macOS Catalina without this. Why do we
need it? Can you provide a patch that inserts it in the exact place
you think it needs to go?
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 01:24:39PM -0400, Robert Haas wrote:
On Fri, Mar 19, 2021 at 12:35 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I sent offlist a couple of times but notice that the latest patch is missing
this bit around AC_CHECK_HEADERS, which apparently can sometimes cause
warnings on mac.ac_save_CPPFLAGS=$CPPFLAGS
CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS"
AC_CHECK_HEADERS(lz4/lz4.h, [],
[AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
CPPFLAGS=$ac_save_CPPFLAGSHmm, it's working for me on macOS Catalina without this. Why do we
need it? Can you provide a patch that inserts it in the exact place
you think it needs to go?
Working with one of Andrey's patches on another thread, he reported offlist
getting this message, resolved by this patch. Do you see this warning during
./configure ? The latest CI is of a single patch without the LZ4 stuff, so I
can't check its log.
configure: WARNING: lz4.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: lz4.h: proceeding with the compiler's result
--
Justin
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 1:44 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Working with one of Andrey's patches on another thread, he reported offlist
getting this message, resolved by this patch. Do you see this warning during
./configure ? The latest CI is of a single patch without the LZ4 stuff, so I
can't check its log.configure: WARNING: lz4.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: lz4.h: proceeding with the compiler's result
No, I don't see this. I wonder whether this could possibly be an
installation issue on Andrey's machine? If not, it must be
version-dependent or installation-dependent in some way.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 10:11 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Also added a test case for vacuum full to recompress the data.
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Robert Haas <robertmhaas@gmail.com> writes:
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.
Since no animals will be using --with-lz4, I'd expect vast silence.
regards, tom lane
Re: [HACKERS] Custom compression methods
I wrote:
Since no animals will be using --with-lz4, I'd expect vast silence.
Nope ... crake's displeased with your assumption that it's OK to
clutter dumps with COMPRESSION clauses. As am I: that is going to
be utterly fatal for cross-version transportation of dumps.
regards, tom lane
Re: [HACKERS] Custom compression methods
Hmm, if I use configure --with-lz4, I get this:
checking whether to build with LZ4 support... yes
checking for liblz4... no
configure: error: Package requirements (liblz4) were not met:
No package 'liblz4' found
Consider adjusting the PKG_CONFIG_PATH environment variable if you
installed software in a non-standard prefix.
Alternatively, you may set the environment variables LZ4_CFLAGS
and LZ4_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
running CONFIG_SHELL=/bin/bash /bin/bash /pgsql/source/master/configure --enable-debug --enable-depend --enable-cassert --enable-nls --cache-file=/home/alvherre/run/pgconfig.master.cache --enable-thread-safety --with-python --with-perl --with-tcl --with-openssl --with-libxml --enable-tap-tests --with-tclconfig=/usr/lib/tcl8.6 PYTHON=/usr/bin/python3 --with-llvm --prefix=/pgsql/install/master --with-pgport=55432 --no-create --no-recursion
...
I find this behavior confusing; I'd rather have configure error out if
it can't find the package support I requested, than continuing with a
set of configure options different from what I gave.
--
�lvaro Herrera 39�49'30"S 73�17'W
"Postgres is bloatware by design: it was built to house
PhD theses." (Joey Hellerstein, SIGMOD annual conference 2002)
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 4:20 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Nope ... crake's displeased with your assumption that it's OK to
clutter dumps with COMPRESSION clauses. As am I: that is going to
be utterly fatal for cross-version transportation of dumps.
Yes, and prion's got this concerning diff:
Column | Type | Collation | Nullable | Default | Storage |
Compression | Stats target | Description
--------+---------+-----------+----------+---------+---------+-------------+--------------+-------------
- f1 | integer | | | | plain |
| |
+ f1 | integer | | | | plain | pglz
| |Since the column is not a varlena, it shouldn't have a compression
method configured, yet on that machine it does, possibly because that
machine uses -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE.
Regarding your point, that does look like clutter. We don't annotate
the dump with a storage clause unless it's non-default, so probably we
should do the same thing here. I think I gave Dilip bad advice here...
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On 2021-03-19 17:35:58 -0300, Alvaro Herrera wrote:
I find this behavior confusing; I'd rather have configure error out if
it can't find the package support I requested, than continuing with a
set of configure options different from what I gave.
+1
Re: [HACKERS] Custom compression methods
On 3/19/21 9:40 PM, Andres Freund wrote:
On 2021-03-19 17:35:58 -0300, Alvaro Herrera wrote:
I find this behavior confusing; I'd rather have configure error out if
it can't find the package support I requested, than continuing with a
set of configure options different from what I gave.+1
Yeah. And why does it even require pkg-config, unlike any other library
that I'm aware of?
checking for liblz4... no
configure: error: in `/home/ubuntu/postgres':
configure: error: The pkg-config script could not be found or is too
old. Make sure it
is in your PATH or set the PKG_CONFIG environment variable to the full
path to pkg-config.
Alternatively, you may set the environment variables LZ4_CFLAGS
and LZ4_LIBS to avoid the need to call pkg-config.
See the pkg-config man page for more details.
To get pkg-config, see <http://pkg-config.freedesktop.org/>.
See `config.log' for more details
I see xml2 also mentions pkg-config in configure (next to XML2_CFLAGS),
but works fine without it.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On 2021-03-19 22:19:49 +0100, Tomas Vondra wrote:
Yeah. And why does it even require pkg-config, unlike any other library
that I'm aware of?
IMO it's fine to require pkg-config to simplify the configure
code. Especially for new optional features. Adding multiple alternative
ways to discover libraries for something like this makes configure
slower, without a comensurate benefit.
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 4:38 PM Robert Haas <robertmhaas@gmail.com> wrote:
Yes, and prion's got this concerning diff:
Column | Type | Collation | Nullable | Default | Storage | Compression | Stats target | Description --------+---------+-----------+----------+---------+---------+-------------+--------------+------------- - f1 | integer | | | | plain | | | + f1 | integer | | | | plain | pglz | |Since the column is not a varlena, it shouldn't have a compression
method configured, yet on that machine it does, possibly because that
machine uses -DRELCACHE_FORCE_RELEASE -DCATCACHE_FORCE_RELEASE.
I could reproduce the problem with those flags. I pushed a fix.
Regarding your point, that does look like clutter. We don't annotate
the dump with a storage clause unless it's non-default, so probably we
should do the same thing here. I think I gave Dilip bad advice here...
Here's a patch for that. It's a little strange because you're going to
skip dumping the toast compression based on the default value on the
source system, but that might not be the default on the system where
the dump is being restored, so you could fail to recreate the state
you had. That is avoidable if you understand how things work, but some
people might not. I don't have a better idea, though, so let me know
what you think of this.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
non-default-toast-compression-only-v1.patchapplication/octet-stream; name=non-default-toast-compression-only-v1.patchDownload
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 0296b9bb5e..02019fcc84 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -210,6 +210,7 @@ typedef struct Archive
/* other important stuff */
char *searchpath; /* search_path to set during restore */
char *use_role; /* Issue SET ROLE to this */
+ char *default_toast_compression;
/* error handling */
bool exit_on_error; /* whether to exit on SQL errors... */
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f8bec3ffcc..ee2c7153e5 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -1248,6 +1248,21 @@ setup_connection(Archive *AH, const char *dumpencoding,
AH->sync_snapshot_id = get_synchronized_snapshot(AH);
}
+
+ /*
+ * Get default TOAST compression method, but not if the server's too
+ * old to support the feature or if the user doesn't want to dump that
+ * information anyway.
+ */
+ if (AH->remoteVersion >= 140000 && !dopt->no_toast_compression)
+ {
+ PGresult *res;
+
+ res = ExecuteSqlQueryForSingleRow(AH,
+ "SHOW default_toast_compression");
+ AH->default_toast_compression = pg_strdup(PQgetvalue(res, 0, 0));
+ PQclear(res);
+ }
}
/* Set up connection for a parallel worker process */
@@ -15926,7 +15941,9 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
break;
}
- if (cmname != NULL)
+ if (cmname != NULL &&
+ strcmp(cmname,
+ fout->default_toast_compression) != 0)
appendPQExpBuffer(q, " COMPRESSION %s", cmname);
}
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 5:29 PM Andres Freund <andres@anarazel.de> wrote:
On 2021-03-19 22:19:49 +0100, Tomas Vondra wrote:
Yeah. And why does it even require pkg-config, unlike any other library
that I'm aware of?IMO it's fine to require pkg-config to simplify the configure
code. Especially for new optional features. Adding multiple alternative
ways to discover libraries for something like this makes configure
slower, without a comensurate benefit.
So, would anyone like to propose a patch to revise the logic in a way
that they like better?
Here's one from me that tries to make the handling of the LZ4 stuff
more like what we already do for zlib, but I'm not sure if it's
correct, or if it's what everyone wants.
Thoughts?
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
redo-lz4-configuration.patchapplication/octet-stream; name=redo-lz4-configuration.patchDownload
diff --git a/configure b/configure
index 8176e99756..fec2f76085 100755
--- a/configure
+++ b/configure
@@ -699,8 +699,6 @@ with_gnu_ld
LD
LDFLAGS_SL
LDFLAGS_EX
-LZ4_LIBS
-LZ4_CFLAGS
with_lz4
with_zlib
with_system_tzdata
@@ -895,8 +893,6 @@ ICU_LIBS
XML2_CONFIG
XML2_CFLAGS
XML2_LIBS
-LZ4_CFLAGS
-LZ4_LIBS
LDFLAGS_EX
LDFLAGS_SL
PERL
@@ -1603,8 +1599,6 @@ Some influential environment variables:
XML2_CONFIG path to xml2-config utility
XML2_CFLAGS C compiler flags for XML2, overriding pkg-config
XML2_LIBS linker flags for XML2, overriding pkg-config
- LZ4_CFLAGS C compiler flags for LZ4, overriding pkg-config
- LZ4_LIBS linker flags for LZ4, overriding pkg-config
LDFLAGS_EX extra linker flags for linking executables only
LDFLAGS_SL extra linker flags for linking shared libraries only
PERL Perl program
@@ -8575,8 +8569,6 @@ fi
#
# LZ4
#
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking whether to build with LZ4 support" >&5
-$as_echo_n "checking whether to build with LZ4 support... " >&6; }
@@ -8585,9 +8577,7 @@ if test "${with_lz4+set}" = set; then :
withval=$with_lz4;
case $withval in
yes)
-
-$as_echo "#define USE_LZ4 1" >>confdefs.h
-
+ :
;;
no)
:
@@ -8603,105 +8593,7 @@ else
fi
-{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $with_lz4" >&5
-$as_echo "$with_lz4" >&6; }
-
-
-if test "$with_lz4" = yes; then
-
-pkg_failed=no
-{ $as_echo "$as_me:${as_lineno-$LINENO}: checking for liblz4" >&5
-$as_echo_n "checking for liblz4... " >&6; }
-
-if test -n "$LZ4_CFLAGS"; then
- pkg_cv_LZ4_CFLAGS="$LZ4_CFLAGS"
- elif test -n "$PKG_CONFIG"; then
- if test -n "$PKG_CONFIG" && \
- { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
- ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- pkg_cv_LZ4_CFLAGS=`$PKG_CONFIG --cflags "liblz4" 2>/dev/null`
- test "x$?" != "x0" && pkg_failed=yes
-else
- pkg_failed=yes
-fi
- else
- pkg_failed=untried
-fi
-if test -n "$LZ4_LIBS"; then
- pkg_cv_LZ4_LIBS="$LZ4_LIBS"
- elif test -n "$PKG_CONFIG"; then
- if test -n "$PKG_CONFIG" && \
- { { $as_echo "$as_me:${as_lineno-$LINENO}: \$PKG_CONFIG --exists --print-errors \"liblz4\""; } >&5
- ($PKG_CONFIG --exists --print-errors "liblz4") 2>&5
- ac_status=$?
- $as_echo "$as_me:${as_lineno-$LINENO}: \$? = $ac_status" >&5
- test $ac_status = 0; }; then
- pkg_cv_LZ4_LIBS=`$PKG_CONFIG --libs "liblz4" 2>/dev/null`
- test "x$?" != "x0" && pkg_failed=yes
-else
- pkg_failed=yes
-fi
- else
- pkg_failed=untried
-fi
-
-
-
-if test $pkg_failed = yes; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
-
-if $PKG_CONFIG --atleast-pkgconfig-version 0.20; then
- _pkg_short_errors_supported=yes
-else
- _pkg_short_errors_supported=no
-fi
- if test $_pkg_short_errors_supported = yes; then
- LZ4_PKG_ERRORS=`$PKG_CONFIG --short-errors --print-errors --cflags --libs "liblz4" 2>&1`
- else
- LZ4_PKG_ERRORS=`$PKG_CONFIG --print-errors --cflags --libs "liblz4" 2>&1`
- fi
- # Put the nasty error message in config.log where it belongs
- echo "$LZ4_PKG_ERRORS" >&5
-
- as_fn_error $? "Package requirements (liblz4) were not met:
-
-$LZ4_PKG_ERRORS
-Consider adjusting the PKG_CONFIG_PATH environment variable if you
-installed software in a non-standard prefix.
-
-Alternatively, you may set the environment variables LZ4_CFLAGS
-and LZ4_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details." "$LINENO" 5
-elif test $pkg_failed = untried; then
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: no" >&5
-$as_echo "no" >&6; }
- { { $as_echo "$as_me:${as_lineno-$LINENO}: error: in \`$ac_pwd':" >&5
-$as_echo "$as_me: error: in \`$ac_pwd':" >&2;}
-as_fn_error $? "The pkg-config script could not be found or is too old. Make sure it
-is in your PATH or set the PKG_CONFIG environment variable to the full
-path to pkg-config.
-
-Alternatively, you may set the environment variables LZ4_CFLAGS
-and LZ4_LIBS to avoid the need to call pkg-config.
-See the pkg-config man page for more details.
-
-To get pkg-config, see <http://pkg-config.freedesktop.org/>.
-See \`config.log' for more details" "$LINENO" 5; }
-else
- LZ4_CFLAGS=$pkg_cv_LZ4_CFLAGS
- LZ4_LIBS=$pkg_cv_LZ4_LIBS
- { $as_echo "$as_me:${as_lineno-$LINENO}: result: yes" >&5
-$as_echo "yes" >&6; }
-
-fi
- LIBS="$LZ4_LIBS $LIBS"
- CFLAGS="$LZ4_CFLAGS $CFLAGS"
-fi
#
# Assignments
@@ -12253,6 +12145,59 @@ fi
fi
+if test "$with_lz4" = yes; then
+ { $as_echo "$as_me:${as_lineno-$LINENO}: checking for LZ4_decompress_safe in -llz4" >&5
+$as_echo_n "checking for LZ4_decompress_safe in -llz4... " >&6; }
+if ${ac_cv_lib_lz4_LZ4_decompress_safe+:} false; then :
+ $as_echo_n "(cached) " >&6
+else
+ ac_check_lib_save_LIBS=$LIBS
+LIBS="-llz4 $LIBS"
+cat confdefs.h - <<_ACEOF >conftest.$ac_ext
+/* end confdefs.h. */
+
+/* Override any GCC internal prototype to avoid an error.
+ Use char because int might match the return type of a GCC
+ builtin and then its argument prototype would still apply. */
+#ifdef __cplusplus
+extern "C"
+#endif
+char LZ4_decompress_safe ();
+int
+main ()
+{
+return LZ4_decompress_safe ();
+ ;
+ return 0;
+}
+_ACEOF
+if ac_fn_c_try_link "$LINENO"; then :
+ ac_cv_lib_lz4_LZ4_decompress_safe=yes
+else
+ ac_cv_lib_lz4_LZ4_decompress_safe=no
+fi
+rm -f core conftest.err conftest.$ac_objext \
+ conftest$ac_exeext conftest.$ac_ext
+LIBS=$ac_check_lib_save_LIBS
+fi
+{ $as_echo "$as_me:${as_lineno-$LINENO}: result: $ac_cv_lib_lz4_LZ4_decompress_safe" >&5
+$as_echo "$ac_cv_lib_lz4_LZ4_decompress_safe" >&6; }
+if test "x$ac_cv_lib_lz4_LZ4_decompress_safe" = xyes; then :
+ cat >>confdefs.h <<_ACEOF
+#define HAVE_LIBLZ4 1
+_ACEOF
+
+ LIBS="-llz4 $LIBS"
+
+else
+ as_fn_error $? "lz4 library not found
+If you have liblz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support." "$LINENO" 5
+fi
+
+fi
+
if test "$enable_spinlocks" = yes; then
$as_echo "#define HAVE_SPINLOCKS 1" >>confdefs.h
diff --git a/configure.ac b/configure.ac
index 54efbb22a3..98c94e7415 100644
--- a/configure.ac
+++ b/configure.ac
@@ -989,18 +989,9 @@ AC_SUBST(with_zlib)
#
# LZ4
#
-AC_MSG_CHECKING([whether to build with LZ4 support])
-PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support],
- [AC_DEFINE([USE_LZ4], 1, [Define to 1 to build with LZ4 support. (--with-lz4)])])
-AC_MSG_RESULT([$with_lz4])
+PGAC_ARG_BOOL(with, lz4, no, [build with LZ4 support])
AC_SUBST(with_lz4)
-if test "$with_lz4" = yes; then
- PKG_CHECK_MODULES(LZ4, liblz4)
- LIBS="$LZ4_LIBS $LIBS"
- CFLAGS="$LZ4_CFLAGS $CFLAGS"
-fi
-
#
# Assignments
#
@@ -1192,6 +1183,14 @@ failure. It is possible the compiler isn't looking in the proper directory.
Use --without-zlib to disable zlib support.])])
fi
+if test "$with_lz4" = yes; then
+ AC_CHECK_LIB(lz4, LZ4_decompress_safe, [],
+ [AC_MSG_ERROR([lz4 library not found
+If you have liblz4 already installed, see config.log for details on the
+failure. It is possible the compiler isn't looking in the proper directory.
+Use --without-lz4 to disable lz4 support.])])
+fi
+
if test "$enable_spinlocks" = yes; then
AC_DEFINE(HAVE_SPINLOCKS, 1, [Define to 1 if you have spinlocks.])
else
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index a6f8b79a9e..6315e8d882 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -13,7 +13,7 @@
*/
#include "postgres.h"
-#ifdef USE_LZ4
+#ifdef HAVE_LIBLZ4
#include <lz4.h>
#endif
@@ -133,7 +133,7 @@ pglz_decompress_datum_slice(const struct varlena *value,
struct varlena *
lz4_compress_datum(const struct varlena *value)
{
-#ifndef USE_LZ4
+#ifndef HAVE_LIBLZ4
NO_LZ4_SUPPORT();
#else
int32 valsize;
@@ -175,7 +175,7 @@ lz4_compress_datum(const struct varlena *value)
struct varlena *
lz4_decompress_datum(const struct varlena *value)
{
-#ifndef USE_LZ4
+#ifndef HAVE_LIBLZ4
NO_LZ4_SUPPORT();
#else
int32 rawsize;
@@ -207,7 +207,7 @@ lz4_decompress_datum(const struct varlena *value)
struct varlena *
lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
{
-#ifndef USE_LZ4
+#ifndef HAVE_LIBLZ4
NO_LZ4_SUPPORT();
#else
int32 rawsize;
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 514df0bed1..e0b164949a 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -85,7 +85,7 @@ CompressionNameToMethod(char *compression)
return TOAST_PGLZ_COMPRESSION;
else if (strcmp(compression, "lz4") == 0)
{
-#ifndef USE_LZ4
+#ifndef HAVE_LIBLZ4
NO_LZ4_SUPPORT();
#endif
return TOAST_LZ4_COMPRESSION;
diff --git a/src/include/pg_config.h.in b/src/include/pg_config.h.in
index 0a6422da4f..b204357c5a 100644
--- a/src/include/pg_config.h.in
+++ b/src/include/pg_config.h.in
@@ -319,6 +319,9 @@
/* Define to 1 if you have the `ldap_r' library (-lldap_r). */
#undef HAVE_LIBLDAP_R
+/* Define to 1 if you have the `lz4' library (-llz4). */
+#undef HAVE_LIBLZ4
+
/* Define to 1 if you have the `m' library (-lm). */
#undef HAVE_LIBM
@@ -358,6 +361,12 @@
/* Define to 1 if `long long int' works and is 64 bits. */
#undef HAVE_LONG_LONG_INT_64
+/* Define to 1 if you have the <lz4.h> header file. */
+#undef HAVE_LZ4_H
+
+/* Define to 1 if you have the <lz4/lz4.h> header file. */
+#undef HAVE_LZ4_LZ4_H
+
/* Define to 1 if you have the <mbarrier.h> header file. */
#undef HAVE_MBARRIER_H
@@ -899,9 +908,6 @@
/* Define to 1 to build with LLVM based JIT support. (--with-llvm) */
#undef USE_LLVM
-/* Define to 1 to build with LZ4 support (--with-lz4) */
-#undef USE_LZ4
-
/* Define to select named POSIX semaphores. */
#undef USE_NAMED_POSIX_SEMAPHORES
diff --git a/src/tools/msvc/Solution.pm b/src/tools/msvc/Solution.pm
index 14605371bb..2520bb2fb0 100644
--- a/src/tools/msvc/Solution.pm
+++ b/src/tools/msvc/Solution.pm
@@ -298,6 +298,7 @@ sub GenerateFiles
HAVE_LIBCRYPTO => undef,
HAVE_LIBLDAP => undef,
HAVE_LIBLDAP_R => undef,
+ HAVE_LIBLZ4 => undef,
HAVE_LIBM => undef,
HAVE_LIBPAM => undef,
HAVE_LIBREADLINE => undef,
@@ -485,7 +486,6 @@ sub GenerateFiles
USE_ICU => $self->{options}->{icu} ? 1 : undef,
USE_LIBXML => undef,
USE_LIBXSLT => undef,
- USE_LZ4 => undef,
USE_LDAP => $self->{options}->{ldap} ? 1 : undef,
USE_LLVM => undef,
USE_NAMED_POSIX_SEMAPHORES => undef,
Re: [HACKERS] Custom compression methods
On 2021-Mar-19, Robert Haas wrote:
Regarding your point, that does look like clutter. We don't annotate
the dump with a storage clause unless it's non-default, so probably we
should do the same thing here. I think I gave Dilip bad advice here...Here's a patch for that. It's a little strange because you're going to
skip dumping the toast compression based on the default value on the
source system, but that might not be the default on the system where
the dump is being restored, so you could fail to recreate the state
you had. That is avoidable if you understand how things work, but some
people might not. I don't have a better idea, though, so let me know
what you think of this.
Do you mean the column storage strategy, attstorage? I don't think
that's really related, because the difference there is not a GUC setting
but a compiled-in default for the type. In the case of compression, I'm
not sure it makes sense to do it like that, but I can see the clutter
argument: if we dump compression for all columns, it's going to be super
noisy.
(At least, for binary upgrade surely you must make sure to apply the
correct setting regardless of defaults on either system).
Maybe it makes sense to dump the compression clause if it is different
from pglz, regardless of the default on the source server. Then, if the
target server has chosen lz4 as default, *all* columns are going to end
up as lz4, and if it hasn't, then only the ones that were lz4 in the
source server are going to. That seems reasonable behavior. Also, if
some columns are lz4 in source, and target does not have lz4, then
everything is going to work out to not-lz4 with just a bunch of errors
in the output.
--
�lvaro Herrera 39�49'30"S 73�17'W
Re: [HACKERS] Custom compression methods
On 2021-Mar-19, Robert Haas wrote:
On Fri, Mar 19, 2021 at 10:11 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Also added a test case for vacuum full to recompress the data.
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.
I updated the coverage script to use --with-lz4; results are updated.
While eyeballing the results I noticed this bit in
lz4_decompress_datum_slice():
+ /* slice decompression not supported prior to 1.8.3 */
+ if (LZ4_versionNumber() < 10803)
+ return lz4_decompress_datum(value);which I read as returning the complete decompressed datum if slice
decompression is not supported. I thought that was a bug, but looking
at the caller I realize that this isn't really a problem, since it's
detoast_attr_slice's responsibility to slice the result further -- no
bug, it's just wasteful. I suggest to add comments to this effect,
perhaps as the attached (feel free to reword, I think mine is awkward.)
--
�lvaro Herrera 39�49'30"S 73�17'W
Si no sabes adonde vas, es muy probable que acabes en otra parte.
Attachments:
comments.patchtext/x-diff; charset=us-asciiDownload
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 2fef40c2e9..6c79bd2a40 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -497,6 +497,9 @@ toast_decompress_datum(struct varlena *attr)
* Decompress the front of a compressed version of a varlena datum.
* offset handling happens in detoast_attr_slice.
* Here we just decompress a slice from the front.
+ *
+ * If slice decompression is not supported, the return datum is the
+ * decompression of the full datum.
*/
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index a6f8b79a9e..080ddcf93c 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -203,6 +203,9 @@ lz4_decompress_datum(const struct varlena *value)
/*
* Decompress part of a varlena that was compressed using LZ4.
+ *
+ * If slice decompression is not supported, the return datum is the
+ * decompression of the full datum.
*/
struct varlena *
lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 10:19:49PM +0100, Tomas Vondra wrote:
On 3/19/21 9:40 PM, Andres Freund wrote:
On 2021-03-19 17:35:58 -0300, Alvaro Herrera wrote:
I find this behavior confusing; I'd rather have configure error out if
it can't find the package support I requested, than continuing with a
set of configure options different from what I gave.+1
Yeah. And why does it even require pkg-config, unlike any other library
that I'm aware of?
The discussion regarding pkg-config started here.
/messages/by-id/20210309071655.GL2021@telsasoft.com
I sent a patch adding it to allow the macos CI to build --with-lz4.
Since LZ4 was only recently installed on the CI environments, it's possible
that's not the ideal way to do it.
--
Justin
Re: [HACKERS] Custom compression methods
On 2021-Mar-19, Robert Haas wrote:
Here's one from me that tries to make the handling of the LZ4 stuff
more like what we already do for zlib, but I'm not sure if it's
correct, or if it's what everyone wants.
This one seems to behave as expected (Debian 10, with and without
liblz4-dev).
--
�lvaro Herrera Valdivia, Chile
"Just treat us the way you want to be treated + some extra allowance
for ignorance." (Michael Brusser)
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 6:22 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
Do you mean the column storage strategy, attstorage? I don't think
that's really related, because the difference there is not a GUC setting
but a compiled-in default for the type. In the case of compression, I'm
not sure it makes sense to do it like that, but I can see the clutter
argument: if we dump compression for all columns, it's going to be super
noisy.
I agree.
(At least, for binary upgrade surely you must make sure to apply the
correct setting regardless of defaults on either system).
It's not critical from a system integrity point of view; the catalog
state just dictates what happens to new data. You could argue that if,
in a future release, we change the default to lz4, it's good for
pg_upgrade to migrate users to a set of column definitions that will
use that for new data.
Maybe it makes sense to dump the compression clause if it is different
from pglz, regardless of the default on the source server. Then, if the
target server has chosen lz4 as default, *all* columns are going to end
up as lz4, and if it hasn't, then only the ones that were lz4 in the
source server are going to. That seems reasonable behavior. Also, if
some columns are lz4 in source, and target does not have lz4, then
everything is going to work out to not-lz4 with just a bunch of errors
in the output.
Well, I really do hope that some day in the bright future, pglz will
no longer be the thing we're shipping as the postgresql.conf default.
So we'd just be postponing the noise until then. I think we need a
better idea than that.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On 2021-Mar-19, Robert Haas wrote:
On Fri, Mar 19, 2021 at 6:22 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
(At least, for binary upgrade surely you must make sure to apply the
correct setting regardless of defaults on either system).It's not critical from a system integrity point of view; the catalog
state just dictates what happens to new data.
Oh, okay.
You could argue that if, in a future release, we change the default to
lz4, it's good for pg_upgrade to migrate users to a set of column
definitions that will use that for new data.
Agreed, that seems a worthy goal.
Maybe it makes sense to dump the compression clause if it is different
from pglz, regardless of the default on the source server.Well, I really do hope that some day in the bright future, pglz will
no longer be the thing we're shipping as the postgresql.conf default.
So we'd just be postponing the noise until then. I think we need a
better idea than that.
Hmm, why? In that future, we can just change the pg_dump behavior to no
longer dump the compression clause if it's lz4 or whatever better
algorithm we choose. So I think I'm clarifying my proposal to be "dump
the compression clause if it's different from the compiled-in default"
rather than "different from the GUC default".
--
�lvaro Herrera Valdivia, Chile
"Para tener m�s hay que desear menos"
Re: [HACKERS] Custom compression methods
On 2021-03-19 15:44:34 -0400, Robert Haas wrote:
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.
Congrats Dilip, Robert, All. The slow toast compression has been a
significant issue for a long time.
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 04:38:03PM -0400, Robert Haas wrote:
On Fri, Mar 19, 2021 at 4:20 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Nope ... crake's displeased with your assumption that it's OK to
clutter dumps with COMPRESSION clauses. As am I: that is going to
be utterly fatal for cross-version transportation of dumps.Regarding your point, that does look like clutter. We don't annotate
the dump with a storage clause unless it's non-default, so probably we
should do the same thing here. I think I gave Dilip bad advice here...
On Fri, Mar 19, 2021 at 05:49:37PM -0400, Robert Haas wrote:
Here's a patch for that. It's a little strange because you're going to
skip dumping the toast compression based on the default value on the
source system, but that might not be the default on the system where
the dump is being restored, so you could fail to recreate the state
you had. That is avoidable if you understand how things work, but some
people might not. I don't have a better idea, though, so let me know
what you think of this.
On Fri, Mar 19, 2021 at 07:22:42PM -0300, Alvaro Herrera wrote:
Do you mean the column storage strategy, attstorage? I don't think
that's really related, because the difference there is not a GUC setting
but a compiled-in default for the type. In the case of compression, I'm
not sure it makes sense to do it like that, but I can see the clutter
argument: if we dump compression for all columns, it's going to be super
noisy.(At least, for binary upgrade surely you must make sure to apply the
correct setting regardless of defaults on either system).Maybe it makes sense to dump the compression clause if it is different
from pglz, regardless of the default on the source server. Then, if the
target server has chosen lz4 as default, *all* columns are going to end
up as lz4, and if it hasn't, then only the ones that were lz4 in the
source server are going to. That seems reasonable behavior. Also, if
some columns are lz4 in source, and target does not have lz4, then
everything is going to work out to not-lz4 with just a bunch of errors
in the output.
I think what's missing is dumping the GUC value itself, and then also dump any
columns that differ from the GUC's setting. An early version of the GUC patch
actually had an "XXX" comment about pg_dump support, and I was waiting for a
review before polishing it. This was modelled after default_tablespace and
default_table_access_method - I've mentioned that before that there's no
pg_restore --no-table-am, and I have an unpublished patch to add it. That may
be how I missed this until now.
Then, this will output COMPRESSION on "a" (x)or "b" depending on the current
default:
| CREATE TABLE a(a text compression lz4, b text compression pglz);
When we restore it, we set the default before restoring columns.
I think it may be a good idea to document that dumps of columns with
non-default compression aren't portable to older server versions, or servers
--without-lz4. This is a consequence of the CREATE command being a big text
blob, so pg_restore can't reasonably elide the COMPRESSION clause.
While looking at this, I realized that someone added the GUC to
postgresql.conf.sample, but not to doc/ - this was a separate patch until
yesterday.
I think since we're not doing catalog access for "pluggable" compression, this
should just be an enum GUC, with #ifdef LZ4. Then we don't need a hook to
validate it.
ALTER and CREATE are silently accepting bogus compression names.
I can write patches for these later.
--
Justin
Import Notes
Reply to msg id not found: 20210319222242.GA23930@alvherre.pgsqlCA+TgmoYQ8hn1apJXbmxutYv-sHkTOUdPxdFJUzvZ7uT4cUT+cQ@mail.gmail.comCA+TgmobGm9umTgno4O4UqSAoGhOPOAA55uiKE8N_wehnHfg4Tw@mail.gmail.com4158590.1616185209@sss.pgh.pa.us | Resolved by subject fallback
Re: [HACKERS] Custom compression methods
On 3/19/21 8:00 PM, Andres Freund wrote:
On 2021-03-19 15:44:34 -0400, Robert Haas wrote:
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.Congrats Dilip, Robert, All. The slow toast compression has been a
significant issue for a long time.
Yes, congratulations! This is a terrific improvement.
Plus, now that lz4 is part of configure it lowers the bar for other
features that want to use it. I'm guessing there will be a few.
Thanks!
--
-David
david@pgmasters.net
Re: [HACKERS] Custom compression methods
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
On 2021-Mar-19, Robert Haas wrote:
Well, I really do hope that some day in the bright future, pglz will
no longer be the thing we're shipping as the postgresql.conf default.
So we'd just be postponing the noise until then. I think we need a
better idea than that.
Hmm, why? In that future, we can just change the pg_dump behavior to no
longer dump the compression clause if it's lz4 or whatever better
algorithm we choose. So I think I'm clarifying my proposal to be "dump
the compression clause if it's different from the compiled-in default"
rather than "different from the GUC default".
Extrapolating from the way we've dealt with similar issues
in the past, I think the structure of pg_dump's output ought to be:
1. SET default_toast_compression = 'source system's value'
in among the existing passel of SETs at the top. Doesn't
matter whether or not that is the compiled-in value.
2. No mention of compression in any CREATE TABLE command.
3. For any column having a compression option different from
the default, emit ALTER TABLE SET ... to set that option after
the CREATE TABLE. (You did implement such a SET, I trust.)
This minimizes the chatter for the normal case where all or most
columns have the same setting, and more importantly it allows the
dump to be read by older PG systems (or non-PG systems, or newer
systems built without --with-lz4) that would fail altogether
if the CREATE TABLE commands contained compression options.
To use the dump that way, you do have to be willing to ignore
errors from the SET and the ALTERs ... but that beats the heck
out of having to manually edit the dump script to get rid of
embedded COMPRESSION clauses.
I'm not sure whether we'd still need to mess around beyond
that to make the buildfarm's existing upgrade tests happy.
But we *must* do this much in any case, because as it stands
this patch has totally destroyed some major use-cases for
pg_dump.
There might be scope for a dump option to suppress mention
of compression altogether (comparable to, eg, --no-tablespaces).
But I think that's optional. In any case, we don't want
to put people in a position where they should have used such
an option and now they have no good way to recover their
dump to the system they want to recover to.
regards, tom lane
Re: [HACKERS] Custom compression methods
On 3/19/21 8:25 PM, Tom Lane wrote:
Alvaro Herrera <alvherre@alvh.no-ip.org> writes:
On 2021-Mar-19, Robert Haas wrote:
Well, I really do hope that some day in the bright future, pglz will
no longer be the thing we're shipping as the postgresql.conf default.
So we'd just be postponing the noise until then. I think we need a
better idea than that.Hmm, why? In that future, we can just change the pg_dump behavior to no
longer dump the compression clause if it's lz4 or whatever better
algorithm we choose. So I think I'm clarifying my proposal to be "dump
the compression clause if it's different from the compiled-in default"
rather than "different from the GUC default".Extrapolating from the way we've dealt with similar issues
in the past, I think the structure of pg_dump's output ought to be:1. SET default_toast_compression = 'source system's value'
in among the existing passel of SETs at the top. Doesn't
matter whether or not that is the compiled-in value.2. No mention of compression in any CREATE TABLE command.
3. For any column having a compression option different from
the default, emit ALTER TABLE SET ... to set that option after
the CREATE TABLE. (You did implement such a SET, I trust.)This minimizes the chatter for the normal case where all or most
columns have the same setting, and more importantly it allows the
dump to be read by older PG systems (or non-PG systems, or newer
systems built without --with-lz4) that would fail altogether
if the CREATE TABLE commands contained compression options.
To use the dump that way, you do have to be willing to ignore
errors from the SET and the ALTERs ... but that beats the heck
out of having to manually edit the dump script to get rid of
embedded COMPRESSION clauses.I'm not sure whether we'd still need to mess around beyond
that to make the buildfarm's existing upgrade tests happy.
But we *must* do this much in any case, because as it stands
this patch has totally destroyed some major use-cases for
pg_dump.There might be scope for a dump option to suppress mention
of compression altogether (comparable to, eg, --no-tablespaces).
But I think that's optional. In any case, we don't want
to put people in a position where they should have used such
an option and now they have no good way to recover their
dump to the system they want to recover to.
I'm fairly sure this prescription would satisfy the buildfarm. It sounds
pretty sane to me - I'd independently come to a very similar conclusion
before reading the above.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 8:25 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Extrapolating from the way we've dealt with similar issues
in the past, I think the structure of pg_dump's output ought to be:1. SET default_toast_compression = 'source system's value'
in among the existing passel of SETs at the top. Doesn't
matter whether or not that is the compiled-in value.2. No mention of compression in any CREATE TABLE command.
3. For any column having a compression option different from
the default, emit ALTER TABLE SET ... to set that option after
the CREATE TABLE. (You did implement such a SET, I trust.)
Actually, *I* didn't implement any of this. But ALTER TABLE sometab
ALTER somecol SET COMPRESSION somealgo works.
This sounds like a reasonable approach.
There might be scope for a dump option to suppress mention
of compression altogether (comparable to, eg, --no-tablespaces).
But I think that's optional. In any case, we don't want
to put people in a position where they should have used such
an option and now they have no good way to recover their
dump to the system they want to recover to.
The patch already has --no-toast-compression.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 6:38 PM Alvaro Herrera <alvherre@alvh.no-ip.org> wrote:
I suggest to add comments to this effect,
perhaps as the attached (feel free to reword, I think mine is awkward.)
It's not bad, although "the decompressed version of the full datum"
might be a little better. I'd probably say instead: "This method might
decompress the entire datum rather than just a slice, if slicing is
not supported." or something of to that effect. Feel free to commit
something you like.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
See attached.
One issue is that the pg_dump tests are no longer exercising the COMPRESSION
clause. I don't know how to improve on that, since lz4 may not be available.
..unless we changed attcompression='\0' to mean (for varlena) "the default
compression". Rather than "resolving" to the default compression at the time
the table is created, columns without an explicit compression set would "defer"
to the GUC (of course, that only affects newly-inserted data).
Then, I think pg_dump would generate an COMPRESSION clause for columns with any
compression other than a null byte, and then the tests could "SET COMPRESSION
pglz" and check the output, since it's set to a specific compression, not just
inheriting the default.
I'm not sure if that'd be desirable, but I think that's similar to tablespaces,
where (if I recall) reltablespace=0 means "this database's default tblspc".
--
Justin
Attachments:
0001-Add-docs-for-default_toast_compression.patchtext/x-diff; charset=us-asciiDownload
From 58335318a9e72d0dbdf8c2009ba6d195a6cba862 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 19:12:53 -0500
Subject: [PATCH 1/9] Add docs for default_toast_compression..
bbe0a81db69bd10bd166907c3701492a29aca294
---
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index ee4925d6d9..5cb851a5eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8151,6 +8151,29 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
+ <term><varname>default_toast_compression</varname> (<type>string</type>)
+ <indexterm>
+ <primary><varname>default_toast_compression</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the default compression method to use
+ when compressing data in <acronym>TOAST</acronym> tables.
+ It applies only to variable-width data types.
+ It may be overriden by compression clauses in the
+ <command>CREATE</command> command, or changed after the relation is
+ created by <command>ALTER TABLE ... SET COMPRESSION</command>.
+
+ The supported compression methods are <literal>pglz</literal> and
+ (if configured at the time <productname>PostgreSQL</productname> was
+ built) <literal>lz4</literal>.
+ The default is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-temp-tablespaces" xreflabel="temp_tablespaces">
<term><varname>temp_tablespaces</varname> (<type>string</type>)
<indexterm>
--
2.17.0
0002-doc-pg_dump-no-toast-compression.patchtext/x-diff; charset=us-asciiDownload
From ae52d1db7fa5f7731bc5c1170dfb2ec5c39b111a Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 21:52:28 -0500
Subject: [PATCH 2/9] doc: pg_dump --no-toast-compression
---
doc/src/sgml/ref/pg_dump.sgml | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index bcbb7a25fb..4a521186fb 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -931,6 +931,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-toast-compression</option></term>
+ <listitem>
+ <para>
+ Do not output commands to set <acronym>TOAST</acronym>compression
+ methods.
+ With this option, all objects will be created using whichever
+ compression method is the default during restore.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-unlogged-table-data</option></term>
<listitem>
--
2.17.0
0003-Compression-method-is-an-char-not-an-OID.patchtext/x-diff; charset=us-asciiDownload
From c47fafc376f847f4463b146f467c6046e44e5afa Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:23:40 -0500
Subject: [PATCH 3/9] Compression method is an char not an OID
---
src/backend/commands/tablecmds.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9b2800bf5e..8e756e59d5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -7847,6 +7847,7 @@ SetIndexStorageProperties(Relation rel, Relation attrelation,
index_close(indrel, lockmode);
}
}
+
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -15070,7 +15071,7 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
char *compression;
char typstorage;
- Oid cmoid;
+ char cmethod;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
@@ -15111,9 +15112,9 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmethod = GetAttributeCompression(atttableform, compression);
- atttableform->attcompression = cmoid;
+ atttableform->attcompression = cmethod;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -15123,7 +15124,7 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
/* apply changes to the index column as well */
- SetIndexStorageProperties(rel, attrel, attnum, cmoid, '\0', lockmode);
+ SetIndexStorageProperties(rel, attrel, attnum, cmethod, '\0', lockmode);
table_close(attrel, RowExclusiveLock);
/* make changes visible */
--
2.17.0
0004-Remove-duplicative-macro.patchtext/x-diff; charset=us-asciiDownload
From f577d25165815993b0289c75d5395d23409863cc Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:07:31 -0500
Subject: [PATCH 4/9] Remove duplicative macro
---
src/include/access/toast_compression.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 514df0bed1..09af6e7810 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -50,8 +50,6 @@ typedef enum ToastCompressionId
errdetail("This functionality requires the server to be built with lz4 support."), \
errhint("You need to rebuild PostgreSQL using --with-lz4.")))
-#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
-
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
--
2.17.0
0005-Error-on-invalid-compression-in-CREATE-and-ALTER.patchtext/x-diff; charset=us-asciiDownload
From 40aabd1aa36878974e84fc04ccff31d1e90f93c5 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:19:55 -0500
Subject: [PATCH 5/9] Error on invalid compression in CREATE and ALTER
---
src/backend/commands/tablecmds.c | 10 +++++++---
src/test/regress/expected/compression.out | 6 ++++++
src/test/regress/sql/compression.sql | 5 +++++
3 files changed, 18 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8e756e59d5..7efe27ac6c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17865,9 +17865,13 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- cmethod = GetDefaultToastCompression();
- else
- cmethod = CompressionNameToMethod(compression);
+ return GetDefaultToastCompression();
+
+ cmethod = CompressionNameToMethod(compression);
+ if (!CompressionMethodIsValid(cmethod))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("invalid compression method \"%s\"", compression)));
return cmethod;
}
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3de2886de0..501e0ead14 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -344,4 +344,10 @@ SELECT length(f1) FROM cmmove3;
10040
(2 rows)
+CREATE TABLE t (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+CREATE TABLE t (a text);
+ALTER TABLE t ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+DROP TABLE t;
\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index d97e26b6ee..afddd5a134 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -133,4 +133,9 @@ SELECT length(f1) FROM cmmove1;
SELECT length(f1) FROM cmmove2;
SELECT length(f1) FROM cmmove3;
+CREATE TABLE t (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+CREATE TABLE t (a text);
+ALTER TABLE t ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+DROP TABLE t;
+
\set HIDE_TOAST_COMPRESSION true
--
2.17.0
0006-WIP-Change-default_toast_compression-GUC-to-an-enum.patchtext/x-diff; charset=us-asciiDownload
From f9c8398bad190f85729c8bdece0fbd205e7b55db Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 19:39:10 -0500
Subject: [PATCH 6/9] WIP: Change default_toast_compression GUC to an enum..
..since the final version of this patch doesn't require catalog access anymore
---
src/backend/access/common/toast_compression.c | 49 +------------------
src/backend/utils/misc/guc.c | 30 +++++++-----
src/include/access/toast_compression.h | 10 ++--
src/test/regress/expected/compression.out | 4 +-
4 files changed, 28 insertions(+), 65 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index a6f8b79a9e..edcd09f9e9 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -23,8 +23,8 @@
#include "fmgr.h"
#include "utils/builtins.h"
-/* Compile-time default */
-char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* GUC */
+int default_toast_compression = TOAST_PGLZ_COMPRESSION_ID;
/*
* Compress a varlena using PGLZ.
@@ -266,48 +266,3 @@ toast_get_compression_id(struct varlena *attr)
return cmid;
}
-
-/*
- * Validate a new value for the default_toast_compression GUC.
- */
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
-{
- if (**newval == '\0')
- {
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
- }
-
- if (strlen(*newval) >= NAMEDATALEN)
- {
- GUC_check_errdetail("%s is too long (maximum %d characters).",
- "default_toast_compression", NAMEDATALEN - 1);
- return false;
- }
-
- if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
- {
- /*
- * When source == PGC_S_TEST, don't throw a hard error for a
- * nonexistent compression method, only a NOTICE. See comments in
- * guc.h.
- */
- if (source == PGC_S_TEST)
- {
- ereport(NOTICE,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("compression method \"%s\" does not exist",
- *newval)));
- }
- else
- {
- GUC_check_errdetail("Compression method \"%s\" does not exist.",
- *newval);
- return false;
- }
- }
-
- return true;
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index fb31c8b05c..0556f2e748 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -510,6 +510,14 @@ static struct config_enum_entry shared_memory_options[] = {
{NULL, 0, false}
};
+static struct config_enum_entry default_toast_compression_options[] = {
+ {"pglz", TOAST_PGLZ_COMPRESSION_ID, false},
+#ifdef USE_LZ4
+ {"lz4", TOAST_LZ4_COMPRESSION_ID, false},
+#endif
+ {NULL, 0, false}
+};
+
/*
* Options for enum values stored in other modules
*/
@@ -3953,17 +3961,6 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
- {
- {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default compression for new columns."),
- NULL,
- GUC_IS_NAME
- },
- &default_toast_compression,
- DEFAULT_TOAST_COMPRESSION,
- check_default_toast_compression, NULL, NULL
- },
-
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
@@ -4605,6 +4602,17 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ 0, /* pglz */
+ default_toast_compression_options, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 09af6e7810..88a46e264d 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -16,10 +16,7 @@
#include "utils/guc.h"
/* GUCs */
-extern char *default_toast_compression;
-
-/* default compression method if not specified. */
-#define DEFAULT_TOAST_COMPRESSION "pglz"
+extern int default_toast_compression;
/*
* Built-in compression method-id. The toast compression header will store
@@ -100,7 +97,10 @@ CompressionNameToMethod(char *compression)
static inline char
GetDefaultToastCompression(void)
{
- return CompressionNameToMethod(default_toast_compression);
+ /* See also guc.c */
+ char *const enumtoname[] = {"pglz", "lz4"};
+ Assert(default_toast_compression < lengthof(enumtoname));
+ return CompressionNameToMethod(enumtoname[default_toast_compression]);
}
/* pglz compression/decompression routines */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 501e0ead14..5cc76cdcd6 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -232,10 +232,10 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
--
2.17.0
0007-attcompression-is-a-list-of-chars-and-never-null.patchtext/x-diff; charset=us-asciiDownload
From a8a477118e7df9316efa2e7f1cdf43d316b51a05 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 20 Mar 2021 01:55:59 -0500
Subject: [PATCH 7/9] attcompression is a list of chars, and never null
Robert seems to have noticed half of this here: CA+TgmoYRsapWz2vDSQXFAEL_BAMq1aY37avcq9GAB=SUUvFhaw@mail.gmail.com
---
src/bin/pg_dump/pg_dump.c | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f8bec3ffcc..aa7c004147 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -8759,7 +8759,7 @@ getTableAttrs(Archive *fout, TableInfo *tblinfo, int numTables)
tbinfo->attcollation = (Oid *) pg_malloc(ntups * sizeof(Oid));
tbinfo->attfdwoptions = (char **) pg_malloc(ntups * sizeof(char *));
tbinfo->attmissingval = (char **) pg_malloc(ntups * sizeof(char *));
- tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char *));
+ tbinfo->attcompression = (char *) pg_malloc(ntups * sizeof(char));
tbinfo->notnull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->inhNotNull = (bool *) pg_malloc(ntups * sizeof(bool));
tbinfo->attrdefs = (AttrDefInfo **) pg_malloc(ntups * sizeof(AttrDefInfo *));
@@ -15908,8 +15908,7 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
/*
* Attribute compression
*/
- if (!dopt->no_toast_compression &&
- tbinfo->attcompression != NULL)
+ if (!dopt->no_toast_compression)
{
char *cmname;
--
2.17.0
0008-WIP-pg_dump-output-default_toast_compression.patchtext/x-diff; charset=us-asciiDownload
From e9e84a09c0e63a9bb487bc6a1800de90492e4ad6 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 22:49:59 -0500
Subject: [PATCH 8/9] WIP: pg_dump: output default_toast_compression..
This is unlike default_tablespace in that tablespaces are associated with an
entire relation, whereas TOAST compression is associated with individual
columns, so cannot SET default_toast_compression between CREATE commands.
Rather, each column should have a COMPRESSION clause if its compression differs
from the dump-time default. This commit sets the default during restore to
match the dump-time default so that does the right thing.
XXX: need to bump archive version?
---
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_backup_archiver.c | 20 +++++++++++-
src/bin/pg_dump/pg_dump.c | 48 ++++++++++++++++++++++++++--
3 files changed, 66 insertions(+), 3 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 0296b9bb5e..dbe2689f23 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -209,6 +209,7 @@ typedef struct Archive
/* other important stuff */
char *searchpath; /* search_path to set during restore */
+ char *default_toast_compression; /* default TOAST compression to set during restore */
char *use_role; /* Issue SET ROLE to this */
/* error handling */
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 35bf971220..d7fcbc5410 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -86,6 +86,7 @@ static void _selectTableAccessMethod(ArchiveHandle *AH, const char *tableam);
static void processEncodingEntry(ArchiveHandle *AH, TocEntry *te);
static void processStdStringsEntry(ArchiveHandle *AH, TocEntry *te);
static void processSearchPathEntry(ArchiveHandle *AH, TocEntry *te);
+static void processToastCompressionEntry(ArchiveHandle *AH, TocEntry *te);
static int _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH);
static RestorePass _tocEntryRestorePass(TocEntry *te);
static bool _tocEntryIsACL(TocEntry *te);
@@ -2696,6 +2697,8 @@ ReadToc(ArchiveHandle *AH)
processStdStringsEntry(AH, te);
else if (strcmp(te->desc, "SEARCHPATH") == 0)
processSearchPathEntry(AH, te);
+ else if (strcmp(te->desc, "TOAST_COMPRESSION") == 0)
+ processToastCompressionEntry(AH, te);
}
}
@@ -2753,6 +2756,16 @@ processSearchPathEntry(ArchiveHandle *AH, TocEntry *te)
AH->public.searchpath = pg_strdup(te->defn);
}
+static void
+processToastCompressionEntry(ArchiveHandle *AH, TocEntry *te)
+{
+ /*
+ * te->defn should contain a command to set default_toast_compression.
+ * We just copy it verbatim for use later.
+ */
+ AH->public.default_toast_compression = pg_strdup(te->defn);
+}
+
static void
StrictNamesCheck(RestoreOptions *ropt)
{
@@ -2812,7 +2825,8 @@ _tocEntryRequired(TocEntry *te, teSection curSection, ArchiveHandle *AH)
/* These items are treated specially */
if (strcmp(te->desc, "ENCODING") == 0 ||
strcmp(te->desc, "STDSTRINGS") == 0 ||
- strcmp(te->desc, "SEARCHPATH") == 0)
+ strcmp(te->desc, "SEARCHPATH") == 0 ||
+ strcmp(te->desc, "TOAST_COMPRESSION") == 0)
return REQ_SPECIAL;
/*
@@ -3135,6 +3149,10 @@ _doSetFixedOutputState(ArchiveHandle *AH)
if (AH->public.searchpath)
ahprintf(AH, "%s", AH->public.searchpath);
+ /* Select the dump-time default_toast_compression */
+ if (AH->public.default_toast_compression)
+ ahprintf(AH, "%s", AH->public.default_toast_compression);
+
/* Make sure function checking is disabled */
ahprintf(AH, "SET check_function_bodies = false;\n");
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index aa7c004147..12d1ebf466 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -270,6 +270,7 @@ static void dumpDatabaseConfig(Archive *AH, PQExpBuffer outbuf,
static void dumpEncoding(Archive *AH);
static void dumpStdStrings(Archive *AH);
static void dumpSearchPath(Archive *AH);
+static void dumpToastCompression(Archive *AH);
static void binary_upgrade_set_type_oids_by_type_oid(Archive *fout,
PQExpBuffer upgrade_buffer,
Oid pg_type_oid,
@@ -909,10 +910,14 @@ main(int argc, char **argv)
* order.
*/
- /* First the special ENCODING, STDSTRINGS, and SEARCHPATH entries. */
+ /*
+ * First the special entries for ENCODING, STDSTRINGS, SEARCHPATH and
+ * TOAST_COMPRESSION.
+ */
dumpEncoding(fout);
dumpStdStrings(fout);
dumpSearchPath(fout);
+ dumpToastCompression(fout);
/* The database items are always next, unless we don't want them at all */
if (dopt.outputCreateDB)
@@ -1048,7 +1053,7 @@ help(const char *progname)
printf(_(" --no-publications do not dump publications\n"));
printf(_(" --no-security-labels do not dump security label assignments\n"));
printf(_(" --no-subscriptions do not dump subscriptions\n"));
- printf(_(" --no-toast-compression do not dump toast compression methods\n"));
+ printf(_(" --no-toast-compression do not dump toast compression methods\n"));
printf(_(" --no-synchronized-snapshots do not use synchronized snapshots in parallel jobs\n"));
printf(_(" --no-tablespaces do not dump tablespace assignments\n"));
printf(_(" --no-unlogged-table-data do not dump unlogged table data\n"));
@@ -3260,6 +3265,45 @@ dumpStdStrings(Archive *AH)
destroyPQExpBuffer(qry);
}
+/*
+ * dumpToastCompression: save the dump-time default TOAST compression in the
+ * archive
+ */
+static void
+dumpToastCompression(Archive *AH)
+{
+ PQExpBuffer qry = createPQExpBuffer();
+ const char *toast_compression;
+ PGresult *res;
+
+ if (AH->dopt->no_toast_compression)
+ return;
+
+ res = ExecuteSqlQueryForSingleRow(AH, "SHOW default_toast_compression"); /* XXX: show ?? */
+ toast_compression = PQgetvalue(res, 0, 0);
+
+ pg_log_info("saving default_toast_compression = %s", toast_compression);
+
+ appendPQExpBufferStr(qry, "SET default_toast_compression = ");
+ appendStringLiteralAH(qry, toast_compression, AH);
+ appendPQExpBufferStr(qry, ";\n");
+
+ ArchiveEntry(AH, nilCatalogId, createDumpId(),
+ ARCHIVE_OPTS(.tag = "TOAST_COMPRESSION",
+ .description = "TOAST_COMPRESSION",
+ .section = SECTION_PRE_DATA,
+ .createStmt = qry->data));
+
+ /*
+ * Also save it in AH->default_toast_compression, in case we're doing plain
+ * text dump
+ */
+ AH->default_toast_compression = pg_strdup(qry->data);
+
+ PQclear(res);
+ destroyPQExpBuffer(qry);
+}
+
/*
* dumpSearchPath: record the active search_path in the archive
*/
--
2.17.0
0009-WIP-pg_dump-use-ALTER-SET-COMPRESSION-rather-than-co.patchtext/x-diff; charset=us-asciiDownload
From f23438aff8df88092c8c36cecec3f8980eb51db2 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 23:43:32 -0500
Subject: [PATCH 9/9] WIP: pg_dump: use ALTER SET COMPRESSION rather than
compression clauses within CREATE TABLE()..
This allows dumps with non-default compression to be restored to a cluster
before v14 compression support, or v14 --without-lz4. Note that allows
pg_restore |psql, but pg_restore -d will still fail, since the entire CREATE
TABLE..ALTER TABLE is execute within a simple query, and a single transaction.
Discussion:
XXX: the list archives are inaccessible...
20210308172915.GF29832@telsasoft.com
4193854.1616199943@sss.pgh.pa.us
20210320000625.GP11765@telsasoft.com
---
src/bin/pg_dump/pg_backup_archiver.c | 24 +++++++++---
src/bin/pg_dump/pg_dump.c | 58 +++++++++++++++-------------
src/bin/pg_dump/t/002_pg_dump.pl | 12 +++---
3 files changed, 56 insertions(+), 38 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index d7fcbc5410..b8a4f5933a 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -2759,11 +2759,22 @@ processSearchPathEntry(ArchiveHandle *AH, TocEntry *te)
static void
processToastCompressionEntry(ArchiveHandle *AH, TocEntry *te)
{
- /*
- * te->defn should contain a command to set default_toast_compression.
- * We just copy it verbatim for use later.
- */
- AH->public.default_toast_compression = pg_strdup(te->defn);
+ /* te->defn should have the form SET default_toast_compression = 'x'; */
+ char *defn = pg_strdup(te->defn);
+ char *ptr1,
+ *ptr2 = NULL;
+
+ ptr1 = strchr(defn, '\'');
+ if (ptr1)
+ ptr2 = strchr(++ptr1, '\'');
+ if (ptr2)
+ {
+ *ptr2 = '\0';
+ AH->public.default_toast_compression = pg_strdup(ptr1);
+ }
+ else
+ fatal("invalid TOAST_COMPRESSION item: %s",
+ te->defn);
}
static void
@@ -3151,7 +3162,8 @@ _doSetFixedOutputState(ArchiveHandle *AH)
/* Select the dump-time default_toast_compression */
if (AH->public.default_toast_compression)
- ahprintf(AH, "%s", AH->public.default_toast_compression);
+ ahprintf(AH, "SET default_toast_compression = '%s';\n",
+ AH->public.default_toast_compression);
/* Make sure function checking is disabled */
ahprintf(AH, "SET check_function_bodies = false;\n");
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index 12d1ebf466..5e4baa6e5e 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -3296,9 +3296,11 @@ dumpToastCompression(Archive *AH)
/*
* Also save it in AH->default_toast_compression, in case we're doing plain
- * text dump
+ * text dump. The compression name is saved, not the create statement, to
+ * allow easily comparing it with each column's compression clause.
+ * XXX: maybe instead should just add a 2nd field to AH for the raw compression.
*/
- AH->default_toast_compression = pg_strdup(qry->data);
+ AH->default_toast_compression = pg_strdup(toast_compression);
PQclear(res);
destroyPQExpBuffer(qry);
@@ -15949,30 +15951,6 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
- /*
- * Attribute compression
- */
- if (!dopt->no_toast_compression)
- {
- char *cmname;
-
- switch (tbinfo->attcompression[j])
- {
- case 'p':
- cmname = "pglz";
- break;
- case 'l':
- cmname = "lz4";
- break;
- default:
- cmname = NULL;
- break;
- }
-
- if (cmname != NULL)
- appendPQExpBuffer(q, " COMPRESSION %s", cmname);
- }
-
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
@@ -16391,6 +16369,34 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression
+ */
+ if (!dopt->no_toast_compression)
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ break;
+ }
+
+ if (cmname != NULL &&
+ strcmp(cmname, fout->default_toast_compression) != 0)
+ appendPQExpBuffer(q, "ALTER %sTABLE ONLY %s ALTER COLUMN %s SET COMPRESSION %s;\n",
+ foreign, qualrelname,
+ fmtId(tbinfo->attnames[j]),
+ cmname);
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index bc91bb12ac..e7885fdd51 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text COMPRESSION\E\D*,\n
- \s+\Qcol3 text COMPRESSION\E\D*,\n
- \s+\Qcol4 text COMPRESSION\E\D*,\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text COMPRESSION\E\D*
+ \n\s+\Qcol2 text\E
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
+ \n\s+\Qcol4 bit(5),\E*,
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text COMPRESSION\E\D*\n
+ \s+\Qcol2 text\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
--
2.17.0
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 8:11 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Mar 19, 2021 at 8:25 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Extrapolating from the way we've dealt with similar issues
in the past, I think the structure of pg_dump's output ought to be:1. SET default_toast_compression = 'source system's value'
in among the existing passel of SETs at the top. Doesn't
matter whether or not that is the compiled-in value.2. No mention of compression in any CREATE TABLE command.
3. For any column having a compression option different from
the default, emit ALTER TABLE SET ... to set that option after
the CREATE TABLE. (You did implement such a SET, I trust.)Actually, *I* didn't implement any of this. But ALTER TABLE sometab
ALTER somecol SET COMPRESSION somealgo works.This sounds like a reasonable approach.
The attached patch implements that.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
0001-Fixup-dump-toast-compression-method.patchtext/x-patch; charset=US-ASCII; name=0001-Fixup-dump-toast-compression-method.patchDownload
From 40b53e24932b1f9203092a7f6972804af5a7a45b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sat, 20 Mar 2021 13:14:39 +0530
Subject: [PATCH] Fixup, dump toast compression method
---
src/bin/pg_dump/pg_backup.h | 1 +
src/bin/pg_dump/pg_backup_archiver.c | 4 +++
src/bin/pg_dump/pg_dump.c | 66 ++++++++++++++++++++++--------------
src/bin/pg_dump/t/002_pg_dump.pl | 12 +++----
4 files changed, 52 insertions(+), 31 deletions(-)
diff --git a/src/bin/pg_dump/pg_backup.h b/src/bin/pg_dump/pg_backup.h
index 0296b9b..02019fc 100644
--- a/src/bin/pg_dump/pg_backup.h
+++ b/src/bin/pg_dump/pg_backup.h
@@ -210,6 +210,7 @@ typedef struct Archive
/* other important stuff */
char *searchpath; /* search_path to set during restore */
char *use_role; /* Issue SET ROLE to this */
+ char *default_toast_compression;
/* error handling */
bool exit_on_error; /* whether to exit on SQL errors... */
diff --git a/src/bin/pg_dump/pg_backup_archiver.c b/src/bin/pg_dump/pg_backup_archiver.c
index 1f82c64..a25f4d1 100644
--- a/src/bin/pg_dump/pg_backup_archiver.c
+++ b/src/bin/pg_dump/pg_backup_archiver.c
@@ -3152,6 +3152,10 @@ _doSetFixedOutputState(ArchiveHandle *AH)
else
ahprintf(AH, "SET row_security = off;\n");
+ /* Select the dump-time default toast compression */
+ if (AH->public.default_toast_compression)
+ ahprintf(AH, "SET default_toast_compression = '%s';\n",
+ AH->public.default_toast_compression);
ahprintf(AH, "\n");
}
diff --git a/src/bin/pg_dump/pg_dump.c b/src/bin/pg_dump/pg_dump.c
index f8bec3f..7bbcaac 100644
--- a/src/bin/pg_dump/pg_dump.c
+++ b/src/bin/pg_dump/pg_dump.c
@@ -1248,6 +1248,21 @@ setup_connection(Archive *AH, const char *dumpencoding,
AH->sync_snapshot_id = get_synchronized_snapshot(AH);
}
+
+ /*
+ * Get default TOAST compression method, but not if the server's too
+ * old to support the feature or if the user doesn't want to dump that
+ * information anyway.
+ */
+ if (AH->remoteVersion >= 140000 && !dopt->no_toast_compression)
+ {
+ PGresult *res;
+
+ res = ExecuteSqlQueryForSingleRow(AH,
+ "SHOW default_toast_compression");
+ AH->default_toast_compression = pg_strdup(PQgetvalue(res, 0, 0));
+ PQclear(res);
+ }
}
/* Set up connection for a parallel worker process */
@@ -15905,31 +15920,6 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
tbinfo->atttypnames[j]);
}
- /*
- * Attribute compression
- */
- if (!dopt->no_toast_compression &&
- tbinfo->attcompression != NULL)
- {
- char *cmname;
-
- switch (tbinfo->attcompression[j])
- {
- case 'p':
- cmname = "pglz";
- break;
- case 'l':
- cmname = "lz4";
- break;
- default:
- cmname = NULL;
- break;
- }
-
- if (cmname != NULL)
- appendPQExpBuffer(q, " COMPRESSION %s", cmname);
- }
-
if (print_default)
{
if (tbinfo->attgenerated[j] == ATTRIBUTE_GENERATED_STORED)
@@ -16348,6 +16338,32 @@ dumpTableSchema(Archive *fout, const TableInfo *tbinfo)
qualrelname,
fmtId(tbinfo->attnames[j]),
tbinfo->attfdwoptions[j]);
+
+ /*
+ * Dump per-column compression options
+ */
+ if (!dopt->no_toast_compression && tbinfo->attcompression != NULL)
+ {
+ char *cmname;
+
+ switch (tbinfo->attcompression[j])
+ {
+ case 'p':
+ cmname = "pglz";
+ break;
+ case 'l':
+ cmname = "lz4";
+ break;
+ default:
+ cmname = NULL;
+ break;
+ }
+
+ if (cmname != NULL &&
+ strcmp(cmname, fout->default_toast_compression) != 0)
+ appendPQExpBuffer(q, "ALTER TABLE %s ALTER COLUMN %s\nSET COMPRESSION %s;\n",
+ qualrelname, fmtId(tbinfo->attnames[j]), cmname);
+ }
}
if (ftoptions)
diff --git a/src/bin/pg_dump/t/002_pg_dump.pl b/src/bin/pg_dump/t/002_pg_dump.pl
index bc91bb1..737e464 100644
--- a/src/bin/pg_dump/t/002_pg_dump.pl
+++ b/src/bin/pg_dump/t/002_pg_dump.pl
@@ -2284,9 +2284,9 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text COMPRESSION\E\D*,\n
- \s+\Qcol3 text COMPRESSION\E\D*,\n
- \s+\Qcol4 text COMPRESSION\E\D*,\n
+ \s+\Qcol2 text,\E\n
+ \s+\Qcol3 text,\E\n
+ \s+\Qcol4 text,\E\n
\s+\QCONSTRAINT test_table_col1_check CHECK ((col1 <= 1000))\E\n
\Q)\E\n
\QWITH (autovacuum_enabled='false', fillfactor='80');\E\n/xm,
@@ -2326,7 +2326,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_second_table (\E
\n\s+\Qcol1 integer,\E
- \n\s+\Qcol2 text COMPRESSION\E\D*
+ \n\s+\Qcol2 text\E
\n\);
/xm,
like =>
@@ -2441,7 +2441,7 @@ my %tests = (
\n\s+\Qcol1 integer,\E
\n\s+\Qcol2 boolean,\E
\n\s+\Qcol3 boolean,\E
- \n\s+\Qcol4 bit(5) COMPRESSION\E\D*,
+ \n\s+\Qcol4 bit(5),\E
\n\s+\Qcol5 double precision\E
\n\);
/xm,
@@ -2459,7 +2459,7 @@ my %tests = (
regexp => qr/^
\QCREATE TABLE dump_test.test_table_identity (\E\n
\s+\Qcol1 integer NOT NULL,\E\n
- \s+\Qcol2 text COMPRESSION\E\D*\n
+ \s+\Qcol2 text\E\n
\);
.*
\QALTER TABLE dump_test.test_table_identity ALTER COLUMN col1 ADD GENERATED ALWAYS AS IDENTITY (\E\n
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 1:22 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sat, Mar 20, 2021 at 8:11 AM Robert Haas <robertmhaas@gmail.com> wrote:
On Fri, Mar 19, 2021 at 8:25 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Extrapolating from the way we've dealt with similar issues
in the past, I think the structure of pg_dump's output ought to be:1. SET default_toast_compression = 'source system's value'
in among the existing passel of SETs at the top. Doesn't
matter whether or not that is the compiled-in value.2. No mention of compression in any CREATE TABLE command.
3. For any column having a compression option different from
the default, emit ALTER TABLE SET ... to set that option after
the CREATE TABLE. (You did implement such a SET, I trust.)Actually, *I* didn't implement any of this. But ALTER TABLE sometab
ALTER somecol SET COMPRESSION somealgo works.This sounds like a reasonable approach.
The attached patch implements that.
After sending this, just saw Justin also included patches for this. I
think the ALTER ..SET COMPRESSION is more or less similar, I just
fetched it from the older version of the patch set. But SET
default_toast_compression are slightly different. I will look into
your version and provide my opinion on which one looks better and we
can commit that and feel free to share your thoughts.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Hi,
I think this bit in brin_tuple.c is wrong:
...
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
Datum cvalue = toast_compress_datum(value,
att->attcompression);
The problem is that this is looking at the index descriptor (i.e. what
types are indexed) instead of the stored type. For BRIN those may be
only loosely related, which is why the code does this a couple lines above:
/* We must look at the stored type, not at the index descriptor. */
TypeCacheEntry *atttype
= brdesc->bd_info[keyno]->oi_typcache[datumno];
For the built-in BRIN opclasses this happens to work, because e.g.
minmax stores two values of the original type. But it may not work for
other out-of-core opclasses, and it certainly doesn't work for the new
BRIN opclasses (bloom and minmax-multi).
Unfortunately, the only thing we have here is the type OID, so I guess
the only option is using GetDefaultToastCompression(). Perhaps we might
include that into BrinOpcInfo too, in the future.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 3:05 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:
Hi,
I think this bit in brin_tuple.c is wrong:
...
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
Datum cvalue = toast_compress_datum(value,
att->attcompression);The problem is that this is looking at the index descriptor (i.e. what
types are indexed) instead of the stored type. For BRIN those may be
only loosely related, which is why the code does this a couple lines above:/* We must look at the stored type, not at the index descriptor. */
TypeCacheEntry *atttype
= brdesc->bd_info[keyno]->oi_typcache[datumno];
Ok, I was not aware of this.
For the built-in BRIN opclasses this happens to work, because e.g.
minmax stores two values of the original type. But it may not work for
other out-of-core opclasses, and it certainly doesn't work for the new
BRIN opclasses (bloom and minmax-multi).
Okay
Unfortunately, the only thing we have here is the type OID, so I guess
the only option is using GetDefaultToastCompression(). Perhaps we might
include that into BrinOpcInfo too, in the future.
Right, I think for now we can use default compression for this case.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On 3/20/21 11:18 AM, Dilip Kumar wrote:
On Sat, Mar 20, 2021 at 3:05 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:Hi,
I think this bit in brin_tuple.c is wrong:
...
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
Datum cvalue = toast_compress_datum(value,
att->attcompression);The problem is that this is looking at the index descriptor (i.e. what
types are indexed) instead of the stored type. For BRIN those may be
only loosely related, which is why the code does this a couple lines above:/* We must look at the stored type, not at the index descriptor. */
TypeCacheEntry *atttype
= brdesc->bd_info[keyno]->oi_typcache[datumno];Ok, I was not aware of this.
Yeah, the BRIN internal structure is not obvious, and the fact that all
the built-in BRIN variants triggers the issue makes it harder to spot.
For the built-in BRIN opclasses this happens to work, because e.g.
minmax stores two values of the original type. But it may not work for
other out-of-core opclasses, and it certainly doesn't work for the new
BRIN opclasses (bloom and minmax-multi).Okay
Unfortunately, the only thing we have here is the type OID, so I guess
the only option is using GetDefaultToastCompression(). Perhaps we might
include that into BrinOpcInfo too, in the future.Right, I think for now we can use default compression for this case.
Good. I wonder if we might have "per type" preferred compression in the
future, which would address this. But for now just using the default
compression seems fine.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 1:14 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
See attached.
I have looked into your patches
- 0001 to 0005 and 0007 look fine to me so maybe you can merge them
all and create a fixup patch. Thanks for fixing this, these were some
silly mistakes I made in my patch.
- 0006 is fine but not sure what is the advantage over what we have today?
- And, 0008 and 0009, I think my
0001-Fixup-dump-toast-compression-method.patch[1]/messages/by-id/CAFiTN-v7EULPqVJ-6J=zH6n0+kO=YFtgpte+FTre=WrwcWBBTA@mail.gmail.com is doing this in a
much simpler way, please have a look and let me know if you think that
has any problems and we need to do the way you are doing here?
[1]: /messages/by-id/CAFiTN-v7EULPqVJ-6J=zH6n0+kO=YFtgpte+FTre=WrwcWBBTA@mail.gmail.com
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On 3/20/21 11:45 AM, Tomas Vondra wrote:
On 3/20/21 11:18 AM, Dilip Kumar wrote:
On Sat, Mar 20, 2021 at 3:05 PM Tomas Vondra
<tomas.vondra@enterprisedb.com> wrote:Hi,
I think this bit in brin_tuple.c is wrong:
...
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
Datum cvalue = toast_compress_datum(value,
att->attcompression);The problem is that this is looking at the index descriptor (i.e. what
types are indexed) instead of the stored type. For BRIN those may be
only loosely related, which is why the code does this a couple lines above:/* We must look at the stored type, not at the index descriptor. */
TypeCacheEntry *atttype
= brdesc->bd_info[keyno]->oi_typcache[datumno];Ok, I was not aware of this.
Yeah, the BRIN internal structure is not obvious, and the fact that all
the built-in BRIN variants triggers the issue makes it harder to spot.For the built-in BRIN opclasses this happens to work, because e.g.
minmax stores two values of the original type. But it may not work for
other out-of-core opclasses, and it certainly doesn't work for the new
BRIN opclasses (bloom and minmax-multi).Okay
Unfortunately, the only thing we have here is the type OID, so I guess
the only option is using GetDefaultToastCompression(). Perhaps we might
include that into BrinOpcInfo too, in the future.Right, I think for now we can use default compression for this case.
Good. I wonder if we might have "per type" preferred compression in the
future, which would address this. But for now just using the default
compression seems fine.
Actually, we can be a bit smarter - when the data types match, we can
use the compression method defined for the attribute. That works fine
for all built-in BRIN opclasses, and it seems quite reasonable - if the
user picked a particular compression method for a column, it's likely
because the data compress better with that method. So why not use that
for the BRIN summary, when possible (even though the BRIN indexes tend
to be tiny).
Attached is a patch doing this. Barring objection I'll push that soon,
so that I can push the BRIN index improvements (bloom etc.).
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Attachments:
0001-Use-valid-compression-method-in-brin_form_tuple.patchtext/x-patch; charset=UTF-8; name=0001-Use-valid-compression-method-in-brin_form_tuple.patchDownload
From 7f3b29dae25b08f28cd84dbbeb069d0c7759fafc Mon Sep 17 00:00:00 2001
From: Tomas Vondra <tomas.vondra@postgresql.org>
Date: Sat, 20 Mar 2021 10:24:00 +0100
Subject: [PATCH] Use valid compression method in brin_form_tuple
When compressing the BRIN summary, we can't simply use the compression
method from the indexed attribute. The summary may use a different data
type, e.g. fixed-length attribute may have varlena summary, leading to
compression failures. For the built-in BRIN opclasses this happens to
work, because the summary uses the same data type as the attribute.
When the data types match, we can inherit use the compression method
specified for the attribute (it's copied into the index descriptor).
Otherwise we don't have much choice and have to use the default one.
Author: Tomas Vondra
Discussion: https://postgr.es/m/e0367f27-392c-321a-7411-a58e1a7e4817%40enterprisedb.com
---
src/backend/access/brin/brin_tuple.c | 14 ++++++++++++--
1 file changed, 12 insertions(+), 2 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 0ab5712c71..279a7e5970 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -213,10 +213,20 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
(atttype->typstorage == TYPSTORAGE_EXTENDED ||
atttype->typstorage == TYPSTORAGE_MAIN))
{
+ Datum cvalue;
+ char compression = GetDefaultToastCompression();
Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc,
keyno);
- Datum cvalue = toast_compress_datum(value,
- att->attcompression);
+
+ /*
+ * If the BRIN summary and indexed attribute use the same data
+ * type, we can the same compression method. Otherwise we have
+ * to use the default method.
+ */
+ if (att->atttypid == atttype->type_id)
+ compression = att->attcompression;
+
+ cvalue = toast_compress_datum(value, compression);
if (DatumGetPointer(cvalue) != NULL)
{
--
2.30.2
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 04:13:47PM +0100, Tomas Vondra wrote:
+++ b/src/backend/access/brin/brin_tuple.c @@ -213,10 +213,20 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple, (atttype->typstorage == TYPSTORAGE_EXTENDED || atttype->typstorage == TYPSTORAGE_MAIN)) { + Datum cvalue; + char compression = GetDefaultToastCompression(); Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc, keyno); - Datum cvalue = toast_compress_datum(value, - att->attcompression); + + /* + * If the BRIN summary and indexed attribute use the same data + * type, we can the same compression method. Otherwise we have
can *use ?
+ * to use the default method. + */ + if (att->atttypid == atttype->type_id) + compression = att->attcompression;
It would be more obvious to me if this said here:
| else: compression = GetDefaultToastCompression
--
Justin
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 04:37:53PM +0530, Dilip Kumar wrote:
- 0006 is fine but not sure what is the advantage over what we have today?
The advantage is that it's dozens of lines shorter, and automatically includes
a HINT.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4. If we use a GUC hook, I think it should be to special case lz4 to say:
"..must be enabled when PG was built".
- And, 0008 and 0009, I think my
0001-Fixup-dump-toast-compression-method.patch[1] is doing this in a
much simpler way, please have a look and let me know if you think that
has any problems and we need to do the way you are doing here?
I tested and saw that your patch doesn't output "SET default_toast_compression"
in non-text dumps (pg_dump -Fc). Also, I think the internal newline should be
removed:
ALTER TABLE public.t ALTER COLUMN b
SET COMPRESSION lz4;
--
Justin
Re: [HACKERS] Custom compression methods
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sat, Mar 20, 2021 at 04:37:53PM +0530, Dilip Kumar wrote:
- And, 0008 and 0009, I think my
0001-Fixup-dump-toast-compression-method.patch[1] is doing this in a
much simpler way, please have a look and let me know if you think that
has any problems and we need to do the way you are doing here?
I tested and saw that your patch doesn't output "SET default_toast_compression"
in non-text dumps (pg_dump -Fc).
Yeah, _doSetFixedOutputState is the wrong place: that runs on the
pg_restore side of the fence, and would not have access to the
necessary info in a separated dump/restore run.
It might be necessary to explicitly pass the state through in a TOC item,
as we do for things like the standard_conforming_strings setting.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 01:36:15PM -0400, Tom Lane wrote:
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sat, Mar 20, 2021 at 04:37:53PM +0530, Dilip Kumar wrote:
- And, 0008 and 0009, I think my
0001-Fixup-dump-toast-compression-method.patch[1] is doing this in a
much simpler way, please have a look and let me know if you think that
has any problems and we need to do the way you are doing here?I tested and saw that your patch doesn't output "SET default_toast_compression"
in non-text dumps (pg_dump -Fc).Yeah, _doSetFixedOutputState is the wrong place: that runs on the
pg_restore side of the fence, and would not have access to the
necessary info in a separated dump/restore run.It might be necessary to explicitly pass the state through in a TOC item,
as we do for things like the standard_conforming_strings setting.
My patches do this in 0008 and 0009 - I'd appreciate if you'd take a look.
0009 edits parts of 0008, and if that's all correct then they should be
squished together.
/messages/by-id/20210320074420.GR11765@telsasoft.com
--
Justin
Re: [HACKERS] Custom compression methods
I wrote:
Yeah, _doSetFixedOutputState is the wrong place: that runs on the
pg_restore side of the fence, and would not have access to the
necessary info in a separated dump/restore run.
It might be necessary to explicitly pass the state through in a TOC item,
as we do for things like the standard_conforming_strings setting.
Ah, now that I read your patch I see that's exactly what you did.
I fixed up some issues in 0008/0009 (mostly cosmetic, except that
you forgot a server version check in dumpToastCompression) and
pushed that, so we can see if it makes crake happy.
regards, tom lane
Re: [HACKERS] Custom compression methods
On 3/20/21 3:03 PM, Tom Lane wrote:
I wrote:
Yeah, _doSetFixedOutputState is the wrong place: that runs on the
pg_restore side of the fence, and would not have access to the
necessary info in a separated dump/restore run.
It might be necessary to explicitly pass the state through in a TOC item,
as we do for things like the standard_conforming_strings setting.Ah, now that I read your patch I see that's exactly what you did.
I fixed up some issues in 0008/0009 (mostly cosmetic, except that
you forgot a server version check in dumpToastCompression) and
pushed that, so we can see if it makes crake happy.
It's still produced a significant amount more difference between the
dumps. For now I've increased the fuzz factor a bit like this:
diff --git a/PGBuild/Modules/TestUpgradeXversion.pm
b/PGBuild/Modules/TestUpgradeXversion.pm
index 1d1d313..567d7cb 100644
--- a/PGBuild/Modules/TestUpgradeXversion.pm
+++ b/PGBuild/Modules/TestUpgradeXversion.pm
@@ -621,7 +621,7 @@ sub test_upgrade ## no critic
(Subroutines::ProhibitManyArgs)
# generally from reordering of larg object output.
# If not we heuristically allow up to 2000 lines of diffs
- if ( ($oversion ne $this_branch && $difflines < 2000)
+ if ( ($oversion ne $this_branch && $difflines < 2700)
|| ($oversion eq $this_branch) && $difflines < 50)
{
return 1;I'll try to come up with something better. Maybe just ignore lines like
SET default_toast_compression = 'pglz';
when taking the diff.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
I wrote:
I fixed up some issues in 0008/0009 (mostly cosmetic, except that
you forgot a server version check in dumpToastCompression) and
pushed that, so we can see if it makes crake happy.
crake was still unhappy with that:
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=crake&dt=2021-03-20%2019%3A03%3A56
but I see it just went green ... did you do something to adjust
the expected output?
regards, tom lane
Re: [HACKERS] Custom compression methods
Andrew Dunstan <andrew@dunslane.net> writes:
On 3/20/21 3:03 PM, Tom Lane wrote:
I fixed up some issues in 0008/0009 (mostly cosmetic, except that
you forgot a server version check in dumpToastCompression) and
pushed that, so we can see if it makes crake happy.
It's still produced a significant amount more difference between the
dumps. For now I've increased the fuzz factor a bit like this:
Ah, our emails crossed.
I'll try to come up with something better. Maybe just ignore lines like
SET default_toast_compression = 'pglz';
when taking the diff.
I noticed that there were a fair number of other diffs besides those.
Seems like we need some better comparison technology, really, but I'm
not certain what.
regards, tom lane
Re: [HACKERS] Custom compression methods (./configure)
On Fri, Mar 19, 2021 at 05:35:58PM -0300, Alvaro Herrera wrote:
Hmm, if I use configure --with-lz4, I get this:
checking whether to build with LZ4 support... yes
checking for liblz4... no
configure: error: Package requirements (liblz4) were not met:No package 'liblz4' found
...
See the pkg-config man page for more details.
running CONFIG_SHELL=/bin/bash /bin/bash /pgsql/source/master/configure --enable-debug --enable-depend --enable-cassert --enable-nls --cache-file=/home/alvherre/run/pgconfig.master.cache --enable-thread-safety --with-python --with-perl --with-tcl --with-openssl --with-libxml --enable-tap-tests --with-tclconfig=/usr/lib/tcl8.6 PYTHON=/usr/bin/python3 --with-llvm --prefix=/pgsql/install/master --with-pgport=55432 --no-create --no-recursion
...I find this behavior confusing; I'd rather have configure error out if
it can't find the package support I requested, than continuing with a
set of configure options different from what I gave.
That's clearly wrong, but that's not the behavior I see:
|$ ./configure --with-lz4 ; echo $?
|...
|checking for liblz4... no
|configure: error: Package requirements (liblz4) were not met:
|
|No package 'liblz4' found
|
|Consider adjusting the PKG_CONFIG_PATH environment variable if you
|installed software in a non-standard prefix.
|
|Alternatively, you may set the environment variables LZ4_CFLAGS
|and LZ4_LIBS to avoid the need to call pkg-config.
|See the pkg-config man page for more details.
|1
I can't reproduce the behavior - is it because of your --cache-file or
something ?
--
Justin
Re: [HACKERS] Custom compression methods (./configure)
On 2021-Mar-20, Justin Pryzby wrote:
On Fri, Mar 19, 2021 at 05:35:58PM -0300, Alvaro Herrera wrote:
Hmm, if I use configure --with-lz4, I get this:
checking whether to build with LZ4 support... yes
checking for liblz4... no
configure: error: Package requirements (liblz4) were not met:No package 'liblz4' found
...
See the pkg-config man page for more details.
running CONFIG_SHELL=/bin/bash /bin/bash /pgsql/source/master/configure --enable-debug --enable-depend --enable-cassert --enable-nls --cache-file=/home/alvherre/run/pgconfig.master.cache --enable-thread-safety --with-python --with-perl --with-tcl --with-openssl --with-libxml --enable-tap-tests --with-tclconfig=/usr/lib/tcl8.6 PYTHON=/usr/bin/python3 --with-llvm --prefix=/pgsql/install/master --with-pgport=55432 --no-create --no-recursion
I can't reproduce the behavior - is it because of your --cache-file or
something ?
Argh, yeah, you're right -- my custom scripting was confusing the issue,
by rerunning configure automatically with the options previously in the
cache file. I had the equivalent of "configure ; make" so when
configure failed, the make step re-ran configure using the options in
the cache file, which did not have --with-lz4.
--
�lvaro Herrera Valdivia, Chile
Re: [HACKERS] Custom compression methods (./configure)
Justin Pryzby <pryzby@telsasoft.com> writes:
On Fri, Mar 19, 2021 at 05:35:58PM -0300, Alvaro Herrera wrote:
I find this behavior confusing; I'd rather have configure error out if
it can't find the package support I requested, than continuing with a
set of configure options different from what I gave.
That's clearly wrong, but that's not the behavior I see:
Yeah, it errors out as-expected for me too, on a couple of different
machines (see sifaka's latest run for documentation).
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Fri, Mar 19, 2021 at 02:07:31PM -0400, Robert Haas wrote:
On Fri, Mar 19, 2021 at 1:44 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Working with one of Andrey's patches on another thread, he reported offlist
getting this message, resolved by this patch. Do you see this warning during
./configure ? The latest CI is of a single patch without the LZ4 stuff, so I
can't check its log.configure: WARNING: lz4.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: lz4.h: proceeding with the compiler's resultNo, I don't see this. I wonder whether this could possibly be an
installation issue on Andrey's machine? If not, it must be
version-dependent or installation-dependent in some way.
Andrey, can you check if latest HEAD (bbe0a81db) has these ./configure warnings ?
If so, can you check if your environment is clean, specifically lz4.h - if
you've installed LZ4 from source, I have to imagine that's relevant.
Most users won't do that, but it should be a supported configuration, too.
--
Justin
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Justin Pryzby <pryzby@telsasoft.com> writes:
On Fri, Mar 19, 2021 at 02:07:31PM -0400, Robert Haas wrote:
On Fri, Mar 19, 2021 at 1:44 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
configure: WARNING: lz4.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: lz4.h: proceeding with the compiler's result
No, I don't see this. I wonder whether this could possibly be an
installation issue on Andrey's machine? If not, it must be
version-dependent or installation-dependent in some way.
Andrey, can you check if latest HEAD (bbe0a81db) has these ./configure warnings ?
FWIW, I also saw that, when building HEAD against MacPorts' version
of liblz4 on an M1 Mac. config.log has
configure:13536: checking lz4.h usability
configure:13536: ccache clang -c -I/opt/local/include -Wall -Wmissing-prototype\
s -Wpointer-arith -Wdeclaration-after-statement -Werror=vla -Wendif-labels -Wmi\
ssing-format-attribute -Wformat-security -fno-strict-aliasing -fwrapv -Wno-unus\
ed-command-line-argument -g -O2 -isysroot /Applications/Xcode.app/Contents/Deve\
loper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk conftest.c >&5
configure:13536: $? = 0
configure:13536: result: yes
configure:13536: checking lz4.h presence
configure:13536: ccache clang -E -isysroot /Applications/Xcode.app/Contents/Dev\
eloper/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk conftest.c
conftest.c:67:10: fatal error: 'lz4.h' file not found
#include <lz4.h>
^~~~~~~
1 error generated.
configure:13536: $? = 1
Digging around, it looks like the "-I/opt/local/include" bit came
from LZ4_CFLAGS, which we then stuck into CFLAGS, but it needed
to be put in CPPFLAGS in order to make this test work.
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Sat, Mar 20, 2021 at 05:37:07PM -0400, Tom Lane wrote:
Justin Pryzby <pryzby@telsasoft.com> writes:
On Fri, Mar 19, 2021 at 02:07:31PM -0400, Robert Haas wrote:
On Fri, Mar 19, 2021 at 1:44 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
configure: WARNING: lz4.h: accepted by the compiler, rejected by the preprocessor!
configure: WARNING: lz4.h: proceeding with the compiler's resultNo, I don't see this. I wonder whether this could possibly be an
installation issue on Andrey's machine? If not, it must be
version-dependent or installation-dependent in some way.Andrey, can you check if latest HEAD (bbe0a81db) has these ./configure warnings ?
FWIW, I also saw that, when building HEAD against MacPorts' version
of liblz4 on an M1 Mac. config.log has
...
Digging around, it looks like the "-I/opt/local/include" bit came
from LZ4_CFLAGS, which we then stuck into CFLAGS, but it needed
to be put in CPPFLAGS in order to make this test work.
If it's the same as the issue Andrey reported, then it causes a ./configure
WARNING, which is resolved by the ac_save hack, which I copied from ICU.
I'll shortly send a patchset including my tentative fix for that.
The configure.ac bits are also on this other thread:
/messages/by-id/20210315180918.GW29463@telsasoft.com
0005-re-add-wal_compression_method-lz4.patch
+if test "$with_lz4" = yes; then
+ ac_save_CPPFLAGS=$CPPFLAGS
+ CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS"
+
+ # Verify we have LZ4's header files
+ AC_CHECK_HEADERS(lz4/lz4.h, [],
+ [AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+
+ CPPFLAGS=$ac_save_CPPFLAGS
+fi--
Justin
Re: [HACKERS] Custom compression methods
Rebased on HEAD.
0005 forgot to update compression_1.out.
Included changes to ./configure.ac and some other patches, but not Tomas's,
since it'll make CFBOT get mad as soon as that's pushed.
--
Justin
Attachments:
v2-0007-Commentary-about-slicing.patchtext/x-diff; charset=iso-8859-1Download
From 533a7281906e42db7329c6e44ca236094ab60ab9 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 20 Mar 2021 16:13:44 -0500
Subject: [PATCH v2 07/10] Commentary about slicing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
My language, per gripe from Ãlvaro Herrera:
| I updated the coverage script to use --with-lz4; results are updated.
| While eyeballing the results I noticed this bit in
| lz4_decompress_datum_slice():
|
| + /* slice decompression not supported prior to 1.8.3 */
| + if (LZ4_versionNumber() < 10803)
| + return lz4_decompress_datum(value);
|
| which I read as returning the complete decompressed datum if slice
| decompression is not supported. I thought that was a bug, but looking
| at the caller I realize that this isn't really a problem, since it's
| detoast_attr_slice's responsibility to slice the result further -- no
| bug, it's just wasteful. I suggest to add comments to this effect,
| perhaps as the attached (feel free to reword, I think mine is awkward.)
---
src/backend/access/common/detoast.c | 3 +++
src/backend/access/common/toast_compression.c | 3 +++
2 files changed, 6 insertions(+)
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index 2fef40c2e9..316f5715e1 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -497,6 +497,9 @@ toast_decompress_datum(struct varlena *attr)
* Decompress the front of a compressed version of a varlena datum.
* offset handling happens in detoast_attr_slice.
* Here we just decompress a slice from the front.
+ *
+ * If slice decompression is not supported, the full datum is decompressed, and
+ * then sliced by detoast_attr_slice.
*/
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index a6f8b79a9e..dd31a54b77 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -203,6 +203,9 @@ lz4_decompress_datum(const struct varlena *value)
/*
* Decompress part of a varlena that was compressed using LZ4.
+ *
+ * If slice decompression is not supported, the full datum is decompressed, and
+ * then sliced by detoast_attr_slice.
*/
struct varlena *
lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
--
2.17.0
v2-0001-Add-docs-for-default_toast_compression.patchtext/x-diff; charset=us-asciiDownload
From bf1336d284792b29e30b42af2ec820bb0c256916 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 19:12:53 -0500
Subject: [PATCH v2 01/10] Add docs for default_toast_compression..
bbe0a81db69bd10bd166907c3701492a29aca294
---
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index ee4925d6d9..5cb851a5eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8151,6 +8151,29 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
+ <term><varname>default_toast_compression</varname> (<type>string</type>)
+ <indexterm>
+ <primary><varname>default_toast_compression</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the default compression method to use
+ when compressing data in <acronym>TOAST</acronym> tables.
+ It applies only to variable-width data types.
+ It may be overriden by compression clauses in the
+ <command>CREATE</command> command, or changed after the relation is
+ created by <command>ALTER TABLE ... SET COMPRESSION</command>.
+
+ The supported compression methods are <literal>pglz</literal> and
+ (if configured at the time <productname>PostgreSQL</productname> was
+ built) <literal>lz4</literal>.
+ The default is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-temp-tablespaces" xreflabel="temp_tablespaces">
<term><varname>temp_tablespaces</varname> (<type>string</type>)
<indexterm>
--
2.17.0
v2-0002-doc-pg_dump-no-toast-compression.patchtext/x-diff; charset=us-asciiDownload
From 3184c87f129ec935fa46773728869c7c6af88025 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 21:52:28 -0500
Subject: [PATCH v2 02/10] doc: pg_dump --no-toast-compression
---
doc/src/sgml/ref/pg_dump.sgml | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index bcbb7a25fb..4a521186fb 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -931,6 +931,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-toast-compression</option></term>
+ <listitem>
+ <para>
+ Do not output commands to set <acronym>TOAST</acronym>compression
+ methods.
+ With this option, all objects will be created using whichever
+ compression method is the default during restore.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-unlogged-table-data</option></term>
<listitem>
--
2.17.0
v2-0003-Compression-method-is-an-char-not-an-OID.patchtext/x-diff; charset=us-asciiDownload
From 2212020a50478c66a830f15b23c3410e4d7bb501 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:23:40 -0500
Subject: [PATCH v2 03/10] Compression method is an char not an OID
---
src/backend/commands/tablecmds.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 9b2800bf5e..8e756e59d5 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -7847,6 +7847,7 @@ SetIndexStorageProperties(Relation rel, Relation attrelation,
index_close(indrel, lockmode);
}
}
+
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -15070,7 +15071,7 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
char *compression;
char typstorage;
- Oid cmoid;
+ char cmethod;
Datum values[Natts_pg_attribute];
bool nulls[Natts_pg_attribute];
bool replace[Natts_pg_attribute];
@@ -15111,9 +15112,9 @@ ATExecSetCompression(AlteredTableInfo *tab,
memset(replace, false, sizeof(replace));
/* get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmethod = GetAttributeCompression(atttableform, compression);
- atttableform->attcompression = cmoid;
+ atttableform->attcompression = cmethod;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -15123,7 +15124,7 @@ ATExecSetCompression(AlteredTableInfo *tab,
ReleaseSysCache(tuple);
/* apply changes to the index column as well */
- SetIndexStorageProperties(rel, attrel, attnum, cmoid, '\0', lockmode);
+ SetIndexStorageProperties(rel, attrel, attnum, cmethod, '\0', lockmode);
table_close(attrel, RowExclusiveLock);
/* make changes visible */
--
2.17.0
v2-0004-Remove-duplicative-macro.patchtext/x-diff; charset=us-asciiDownload
From d73643d7569ef74b9008d232f8c0ea33cb96ff3c Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:07:31 -0500
Subject: [PATCH v2 04/10] Remove duplicative macro
---
src/include/access/toast_compression.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 514df0bed1..09af6e7810 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -50,8 +50,6 @@ typedef enum ToastCompressionId
errdetail("This functionality requires the server to be built with lz4 support."), \
errhint("You need to rebuild PostgreSQL using --with-lz4.")))
-#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
-
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
--
2.17.0
v2-0005-Error-on-invalid-compression-in-CREATE-and-ALTER.patchtext/x-diff; charset=us-asciiDownload
From 79e6dd441a0624f611ee1f7ee037285feccda426 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:19:55 -0500
Subject: [PATCH v2 05/10] Error on invalid compression in CREATE and ALTER
---
src/backend/commands/tablecmds.c | 10 +++++++---
src/test/regress/expected/compression.out | 6 ++++++
src/test/regress/expected/compression_1.out | 6 ++++++
src/test/regress/sql/compression.sql | 5 +++++
4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 8e756e59d5..7efe27ac6c 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17865,9 +17865,13 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- cmethod = GetDefaultToastCompression();
- else
- cmethod = CompressionNameToMethod(compression);
+ return GetDefaultToastCompression();
+
+ cmethod = CompressionNameToMethod(compression);
+ if (!CompressionMethodIsValid(cmethod))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("invalid compression method \"%s\"", compression)));
return cmethod;
}
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3de2886de0..501e0ead14 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -344,4 +344,10 @@ SELECT length(f1) FROM cmmove3;
10040
(2 rows)
+CREATE TABLE t (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+CREATE TABLE t (a text);
+ALTER TABLE t ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+DROP TABLE t;
\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 40aad81fa1..2a704e9b14 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -337,4 +337,10 @@ SELECT length(f1) FROM cmmove3;
10000
(1 row)
+CREATE TABLE t (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+CREATE TABLE t (a text);
+ALTER TABLE t ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+DROP TABLE t;
\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index d97e26b6ee..afddd5a134 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -133,4 +133,9 @@ SELECT length(f1) FROM cmmove1;
SELECT length(f1) FROM cmmove2;
SELECT length(f1) FROM cmmove3;
+CREATE TABLE t (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+CREATE TABLE t (a text);
+ALTER TABLE t ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+DROP TABLE t;
+
\set HIDE_TOAST_COMPRESSION true
--
2.17.0
v2-0006-.-configure-Avoid-warnings-on-Mac-with-lz4.patchtext/x-diff; charset=us-asciiDownload
From 2382b3fe3e609e99ee1742b872de5b4ae61b2ecc Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 20 Mar 2021 17:09:36 -0500
Subject: [PATCH v2 06/10] ./configure: Avoid warnings on Mac --with-lz4
Reported by Andrey Borodin and Tom Lane.
>>> configure: WARNING: lz4.h: accepted by the compiler, rejected by the preprocessor!
>>> configure: WARNING: lz4.h: proceeding with the compiler's result
Possibly we could instead just *remove* the AC_CHECK_HEADERS?
---
configure | 6 ++++++
configure.ac | 6 ++++++
2 files changed, 12 insertions(+)
diff --git a/configure b/configure
index 6d34243dca..fa2b7692ca 100755
--- a/configure
+++ b/configure
@@ -13522,6 +13522,10 @@ fi
fi
if test "$with_lz4" = yes; then
+ ac_save_CPPFLAGS=$CPPFLAGS
+ CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS"
+
+ # Verify we have LZ4's header files
for ac_header in lz4/lz4.h
do :
ac_fn_c_check_header_mongrel "$LINENO" "lz4/lz4.h" "ac_cv_header_lz4_lz4_h" "$ac_includes_default"
@@ -13549,6 +13553,8 @@ fi
done
+
+ CPPFLAGS=$ac_save_CPPFLAGS
fi
if test "$with_gssapi" = yes ; then
diff --git a/configure.ac b/configure.ac
index e54e2fb632..bd92643a3b 100644
--- a/configure.ac
+++ b/configure.ac
@@ -1426,8 +1426,14 @@ Use --without-zlib to disable zlib support.])])
fi
if test "$with_lz4" = yes; then
+ ac_save_CPPFLAGS=$CPPFLAGS
+ CPPFLAGS="$LZ4_CFLAGS $CPPFLAGS"
+
+ # Verify we have LZ4's header files
AC_CHECK_HEADERS(lz4/lz4.h, [],
[AC_CHECK_HEADERS(lz4.h, [], [AC_MSG_ERROR([lz4.h header file is required for LZ4])])])
+
+ CPPFLAGS=$ac_save_CPPFLAGS
fi
if test "$with_gssapi" = yes ; then
--
2.17.0
v2-0008-specially-handle-SET-default_toast_compression-lz.patchtext/x-diff; charset=us-asciiDownload
From adc6eb10f79d1edf3df0d241557b7d23106b7cc6 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 20 Mar 2021 17:25:46 -0500
Subject: [PATCH v2 08/10] specially handle SET default_toast_compression=lz4
---
src/backend/access/common/toast_compression.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index dd31a54b77..1617c382c2 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -304,6 +304,11 @@ check_default_toast_compression(char **newval, void **extra, GucSource source)
errmsg("compression method \"%s\" does not exist",
*newval)));
}
+ else if (strcmp(*newval, "lz4") == 0)
+ {
+ GUC_check_errdetail("This functionality requires the server to be built with lz4 support.");
+ GUC_check_errhint("You need to rebuild PostgreSQL using --with-lz4.");
+ }
else
{
GUC_check_errdetail("Compression method \"%s\" does not exist.",
--
2.17.0
v2-0009-Alternately-WIP-Change-default_toast_compression-.patchtext/x-diff; charset=us-asciiDownload
From 232987c6b79de7cf635664c28fa0062498485146 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 19:39:10 -0500
Subject: [PATCH v2 09/10] Alternately: WIP: Change default_toast_compression
GUC to an enum..
..since the final version of the TOAST patch doesn't require catalog access
---
src/backend/access/common/toast_compression.c | 54 +------------------
src/backend/utils/misc/guc.c | 30 +++++++----
src/include/access/toast_compression.h | 10 ++--
src/test/regress/expected/compression.out | 4 +-
4 files changed, 28 insertions(+), 70 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 1617c382c2..9d9ffc6f97 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -23,8 +23,8 @@
#include "fmgr.h"
#include "utils/builtins.h"
-/* Compile-time default */
-char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* GUC */
+int default_toast_compression = TOAST_PGLZ_COMPRESSION_ID;
/*
* Compress a varlena using PGLZ.
@@ -269,53 +269,3 @@ toast_get_compression_id(struct varlena *attr)
return cmid;
}
-
-/*
- * Validate a new value for the default_toast_compression GUC.
- */
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
-{
- if (**newval == '\0')
- {
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
- }
-
- if (strlen(*newval) >= NAMEDATALEN)
- {
- GUC_check_errdetail("%s is too long (maximum %d characters).",
- "default_toast_compression", NAMEDATALEN - 1);
- return false;
- }
-
- if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
- {
- /*
- * When source == PGC_S_TEST, don't throw a hard error for a
- * nonexistent compression method, only a NOTICE. See comments in
- * guc.h.
- */
- if (source == PGC_S_TEST)
- {
- ereport(NOTICE,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("compression method \"%s\" does not exist",
- *newval)));
- }
- else if (strcmp(*newval, "lz4") == 0)
- {
- GUC_check_errdetail("This functionality requires the server to be built with lz4 support.");
- GUC_check_errhint("You need to rebuild PostgreSQL using --with-lz4.");
- }
- else
- {
- GUC_check_errdetail("Compression method \"%s\" does not exist.",
- *newval);
- return false;
- }
- }
-
- return true;
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4272cf4821..eb1b343c09 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -510,6 +510,14 @@ static struct config_enum_entry shared_memory_options[] = {
{NULL, 0, false}
};
+static struct config_enum_entry default_toast_compression_options[] = {
+ {"pglz", TOAST_PGLZ_COMPRESSION_ID, false},
+#ifdef USE_LZ4
+ {"lz4", TOAST_LZ4_COMPRESSION_ID, false},
+#endif
+ {NULL, 0, false}
+};
+
/*
* Options for enum values stored in other modules
*/
@@ -3953,17 +3961,6 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
- {
- {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default compression for new columns."),
- NULL,
- GUC_IS_NAME
- },
- &default_toast_compression,
- DEFAULT_TOAST_COMPRESSION,
- check_default_toast_compression, NULL, NULL
- },
-
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
@@ -4605,6 +4602,17 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ 0, /* pglz */
+ default_toast_compression_options, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 09af6e7810..88a46e264d 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -16,10 +16,7 @@
#include "utils/guc.h"
/* GUCs */
-extern char *default_toast_compression;
-
-/* default compression method if not specified. */
-#define DEFAULT_TOAST_COMPRESSION "pglz"
+extern int default_toast_compression;
/*
* Built-in compression method-id. The toast compression header will store
@@ -100,7 +97,10 @@ CompressionNameToMethod(char *compression)
static inline char
GetDefaultToastCompression(void)
{
- return CompressionNameToMethod(default_toast_compression);
+ /* See also guc.c */
+ char *const enumtoname[] = {"pglz", "lz4"};
+ Assert(default_toast_compression < lengthof(enumtoname));
+ return CompressionNameToMethod(enumtoname[default_toast_compression]);
}
/* pglz compression/decompression routines */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 501e0ead14..5cc76cdcd6 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -232,10 +232,10 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
--
2.17.0
Re: [HACKERS] Custom compression methods
On 3/20/21 4:21 PM, Justin Pryzby wrote:
On Sat, Mar 20, 2021 at 04:13:47PM +0100, Tomas Vondra wrote:
+++ b/src/backend/access/brin/brin_tuple.c @@ -213,10 +213,20 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple, (atttype->typstorage == TYPSTORAGE_EXTENDED || atttype->typstorage == TYPSTORAGE_MAIN)) { + Datum cvalue; + char compression = GetDefaultToastCompression(); Form_pg_attribute att = TupleDescAttr(brdesc->bd_tupdesc, keyno); - Datum cvalue = toast_compress_datum(value, - att->attcompression); + + /* + * If the BRIN summary and indexed attribute use the same data + * type, we can the same compression method. Otherwise we havecan *use ?
+ * to use the default method. + */ + if (att->atttypid == atttype->type_id) + compression = att->attcompression;It would be more obvious to me if this said here:
| else: compression = GetDefaultToastCompression
Thanks. I've pushed a patch tweaked per your feedback.
regards
--
Tomas Vondra
EnterpriseDB: http://www.enterprisedb.com
The Enterprise PostgreSQL Company
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sat, Mar 20, 2021 at 05:37:07PM -0400, Tom Lane wrote:
Digging around, it looks like the "-I/opt/local/include" bit came
from LZ4_CFLAGS, which we then stuck into CFLAGS, but it needed
to be put in CPPFLAGS in order to make this test work.
If it's the same as the issue Andrey reported, then it causes a ./configure
WARNING, which is resolved by the ac_save hack, which I copied from ICU.
I think probably what we need to do, rather than shove the pkg-config
results willy-nilly into our flags, is to disassemble them like we do
with the same results for xml2. If you ask me, the way we are handling
ICU flags is a poor precedent that is going to blow up at some point;
the only reason it hasn't is that people aren't building --with-icu that
much yet.
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
BTW, I tried doing "make installcheck" after having adjusted
default_toast_compression to be "lz4". The compression test
itself fails because it's expecting the other setting; that
ought to be made more robust. Also, I see some diffs in the
indirect_toast test, which seems perhaps worthy of investigation.
(The diffs look to be just row ordering, but why?)
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Sun, Mar 21, 2021 at 7:03 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
BTW, I tried doing "make installcheck" after having adjusted
default_toast_compression to be "lz4". The compression test
itself fails because it's expecting the other setting; that
ought to be made more robust.
Yeah, we need to set the default_toast_compression in the beginning of
the test as attached.
Also, I see some diffs in the
indirect_toast test, which seems perhaps worthy of investigation.
(The diffs look to be just row ordering, but why?)
I will look into this.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
0001-fix-compression-test.patchapplication/octet-stream; name=0001-fix-compression-test.patchDownload
From d65f952287b443e95e631253430340e3f9cb5b2b Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sun, 21 Mar 2021 09:08:47 +0530
Subject: [PATCH] fix compression test
---
src/test/regress/expected/compression.out | 2 ++
src/test/regress/sql/compression.sql | 3 +++
2 files changed, 5 insertions(+)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3de2886..15af027 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -1,4 +1,5 @@
\set HIDE_TOAST_COMPRESSION false
+SET default_toast_compression = 'pglz';
-- test creating table with compression method
CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
@@ -245,6 +246,7 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+SET default_toast_compression = 'pglz';
-- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
INSERT INTO cmdata VALUES (repeat('123456789', 4004));
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index d97e26b..1280fa4 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -1,5 +1,7 @@
\set HIDE_TOAST_COMPRESSION false
+SET default_toast_compression = 'pglz';
+
-- test creating table with compression method
CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
@@ -100,6 +102,7 @@ SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+SET default_toast_compression = 'pglz';
-- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
--
1.8.3.1
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Sun, Mar 21, 2021 at 9:10 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Sun, Mar 21, 2021 at 7:03 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
BTW, I tried doing "make installcheck" after having adjusted
default_toast_compression to be "lz4". The compression test
itself fails because it's expecting the other setting; that
ought to be made more robust.Yeah, we need to set the default_toast_compression in the beginning of
the test as attached.
In the last patch, I did not adjust the compression_1.out so fixed
that in the attached patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v1-0001-fix-compression-test.patchapplication/octet-stream; name=v1-0001-fix-compression-test.patchDownload
From 46f87e52c1b1de903977480838d346e37d1be567 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Sun, 21 Mar 2021 09:08:47 +0530
Subject: [PATCH v1] fix compression test
---
src/test/regress/expected/compression.out | 2 ++
src/test/regress/expected/compression_1.out | 2 ++
src/test/regress/sql/compression.sql | 3 +++
3 files changed, 7 insertions(+)
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 3de2886..15af027 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -1,4 +1,5 @@
\set HIDE_TOAST_COMPRESSION false
+SET default_toast_compression = 'pglz';
-- test creating table with compression method
CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
@@ -245,6 +246,7 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | lz4 | |
+SET default_toast_compression = 'pglz';
-- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
INSERT INTO cmdata VALUES (repeat('123456789', 4004));
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 40aad81..8842997 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -1,4 +1,5 @@
\set HIDE_TOAST_COMPRESSION false
+SET default_toast_compression = 'pglz';
-- test creating table with compression method
CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
@@ -241,6 +242,7 @@ CREATE TABLE cmdata2 (f1 text);
--------+------+-----------+----------+---------+----------+-------------+--------------+-------------
f1 | text | | | | extended | pglz | |
+SET default_toast_compression = 'pglz';
-- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
ERROR: unsupported LZ4 compression method
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index d97e26b..1280fa4 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -1,5 +1,7 @@
\set HIDE_TOAST_COMPRESSION false
+SET default_toast_compression = 'pglz';
+
-- test creating table with compression method
CREATE TABLE cmdata(f1 text COMPRESSION pglz);
CREATE INDEX idx ON cmdata(f1);
@@ -100,6 +102,7 @@ SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
+SET default_toast_compression = 'pglz';
-- test alter compression method
ALTER TABLE cmdata ALTER COLUMN f1 SET COMPRESSION lz4;
--
1.8.3.1
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Sun, Mar 21, 2021 at 7:03 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Also, I see some diffs in the
indirect_toast test, which seems perhaps worthy of investigation.
(The diffs look to be just row ordering, but why?)
I have investigated that, actually in the below insert, after
compression the data size of (repeat('1234567890',50000)) is 1980
bytes with the lz4 whereas with pglz it is 5737 bytes. So with lz4,
the compressed data are stored inline whereas with pglz those are
getting externalized. Due to this for one of the update statements
followed by an insert, there was no space on the first page as data
are stored inline so the new tuple is stored on the next page and that
is what affecting the order. I hope this makes sense.
INSERT INTO indtoasttest(descr, f1, f2) VALUES('one-toasted,one-null',
NULL, repeat('1234567890',50000));
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Dilip Kumar <dilipbalaut@gmail.com> writes:
Yeah, we need to set the default_toast_compression in the beginning of
the test as attached.In the last patch, I did not adjust the compression_1.out so fixed
that in the attached patch.
Pushed that; however, while testing that it works as expected,
I saw a new and far more concerning regression diff:
diff -U3 /home/postgres/pgsql/src/test/regress/expected/strings.out /home/postgres/pgsql/src/test/regress/results/strings.out
--- /home/postgres/pgsql/src/test/regress/expected/strings.out 2021-02-18 10:34:58.190304138 -0500
+++ /home/postgres/pgsql/src/test/regress/results/strings.out 2021-03-21 16:27:22.029402834 -0400
@@ -1443,10 +1443,10 @@
-- If start plus length is > string length, the result is truncated to
-- string length
SELECT substr(f1, 99995, 10) from toasttest;
- substr
---------
- 567890
- 567890
+ substr
+------------------------
+ 567890\x7F\x7F\x7F\x7F
+ 567890\x7F\x7F\x7F\x7F
567890
567890
(4 rows)
@@ -1520,10 +1520,10 @@
-- If start plus length is > string length, the result is truncated to
-- string length
SELECT substr(f1, 99995, 10) from toasttest;
- substr
---------
- 567890
- 567890
+ substr
+------------------------
+ 567890\177\177\177\177
+ 567890\177\177\177\177
567890
567890
(4 rows)This seems somewhat repeatable (three identical failures in three
attempts). Not sure why I did not see it yesterday; but anyway,
there is something wrong with partial detoasting for LZ4.
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote:
This seems somewhat repeatable (three identical failures in three
attempts). Not sure why I did not see it yesterday; but anyway,
there is something wrong with partial detoasting for LZ4.
With what version of LZ4 ?
--
Justin
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote:
This seems somewhat repeatable (three identical failures in three
attempts). Not sure why I did not see it yesterday; but anyway,
there is something wrong with partial detoasting for LZ4.
With what version of LZ4 ?
RHEL8's, which is
lz4-1.8.3-2.el8.x86_64
regards, tom lane
Re: [HACKERS] Custom compression methods
Justin Pryzby <pryzby@telsasoft.com> writes:
Rebased on HEAD.
0005 forgot to update compression_1.out.
Included changes to ./configure.ac and some other patches, but not Tomas's,
since it'll make CFBOT get mad as soon as that's pushed.
I pushed a version of the configure fixes that passes my own sanity
checks, and removes the configure warning with MacPorts. That
obsoletes your 0006. Of the rest, I prefer the 0009 approach
(make the GUC an enum) to 0008, and the others seem sane but I haven't
studied the code, so I'll leave it to Robert to handle them.
regards, tom lane
Re: [HACKERS] Custom compression methods
... btw, now that I look at this, why are we expending a configure
probe for <lz4/lz4.h> ? If we need to cater for that spelling of
the header name, the C code proper is not ready for it.
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
I wrote:
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote:
This seems somewhat repeatable (three identical failures in three
attempts). Not sure why I did not see it yesterday; but anyway,
there is something wrong with partial detoasting for LZ4.
With what version of LZ4 ?
RHEL8's, which is
lz4-1.8.3-2.el8.x86_64
I hate to be the bearer of bad news, but this suggests that
LZ4_decompress_safe_partial is seriously broken in 1.9.2
as well:
https://github.com/lz4/lz4/issues/783
Maybe we cannot rely on that function for a few more years yet.
Also, I don't really understand why this code:
/* slice decompression not supported prior to 1.8.3 */
if (LZ4_versionNumber() < 10803)
return lz4_decompress_datum(value);
It seems likely to me that we'd get a flat out build failure
from library versions lacking LZ4_decompress_safe_partial,
and thus that this run-time test is dead code and we should
better be using a configure probe if we intend to allow old
liblz4 versions. Though that might be moot.
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Sun, Mar 21, 2021 at 07:11:50PM -0400, Tom Lane wrote:
I wrote:
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sun, Mar 21, 2021 at 04:32:31PM -0400, Tom Lane wrote:
This seems somewhat repeatable (three identical failures in three
attempts). Not sure why I did not see it yesterday; but anyway,
there is something wrong with partial detoasting for LZ4.With what version of LZ4 ?
RHEL8's, which is
lz4-1.8.3-2.el8.x86_64I hate to be the bearer of bad news, but this suggests that
LZ4_decompress_safe_partial is seriously broken in 1.9.2
as well:
Ouch
Maybe we cannot rely on that function for a few more years yet.
Also, I don't really understand why this code:
/* slice decompression not supported prior to 1.8.3 */
if (LZ4_versionNumber() < 10803)
return lz4_decompress_datum(value);It seems likely to me that we'd get a flat out build failure
from library versions lacking LZ4_decompress_safe_partial,
and thus that this run-time test is dead code and we should
better be using a configure probe if we intend to allow old
liblz4 versions. Though that might be moot.
The function existed before 1.8.3, but didn't handle slicing.
https://github.com/lz4/lz4/releases/tag/v1.8.3
|Finally, an existing function, LZ4_decompress_safe_partial(), has been enhanced to make it possible to decompress only the beginning of an LZ4 block, up to a specified number of bytes. Partial decoding can be useful to save CPU time and memory, when the objective is to extract a limited portion from a larger block.
Possibly we could allow v >= 1.9.3 || (ver >= 1.8.3 && ver < 1.9.2).
Or maybe not: the second half apparently worked "by accident", and we shouldn't
need to have intimate knowledge of someone else's patchlevel releases,
--
Justin
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Justin Pryzby <pryzby@telsasoft.com> writes:
On Sun, Mar 21, 2021 at 07:11:50PM -0400, Tom Lane wrote:
I hate to be the bearer of bad news, but this suggests that
LZ4_decompress_safe_partial is seriously broken in 1.9.2
as well:
https://github.com/lz4/lz4/issues/783
Ouch
Actually, after reading that closer, the problem only affects the
case where the compressed-data-length passed to the function is
a lie. So it shouldn't be a problem for our usage.
Also, after studying the documentation for LZ4_decompress_safe
and LZ4_decompress_safe_partial, I realized that liblz4 is also
counting on the *output* buffer size to not be a lie. So we
cannot pass it a number larger than the chunk's true decompressed
size. The attached patch resolves the issue I'm seeing.
regards, tom lane
Attachments:
fix-partial-lz4-decompression.patchtext/x-diff; charset=us-ascii; name=fix-partial-lz4-decompression.patchDownload
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 00af1740cf..74e449992a 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -220,6 +220,10 @@ lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
if (LZ4_versionNumber() < 10803)
return lz4_decompress_datum(value);
+ /* liblz4 assumes that slicelength is not an overestimate */
+ if (slicelength >= VARRAWSIZE_4B_C(value))
+ return lz4_decompress_datum(value);
+
/* allocate memory for the uncompressed data */
result = (struct varlena *) palloc(slicelength + VARHDRSZ);
Re: [HACKERS] Custom compression methods
On Sat, Mar 20, 2021 at 06:20:39PM -0500, Justin Pryzby wrote:
Rebased on HEAD.
0005 forgot to update compression_1.out.
Included changes to ./configure.ac and some other patches, but not Tomas's,
since it'll make CFBOT get mad as soon as that's pushed.
Rebased again.
Renamed "t" to a badcompresstbl to avoid name conflicts.
Polish the enum GUC patch some.
I noticed that TOAST_INVALID_COMPRESSION_ID was unused ... but then I found a
use for it.
Attachments:
v3-0001-Add-docs-for-default_toast_compression.patchtext/x-diff; charset=us-asciiDownload
From edc0e6ab248600c6d33dd681359554550c88a679 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 19:12:53 -0500
Subject: [PATCH v3 01/12] Add docs for default_toast_compression..
bbe0a81db69bd10bd166907c3701492a29aca294
---
doc/src/sgml/config.sgml | 23 +++++++++++++++++++++++
1 file changed, 23 insertions(+)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index ee4925d6d9..5cb851a5eb 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8151,6 +8151,29 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
+ <term><varname>default_toast_compression</varname> (<type>string</type>)
+ <indexterm>
+ <primary><varname>default_toast_compression</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This parameter specifies the default compression method to use
+ when compressing data in <acronym>TOAST</acronym> tables.
+ It applies only to variable-width data types.
+ It may be overriden by compression clauses in the
+ <command>CREATE</command> command, or changed after the relation is
+ created by <command>ALTER TABLE ... SET COMPRESSION</command>.
+
+ The supported compression methods are <literal>pglz</literal> and
+ (if configured at the time <productname>PostgreSQL</productname> was
+ built) <literal>lz4</literal>.
+ The default is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-temp-tablespaces" xreflabel="temp_tablespaces">
<term><varname>temp_tablespaces</varname> (<type>string</type>)
<indexterm>
--
2.17.0
v3-0002-doc-pg_dump-no-toast-compression.patchtext/x-diff; charset=us-asciiDownload
From 02a9038381bdfe577aa32b67bcd231acd61f2c20 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 21:52:28 -0500
Subject: [PATCH v3 02/12] doc: pg_dump --no-toast-compression
---
doc/src/sgml/ref/pg_dump.sgml | 12 ++++++++++++
1 file changed, 12 insertions(+)
diff --git a/doc/src/sgml/ref/pg_dump.sgml b/doc/src/sgml/ref/pg_dump.sgml
index bcbb7a25fb..989b8e2381 100644
--- a/doc/src/sgml/ref/pg_dump.sgml
+++ b/doc/src/sgml/ref/pg_dump.sgml
@@ -931,6 +931,18 @@ PostgreSQL documentation
</listitem>
</varlistentry>
+ <varlistentry>
+ <term><option>--no-toast-compression</option></term>
+ <listitem>
+ <para>
+ Do not output commands to set <acronym>TOAST</acronym> compression
+ methods.
+ With this option, all objects will be created using whichever
+ compression method is the default during restore.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry>
<term><option>--no-unlogged-table-data</option></term>
<listitem>
--
2.17.0
v3-0003-Compression-method-is-an-char-not-an-OID.patchtext/x-diff; charset=us-asciiDownload
From 7e9b4b3c6f12abaf7531ef5109e084dc188e7dda Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:23:40 -0500
Subject: [PATCH v3 03/12] Compression method is an char not an OID
---
src/backend/commands/tablecmds.c | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 877c08814e..22f3c5efc0 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -7847,6 +7847,7 @@ SetIndexStorageProperties(Relation rel, Relation attrelation,
index_close(indrel, lockmode);
}
}
+
/*
* ALTER TABLE ALTER COLUMN SET STORAGE
*
@@ -15070,7 +15071,7 @@ ATExecSetCompression(AlteredTableInfo *tab,
AttrNumber attnum;
char *compression;
char typstorage;
- Oid cmoid;
+ char cmethod;
ObjectAddress address;
Assert(IsA(newValue, String));
@@ -15104,10 +15105,10 @@ ATExecSetCompression(AlteredTableInfo *tab,
format_type_be(atttableform->atttypid))));
/* get the attribute compression method. */
- cmoid = GetAttributeCompression(atttableform, compression);
+ cmethod = GetAttributeCompression(atttableform, compression);
/* update pg_attribute entry */
- atttableform->attcompression = cmoid;
+ atttableform->attcompression = cmethod;
CatalogTupleUpdate(attrel, &tuple->t_self, tuple);
InvokeObjectPostAlterHook(RelationRelationId,
@@ -15118,7 +15119,7 @@ ATExecSetCompression(AlteredTableInfo *tab,
* Apply the change to indexes as well (only for simple index columns,
* matching behavior of index.c ConstructTupleDescriptor()).
*/
- SetIndexStorageProperties(rel, attrel, attnum, cmoid, '\0', lockmode);
+ SetIndexStorageProperties(rel, attrel, attnum, cmethod, '\0', lockmode);
heap_freetuple(tuple);
--
2.17.0
v3-0004-Remove-duplicative-macro.patchtext/x-diff; charset=us-asciiDownload
From 2262c8ffb041e8a7452bda724c676b6118895f4b Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:07:31 -0500
Subject: [PATCH v3 04/12] Remove duplicative macro
---
src/include/access/toast_compression.h | 2 --
1 file changed, 2 deletions(-)
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 5c9220c171..44b73bd57c 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -50,8 +50,6 @@ typedef enum ToastCompressionId
errdetail("This functionality requires the server to be built with lz4 support."), \
errhint("You need to rebuild PostgreSQL using --with-lz4.")))
-#define IsValidCompression(cm) ((cm) != InvalidCompressionMethod)
-
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
--
2.17.0
v3-0005-Error-on-invalid-compression-in-CREATE-and-ALTER.patchtext/x-diff; charset=us-asciiDownload
From 2deb4e72773a0d0fe9cff73527b3464329b8763f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 20:19:55 -0500
Subject: [PATCH v3 05/12] Error on invalid compression in CREATE and ALTER
---
src/backend/commands/tablecmds.c | 10 +++++++---
src/test/regress/expected/compression.out | 6 ++++++
src/test/regress/expected/compression_1.out | 6 ++++++
src/test/regress/sql/compression.sql | 5 +++++
4 files changed, 24 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 22f3c5efc0..54fea31e43 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -17863,9 +17863,13 @@ GetAttributeCompression(Form_pg_attribute att, char *compression)
/* fallback to default compression if it's not specified */
if (compression == NULL)
- cmethod = GetDefaultToastCompression();
- else
- cmethod = CompressionNameToMethod(compression);
+ return GetDefaultToastCompression();
+
+ cmethod = CompressionNameToMethod(compression);
+ if (!CompressionMethodIsValid(cmethod))
+ ereport(ERROR,
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED),
+ errmsg("invalid compression method \"%s\"", compression)));
return cmethod;
}
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 18ac5f05bb..c2f2e0e230 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -347,4 +347,10 @@ SELECT length(f1) FROM cmmove3;
10040
(2 rows)
+CREATE TABLE badcompresstbl (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+CREATE TABLE badcompresstbl (a text);
+ALTER TABLE badcompresstbl ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+DROP TABLE badcompresstbl;
\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index c4a2cea4cd..6626f8e927 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -340,4 +340,10 @@ SELECT length(f1) FROM cmmove3;
10000
(1 row)
+CREATE TABLE badcompresstbl (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+CREATE TABLE badcompresstbl (a text);
+ALTER TABLE badcompresstbl ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+ERROR: invalid compression method "i_do_not_exist_compression"
+DROP TABLE badcompresstbl;
\set HIDE_TOAST_COMPRESSION true
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index e23669cc94..5e178be04a 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -137,4 +137,9 @@ SELECT length(f1) FROM cmmove1;
SELECT length(f1) FROM cmmove2;
SELECT length(f1) FROM cmmove3;
+CREATE TABLE badcompresstbl (a text COMPRESSION I_Do_Not_Exist_Compression); -- fails
+CREATE TABLE badcompresstbl (a text);
+ALTER TABLE badcompresstbl ALTER a SET COMPRESSION I_Do_Not_Exist_Compression; -- fails
+DROP TABLE badcompresstbl;
+
\set HIDE_TOAST_COMPRESSION true
--
2.17.0
v3-0006-WIP-Change-default_toast_compression-GUC-to-an-en.patchtext/x-diff; charset=us-asciiDownload
From 005ae4b7668a31dd4a7ab1c25ce3af3420346d2f Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Fri, 19 Mar 2021 19:39:10 -0500
Subject: [PATCH v3 06/12] WIP: Change default_toast_compression GUC to an
enum..
..since the final version of the TOAST lz4 patch doesn't require catalog access
---
src/backend/access/common/toast_compression.c | 57 ++++---------------
src/backend/utils/misc/guc.c | 23 ++++----
src/include/access/toast_compression.h | 11 ++--
src/test/regress/expected/compression.out | 4 +-
src/test/regress/expected/compression_1.out | 9 ++-
5 files changed, 33 insertions(+), 71 deletions(-)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 00af1740cf..aa28772159 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -23,8 +23,16 @@
#include "fmgr.h"
#include "utils/builtins.h"
-/* Compile-time default */
-char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* GUC */
+int default_toast_compression = TOAST_PGLZ_COMPRESSION_ID;
+
+const struct config_enum_entry default_toast_compression_options[] = {
+ {"pglz", TOAST_PGLZ_COMPRESSION_ID, false},
+#ifdef USE_LZ4
+ {"lz4", TOAST_LZ4_COMPRESSION_ID, false},
+#endif
+ {NULL, 0, false}
+};
/*
* Compress a varlena using PGLZ.
@@ -269,48 +277,3 @@ toast_get_compression_id(struct varlena *attr)
return cmid;
}
-
-/*
- * Validate a new value for the default_toast_compression GUC.
- */
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
-{
- if (**newval == '\0')
- {
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
- }
-
- if (strlen(*newval) >= NAMEDATALEN)
- {
- GUC_check_errdetail("%s is too long (maximum %d characters).",
- "default_toast_compression", NAMEDATALEN - 1);
- return false;
- }
-
- if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
- {
- /*
- * When source == PGC_S_TEST, don't throw a hard error for a
- * nonexistent compression method, only a NOTICE. See comments in
- * guc.h.
- */
- if (source == PGC_S_TEST)
- {
- ereport(NOTICE,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("compression method \"%s\" does not exist",
- *newval)));
- }
- else
- {
- GUC_check_errdetail("Compression method \"%s\" does not exist.",
- *newval);
- return false;
- }
- }
-
- return true;
-}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 4272cf4821..5480d3d088 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -518,6 +518,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry default_toast_compression_options[];
/*
* GUC option variables that are exported from this module
@@ -3953,17 +3954,6 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
- {
- {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default compression for new columns."),
- NULL,
- GUC_IS_NAME
- },
- &default_toast_compression,
- DEFAULT_TOAST_COMPRESSION,
- check_default_toast_compression, NULL, NULL
- },
-
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
@@ -4605,6 +4595,17 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ TOAST_PGLZ_COMPRESSION_ID,
+ default_toast_compression_options, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 44b73bd57c..c5cea08ccd 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -16,10 +16,8 @@
#include "utils/guc.h"
/* GUCs */
-extern char *default_toast_compression;
-
-/* default compression method if not specified. */
-#define DEFAULT_TOAST_COMPRESSION "pglz"
+extern int default_toast_compression;
+extern const struct config_enum_entry default_toast_compression_options[];
/*
* Built-in compression method-id. The toast compression header will store
@@ -78,7 +76,7 @@ GetCompressionMethodName(char method)
* in the built-in methods then return InvalidCompressionMethod.
*/
static inline char
-CompressionNameToMethod(char *compression)
+CompressionNameToMethod(const char *compression)
{
if (strcmp(compression, "pglz") == 0)
return TOAST_PGLZ_COMPRESSION;
@@ -101,7 +99,8 @@ CompressionNameToMethod(char *compression)
static inline char
GetDefaultToastCompression(void)
{
- return CompressionNameToMethod(default_toast_compression);
+ Assert(default_toast_compression < TOAST_INVALID_COMPRESSION_ID);
+ return CompressionNameToMethod(default_toast_compression_options[default_toast_compression].name);
}
/* pglz compression/decompression routines */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index c2f2e0e230..566a1877ea 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -234,10 +234,10 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6626f8e927..3990933415 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,14 +227,13 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz.
SET default_toast_compression = 'lz4';
-ERROR: unsupported LZ4 compression method
-DETAIL: This functionality requires the server to be built with lz4 support.
-HINT: You need to rebuild PostgreSQL using --with-lz4.
+ERROR: invalid value for parameter "default_toast_compression": "lz4"
+HINT: Available values: pglz.
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
--
2.17.0
v3-0008-Commentary-about-slicing.patchtext/x-diff; charset=iso-8859-1Download
From fc2b846ed482f5407278a171da3ce3c847425a00 Mon Sep 17 00:00:00 2001
From: Justin Pryzby <pryzbyj@telsasoft.com>
Date: Sat, 20 Mar 2021 16:13:44 -0500
Subject: [PATCH v3 08/12] Commentary about slicing
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
My language, per gripe from Ãlvaro Herrera:
| I updated the coverage script to use --with-lz4; results are updated.
| While eyeballing the results I noticed this bit in
| lz4_decompress_datum_slice():
|
| + /* slice decompression not supported prior to 1.8.3 */
| + if (LZ4_versionNumber() < 10803)
| + return lz4_decompress_datum(value);
|
| which I read as returning the complete decompressed datum if slice
| decompression is not supported. I thought that was a bug, but looking
| at the caller I realize that this isn't really a problem, since it's
| detoast_attr_slice's responsibility to slice the result further -- no
| bug, it's just wasteful. I suggest to add comments to this effect,
| perhaps as the attached (feel free to reword, I think mine is awkward.)
---
src/backend/access/common/detoast.c | 3 +++
src/backend/access/common/toast_compression.c | 3 +++
2 files changed, 6 insertions(+)
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index bed50e8603..519fe8c6a6 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -498,6 +498,9 @@ toast_decompress_datum(struct varlena *attr)
* Decompress the front of a compressed version of a varlena datum.
* offset handling happens in detoast_attr_slice.
* Here we just decompress a slice from the front.
+ *
+ * If slice decompression is not supported, the full datum is decompressed, and
+ * then sliced by detoast_attr_slice.
*/
static struct varlena *
toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index aa28772159..9107d16445 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -213,6 +213,9 @@ lz4_decompress_datum(const struct varlena *value)
/*
* Decompress part of a varlena that was compressed using LZ4.
+ *
+ * If slice decompression is not supported, the full datum is decompressed, and
+ * then sliced by detoast_attr_slice.
*/
struct varlena *
lz4_decompress_datum_slice(const struct varlena *value, int32 slicelength)
--
2.17.0
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Mon, Mar 22, 2021 at 5:22 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Actually, after reading that closer, the problem only affects the
case where the compressed-data-length passed to the function is
a lie. So it shouldn't be a problem for our usage.Also, after studying the documentation for LZ4_decompress_safe
and LZ4_decompress_safe_partial, I realized that liblz4 is also
counting on the *output* buffer size to not be a lie. So we
cannot pass it a number larger than the chunk's true decompressed
size. The attached patch resolves the issue I'm seeing.
Okay, the fix makes sense. In fact, IMHO, in general also this fix
looks like an optimization, I mean when slicelength >=
VARRAWSIZE_4B_C(value), then why do we need to allocate extra memory
even in the case of pglz. So shall we put this check directly in
toast_decompress_datum_slice instead of handling it at the lz4 level?
Like this.
diff --git a/src/backend/access/common/detoast.c
b/src/backend/access/common/detoast.c
index bed50e8..099ac15 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -506,6 +506,10 @@ toast_decompress_datum_slice(struct varlena
*attr, int32 slicelength)Assert(VARATT_IS_COMPRESSED(attr));
+ /* liblz4 assumes that slicelength is not an overestimate */
+ if (slicelength >= VARRAWSIZE_4B_C(attr))
+ return toast_decompress_datum(attr);
+
/*
* Fetch the compression method id stored in the compression header and
* decompress the data slice using the appropriate
decompression routine.--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 5:25 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Sat, Mar 20, 2021 at 06:20:39PM -0500, Justin Pryzby wrote:
Rebased on HEAD.
0005 forgot to update compression_1.out.
Included changes to ./configure.ac and some other patches, but not Tomas's,
since it'll make CFBOT get mad as soon as that's pushed.Rebased again.
Renamed "t" to a badcompresstbl to avoid name conflicts.
Polish the enum GUC patch some.I noticed that TOAST_INVALID_COMPRESSION_ID was unused ... but then I found a
use for it.
Yeah, it is used in toast_compress_datum, toast_get_compression_id,
reform_and_rewrite_tuple and pg_column_compression function. Your
patches look fine to me. I agree that v3-0006 also makes sense as it
is simplifying the GUC handling. Thanks for fixing these.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Sun, Mar 21, 2021 at 7:55 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Rebased again.
Thanks, Justin. I committed 0003 and 0004 together as
226e2be3876d0bda3dc33d16dfa0bed246b7b74f. I also committed 0001 and
0002 together as 24f0e395ac5892cd12e8914646fe921fac5ba23d, but with
some revisions, because your text was not clear that this is setting
the default for new tables, not new values; it also implied that this
only affects out-of-line compression, which is not true. In lieu of
trying to explain how TOAST works here, I added a link. It looks,
though, like that documentation also needs to be patched for this
change. I'll look into that, and your remaining patches, next.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Dilip Kumar <dilipbalaut@gmail.com> writes:
On Mon, Mar 22, 2021 at 5:22 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Also, after studying the documentation for LZ4_decompress_safe
and LZ4_decompress_safe_partial, I realized that liblz4 is also
counting on the *output* buffer size to not be a lie. So we
cannot pass it a number larger than the chunk's true decompressed
size. The attached patch resolves the issue I'm seeing.
Okay, the fix makes sense. In fact, IMHO, in general also this fix
looks like an optimization, I mean when slicelength >=
VARRAWSIZE_4B_C(value), then why do we need to allocate extra memory
even in the case of pglz. So shall we put this check directly in
toast_decompress_datum_slice instead of handling it at the lz4 level?
Yeah, I thought about that too, but do we want to assume that
VARRAWSIZE_4B_C is the correct way to get the decompressed size
for all compression methods?
(If so, I think it would be better style to have a less opaque macro
name for the purpose.)
regards, tom lane
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 10:41:33AM -0400, Robert Haas wrote:
On Sun, Mar 21, 2021 at 7:55 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
Rebased again.
Thanks, Justin. I committed 0003 and 0004 together as
226e2be3876d0bda3dc33d16dfa0bed246b7b74f. I also committed 0001 and
0002 together as 24f0e395ac5892cd12e8914646fe921fac5ba23d, but with
some revisions, because your text was not clear that this is setting
the default for new tables, not new values; it also implied that this
only affects out-of-line compression, which is not true. In lieu of
trying to explain how TOAST works here, I added a link. It looks,
though, like that documentation also needs to be patched for this
change. I'll look into that, and your remaining patches, next.
Thanks. I just realized that if you also push the GUC change, then the docs
should change from <string> to <enum>
doc/src/sgml/config.sgml: <term><varname>default_toast_compression</varname> (<type>string</type>)
--
Justin
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 10:44 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks. I just realized that if you also push the GUC change, then the docs
should change from <string> to <enum>doc/src/sgml/config.sgml: <term><varname>default_toast_compression</varname> (<type>string</type>)
I've now also committed your 0005. As for 0006, aside from the note
above, which is a good one, is there any particular reason why this
patch is labelled as WIP? I think this change makes sense and we
should just do it unless there's some problem with it.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 11:05:19AM -0400, Robert Haas wrote:
On Mon, Mar 22, 2021 at 10:44 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
Thanks. I just realized that if you also push the GUC change, then the docs
should change from <string> to <enum>doc/src/sgml/config.sgml: <term><varname>default_toast_compression</varname> (<type>string</type>)
I've now also committed your 0005. As for 0006, aside from the note
above, which is a good one, is there any particular reason why this
patch is labelled as WIP? I think this change makes sense and we
should just do it unless there's some problem with it.
The first iteration was pretty rough, and there's still some question in my
mind about where default_toast_compression_options[] should be defined. If
it's in the header file, then I could use lengthof() - but then it probably
gets multiply defined. In the latest patch, there's multiple "externs". Maybe
guc.c doesn't need the extern, since it includes toast_compression.h. But then
it's the only "struct config_enum_entry" which has an "extern" outside of
guc.c.
Also, it looks like you added default_toast_compression out of order, so maybe
you'd fix that at the same time.
--
Justin
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Mon, Mar 22, 2021 at 8:11 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Dilip Kumar <dilipbalaut@gmail.com> writes:
On Mon, Mar 22, 2021 at 5:22 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Also, after studying the documentation for LZ4_decompress_safe
and LZ4_decompress_safe_partial, I realized that liblz4 is also
counting on the *output* buffer size to not be a lie. So we
cannot pass it a number larger than the chunk's true decompressed
size. The attached patch resolves the issue I'm seeing.Okay, the fix makes sense. In fact, IMHO, in general also this fix
looks like an optimization, I mean when slicelength >=
VARRAWSIZE_4B_C(value), then why do we need to allocate extra memory
even in the case of pglz. So shall we put this check directly in
toast_decompress_datum_slice instead of handling it at the lz4 level?Yeah, I thought about that too, but do we want to assume that
VARRAWSIZE_4B_C is the correct way to get the decompressed size
for all compression methods?
Yeah, VARRAWSIZE_4B_C is the macro getting the rawsize of the data
stored in the compressed varlena.
(If so, I think it would be better style to have a less opaque macro
name for the purpose.)
Okay, I have added another macro that is less opaque and came up with
this patch.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v1-0001-fix-slice-decompression.patchapplication/octet-stream; name=v1-0001-fix-slice-decompression.patchDownload
From 343801cf8dfa79c3983cab813c5c48db3a390adb Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Mon, 22 Mar 2021 20:40:43 +0530
Subject: [PATCH v1] fix slice decompression
If the slicelength is more than the actual size of the data then no
point in trying to decompress the slice so directly decompress complete
data.
---
src/backend/access/common/detoast.c | 8 ++++++++
src/include/access/toast_internals.h | 2 ++
2 files changed, 10 insertions(+)
diff --git a/src/backend/access/common/detoast.c b/src/backend/access/common/detoast.c
index bed50e8..e6e57ad 100644
--- a/src/backend/access/common/detoast.c
+++ b/src/backend/access/common/detoast.c
@@ -506,6 +506,14 @@ toast_decompress_datum_slice(struct varlena *attr, int32 slicelength)
Assert(VARATT_IS_COMPRESSED(attr));
+ /*
+ * If the slicelength is more than the actual size of the data then no
+ * point in trying to decompress the slice so directly decompress complete
+ * data.
+ */
+ if (slicelength >= TOAST_COMPRESS_RAWSIZE(attr))
+ return toast_decompress_datum(attr);
+
/*
* Fetch the compression method id stored in the compression header and
* decompress the data slice using the appropriate decompression routine.
diff --git a/src/include/access/toast_internals.h b/src/include/access/toast_internals.h
index b4d0684..e1e3521 100644
--- a/src/include/access/toast_internals.h
+++ b/src/include/access/toast_internals.h
@@ -31,6 +31,8 @@ typedef struct toast_compress_header
* Utilities for manipulation of header information for compressed
* toast entries.
*/
+#define TOAST_COMPRESS_RAWSIZE(ptr) \
+ (((toast_compress_header *) (ptr))->tcinfo & VARLENA_RAWSIZE_MASK)
#define TOAST_COMPRESS_METHOD(ptr) \
(((toast_compress_header *) (ptr))->tcinfo >> VARLENA_RAWSIZE_BITS)
#define TOAST_COMPRESS_SET_SIZE_AND_METHOD(ptr, len, cm_method) \
--
1.8.3.1
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Mon, Mar 22, 2021 at 10:41 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Okay, the fix makes sense. In fact, IMHO, in general also this fix
looks like an optimization, I mean when slicelength >=
VARRAWSIZE_4B_C(value), then why do we need to allocate extra memory
even in the case of pglz. So shall we put this check directly in
toast_decompress_datum_slice instead of handling it at the lz4 level?Yeah, I thought about that too, but do we want to assume that
VARRAWSIZE_4B_C is the correct way to get the decompressed size
for all compression methods?
I think it's OK to assume this. If and when we add a third compression
method, it seems certain to just grab one of the two remaining bit
patterns. Now, things get a bit more complicated if and when we want
to add a fourth method, because at that point you've got to ask
yourself how comfortable you feel about stealing the last bit pattern
for your feature. But, if the solution to that problem were to decide
that whenever that last bit pattern is used, we will add an extra byte
(or word) after va_tcinfo indicating the real compression method, then
using VARRAWSIZE_4B_C here would still be correct. To imagine this
decision being wrong, you have to posit a world in which one of the
two remaining bit patterns for the high 2 bits cause the low 30 bits
to be interpreted as something other than the size, which I guess is
not totally impossible, but my first reaction is to think that such a
design would be (1) hard to make work and (2) unnecessarily painful.
(If so, I think it would be better style to have a less opaque macro
name for the purpose.)
Complaining about the name of one particular TOAST-related macro name
seems a bit like complaining about the greenhouse gasses emitted by
one particular car. They're pretty uniformly terrible. Does anyone
really know when to use VARATT_IS_1B_E or VARATT_IS_4B_U or any of
that cruft? Like, who decided that "is this varatt 1B E?" would be a
perfectly reasonable way of asking "is this varlena is TOAST
pointer?". While I'm complaining, it's hard to say enough bad things
about the fact that we have 12 consecutive completely obscure macro
definitions for which the only comments are (a) that they are
endian-dependent - which isn't even true for all of them - and (b)
that they are "considered internal." Apparently, they're SO internal
that they don't even need to be understandable to other developers.
Anyway, this particular macro name was chosen, it seems, for symmetry
with VARDATA_4B_C, but if you want to change it to something else, I'm
OK with that, too.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Robert Haas <robertmhaas@gmail.com> writes:
On Mon, Mar 22, 2021 at 10:41 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Yeah, I thought about that too, but do we want to assume that
VARRAWSIZE_4B_C is the correct way to get the decompressed size
for all compression methods?
I think it's OK to assume this.
OK, cool.
(If so, I think it would be better style to have a less opaque macro
name for the purpose.)
Complaining about the name of one particular TOAST-related macro name
seems a bit like complaining about the greenhouse gasses emitted by
one particular car.
Maybe, but that's not a reason to make it worse. Anyway, my understanding
of that is that the really opaque names are *only* meant to be used in
this very stretch of postgres.h, ie they are just intermediate steps on
the way to the macros below them. As an example, the only use of
VARDATA_1B_E() is in VARDATA_EXTERNAL().
Anyway, this particular macro name was chosen, it seems, for symmetry
with VARDATA_4B_C, but if you want to change it to something else, I'm
OK with that, too.
After looking at postgres.h for a bit, I'm thinking that what these
should have been symmetric with is the considerably-less-terrible
names used for the corresponding VARATT_EXTERNAL cases. Thus,
something like
s/VARRAWSIZE_4B_C/VARDATA_COMPRESSED_GET_RAWSIZE/
s/VARCOMPRESS_4B_C/VARDATA_COMPRESSED_GET_COMPRESSION/
Possibly the former names should survive and the latter become
wrappers around them, not sure. But we shouldn't be using the "4B"
terminology anyplace except this part of postgres.h.
regards, tom lane
Re: [HACKERS] Custom compression methods (mac+lz4.h)
On Mon, Mar 22, 2021 at 11:48 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Anyway, this particular macro name was chosen, it seems, for symmetry
with VARDATA_4B_C, but if you want to change it to something else, I'm
OK with that, too.After looking at postgres.h for a bit, I'm thinking that what these
should have been symmetric with is the considerably-less-terrible
names used for the corresponding VARATT_EXTERNAL cases. Thus,
something likes/VARRAWSIZE_4B_C/VARDATA_COMPRESSED_GET_RAWSIZE/
s/VARCOMPRESS_4B_C/VARDATA_COMPRESSED_GET_COMPRESSION/
Works for me.
Possibly the former names should survive and the latter become
wrappers around them, not sure. But we shouldn't be using the "4B"
terminology anyplace except this part of postgres.h.
I would argue that it shouldn't be used any place at all, and that we
ought to go the other direction and get rid of the existing macros -
e.g. change #define VARATT_IS_1B_E to #define VARATT_IS_EXTERNAL
instead of defining the latter as a no-value-added wrapper around the
former. Maybe at one time somebody thought that the test for
VARATT_IS_EXTERNAL might someday have more cases than just
VARATT_IS_1B_E, but that's not looking like a good bet in 2021.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods (mac+lz4.h)
Robert Haas <robertmhaas@gmail.com> writes:
On Mon, Mar 22, 2021 at 11:48 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Possibly the former names should survive and the latter become
wrappers around them, not sure. But we shouldn't be using the "4B"
terminology anyplace except this part of postgres.h.
I would argue that it shouldn't be used any place at all, and that we
ought to go the other direction and get rid of the existing macros -
e.g. change #define VARATT_IS_1B_E to #define VARATT_IS_EXTERNAL
instead of defining the latter as a no-value-added wrapper around the
former. Maybe at one time somebody thought that the test for
VARATT_IS_EXTERNAL might someday have more cases than just
VARATT_IS_1B_E, but that's not looking like a good bet in 2021.
Maybe. I think the original idea was exactly what the comment says,
to have a layer of macros that'd deal with endianness issues and no more.
That still seems like a reasonable plan to me, though perhaps it wasn't
executed very well.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 11:13 AM Justin Pryzby <pryzby@telsasoft.com> wrote:
The first iteration was pretty rough, and there's still some question in my
mind about where default_toast_compression_options[] should be defined. If
it's in the header file, then I could use lengthof() - but then it probably
gets multiply defined.
What do you want to use lengthof() for?
In the latest patch, there's multiple "externs". Maybe
guc.c doesn't need the extern, since it includes toast_compression.h. But then
it's the only "struct config_enum_entry" which has an "extern" outside of
guc.c.
Oh, yeah, we certainly shouldn't have an extern in guc.c itself, if
we've already got it in the header file.
As to the more general question of where to put stuff, I don't think
there's any conceptual problem with putting it in a header file rather
than in guc.c. It's not very scalable to just keeping inventing new
GUCs and sticking all their accoutrement into guc.c. That might have
kind of made sense when guc.c was invented, since there were probably
fewer settings there and guc.c itself was new, but at this point it's
a well-established part of the infrastructure and having other
subsystems cater to what it needs rather than the other way around
seems logical. However, it's not great to have "utils/guc.h" included
in "access/toast_compression.h", because then anything that includes
"access/toast_compression.h" or "access/toast_internals.h" sucks in
"utils/guc.h" even though it's not really topically related to what
they intended to include. We can't avoid that just by choosing to put
this enum in guc.c, because GetDefaultToastCompression() also uses it.
But, what about giving the default_toast_compression_method GUC an
assign hook that sets a global variable of type "char" to the
appropriate value? Then GetDefaultToastCompression() goes away
entirely. That might be worth exploring.
Also, it looks like you added default_toast_compression out of order, so maybe
you'd fix that at the same time.
You know, I looked at where you had it and said to myself, "surely
this is a silly place to put this, it would make much more sense to
move this up a bit." Now I feel dumb.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 12:16 PM Robert Haas <robertmhaas@gmail.com> wrote:
But, what about giving the default_toast_compression_method GUC an
assign hook that sets a global variable of type "char" to the
appropriate value? Then GetDefaultToastCompression() goes away
entirely. That might be worth exploring.
Actually, we can do even better. We should just make the values
actually assigned to the GUC be TOAST_PGLZ_COMPRESSION etc. rather
than TOAST_PGLZ_COMPRESSION_ID etc. Then a whole lot of complexity
just goes away. I added some comments explaining why using
TOAST_PGLZ_COMPRESSION is the wrong thing anyway. Then I got hacking
and rearranged a few other things. So the attached patch does these
thing:
- Changes default_toast_compression to an enum, as in your patch, but
now with values that are the same as what ultimately gets stored in
attcompression.
- Adds a comment warning against incautious use of
TOAST_PGLZ_COMPRESSION_ID, etc.
- Moves default_toast_compression_options to guc.c.
- After doing the above two things, we can remove the #include of
utils/guc.h into access/toast_compression.h, so the patch does that.
- Moves NO_LZ4_SUPPORT, GetCompressionMethodName, and
CompressionNameToMethod to guc.c. Making these inline functions
doesn't save anything meaningful; it's more important not to export a
bunch of random identifiers.
- Removes an unnecessary cast to bool from the definition of
CompressionMethodIsValid.
I think this is significantly cleaner than what we have now, and I
also prefer it to your proposal.
Comments?
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
toast-compression-guc-rmh.patchapplication/octet-stream; name=toast-compression-guc-rmh.patchDownload
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 00af1740cf..d772cc0aaa 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -23,8 +23,15 @@
#include "fmgr.h"
#include "utils/builtins.h"
-/* Compile-time default */
-char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* GUC */
+int default_toast_compression = TOAST_PGLZ_COMPRESSION;
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
/*
* Compress a varlena using PGLZ.
@@ -271,46 +278,41 @@ toast_get_compression_id(struct varlena *attr)
}
/*
- * Validate a new value for the default_toast_compression GUC.
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
*/
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
+char
+CompressionNameToMethod(const char *compression)
{
- if (**newval == '\0')
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
{
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
}
- if (strlen(*newval) >= NAMEDATALEN)
- {
- GUC_check_errdetail("%s is too long (maximum %d characters).",
- "default_toast_compression", NAMEDATALEN - 1);
- return false;
- }
+ return InvalidCompressionMethod;
+}
- if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+const char *
+GetCompressionMethodName(char method)
+{
+ switch (method)
{
- /*
- * When source == PGC_S_TEST, don't throw a hard error for a
- * nonexistent compression method, only a NOTICE. See comments in
- * guc.h.
- */
- if (source == PGC_S_TEST)
- {
- ereport(NOTICE,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("compression method \"%s\" does not exist",
- *newval)));
- }
- else
- {
- GUC_check_errdetail("Compression method \"%s\" does not exist.",
- *newval);
- return false;
- }
+ case TOAST_PGLZ_COMPRESSION:
+ return "pglz";
+ case TOAST_LZ4_COMPRESSION:
+ return "lz4";
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ return NULL; /* keep compiler quiet */
}
-
- return true;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 3b36a31a47..a4169a6517 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -509,6 +509,14 @@ static struct config_enum_entry shared_memory_options[] = {
{NULL, 0, false}
};
+const struct config_enum_entry default_toast_compression_options[] = {
+ {"pglz", TOAST_PGLZ_COMPRESSION, false},
+#ifdef USE_LZ4
+ {"lz4", TOAST_LZ4_COMPRESSION, false},
+#endif
+ {NULL, 0, false}
+};
+
/*
* Options for enum values stored in other modules
*/
@@ -517,6 +525,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry default_toast_compression_options[];
/*
* GUC option variables that are exported from this module
@@ -3943,17 +3952,6 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
- {
- {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default compression for new columns."),
- NULL,
- GUC_IS_NAME
- },
- &default_toast_compression,
- DEFAULT_TOAST_COMPRESSION,
- check_default_toast_compression, NULL, NULL
- },
-
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
@@ -4595,6 +4593,17 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ TOAST_PGLZ_COMPRESSION,
+ default_toast_compression_options, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 44b73bd57c..6cf439ef1a 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,18 +13,20 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
-#include "utils/guc.h"
-
/* GUCs */
-extern char *default_toast_compression;
-
-/* default compression method if not specified. */
-#define DEFAULT_TOAST_COMPRESSION "pglz"
+extern int default_toast_compression;
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression methods.
+ *
+ * Don't use these values for anything other than understanding the meaning
+ * of the raw bits from a varlena; in particular, if the goal is to identify
+ * a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc.
+ * below. We might someday support more than 4 compression methods, but
+ * we can never have more than 4 values in this enum, because there are
+ * only 2 bits available in the places where this is used.
*/
typedef enum ToastCompressionId
{
@@ -39,60 +41,13 @@ typedef enum ToastCompressionId
*/
#define TOAST_PGLZ_COMPRESSION 'p'
#define TOAST_LZ4_COMPRESSION 'l'
+#define InvalidCompressionMethod '\0'
-#define InvalidCompressionMethod '\0'
-#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
-
-#define NO_LZ4_SUPPORT() \
- ereport(ERROR, \
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
- errmsg("unsupported LZ4 compression method"), \
- errdetail("This functionality requires the server to be built with lz4 support."), \
- errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+#define CompressionMethodIsValid(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
-/*
- * GetCompressionMethodName - Get compression method name
- */
-static inline const char *
-GetCompressionMethodName(char method)
-{
- switch (method)
- {
- case TOAST_PGLZ_COMPRESSION:
- return "pglz";
- case TOAST_LZ4_COMPRESSION:
- return "lz4";
- default:
- elog(ERROR, "invalid compression method %c", method);
- return NULL; /* keep compiler quiet */
- }
-}
-
-/*
- * CompressionNameToMethod - Get compression method from compression name
- *
- * Search in the available built-in methods. If the compression not found
- * in the built-in methods then return InvalidCompressionMethod.
- */
-static inline char
-CompressionNameToMethod(char *compression)
-{
- if (strcmp(compression, "pglz") == 0)
- return TOAST_PGLZ_COMPRESSION;
- else if (strcmp(compression, "lz4") == 0)
- {
-#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
-#endif
- return TOAST_LZ4_COMPRESSION;
- }
-
- return InvalidCompressionMethod;
-}
-
/*
* GetDefaultToastCompression -- get the default toast compression method
*
@@ -101,7 +56,7 @@ CompressionNameToMethod(char *compression)
static inline char
GetDefaultToastCompression(void)
{
- return CompressionNameToMethod(default_toast_compression);
+ return (char) default_toast_compression;
}
/* pglz compression/decompression routines */
@@ -115,8 +70,10 @@ extern struct varlena *lz4_compress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value,
int32 slicelength);
+
+/* other stuff */
extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
-extern bool check_default_toast_compression(char **newval, void **extra,
- GucSource source);
+extern char CompressionNameToMethod(const char *compression);
+extern const char *GetCompressionMethodName(char method);
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index c2f2e0e230..566a1877ea 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -234,10 +234,10 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6626f8e927..3990933415 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,14 +227,13 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz.
SET default_toast_compression = 'lz4';
-ERROR: unsupported LZ4 compression method
-DETAIL: This functionality requires the server to be built with lz4 support.
-HINT: You need to rebuild PostgreSQL using --with-lz4.
+ERROR: invalid value for parameter "default_toast_compression": "lz4"
+HINT: Available values: pglz.
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 01:38:36PM -0400, Robert Haas wrote:
On Mon, Mar 22, 2021 at 12:16 PM Robert Haas <robertmhaas@gmail.com> wrote:
But, what about giving the default_toast_compression_method GUC an
assign hook that sets a global variable of type "char" to the
appropriate value? Then GetDefaultToastCompression() goes away
entirely. That might be worth exploring.Actually, we can do even better. We should just make the values
actually assigned to the GUC be TOAST_PGLZ_COMPRESSION etc. rather
than TOAST_PGLZ_COMPRESSION_ID etc. Then a whole lot of complexity
just goes away. I added some comments explaining why using
TOAST_PGLZ_COMPRESSION is the wrong thing anyway. Then I got hacking
and rearranged a few other things. So the attached patch does these
thing:- Changes default_toast_compression to an enum, as in your patch, but
now with values that are the same as what ultimately gets stored in
attcompression.
- Adds a comment warning against incautious use of
TOAST_PGLZ_COMPRESSION_ID, etc.
- Moves default_toast_compression_options to guc.c.
- After doing the above two things, we can remove the #include of
utils/guc.h into access/toast_compression.h, so the patch does that.
- Moves NO_LZ4_SUPPORT, GetCompressionMethodName, and
CompressionNameToMethod to guc.c. Making these inline functions
doesn't save anything meaningful; it's more important not to export a
bunch of random identifiers.
- Removes an unnecessary cast to bool from the definition of
CompressionMethodIsValid.I think this is significantly cleaner than what we have now, and I
also prefer it to your proposal.
+1
guc.c should not longer define this as extern:
default_toast_compression_options
I think you should comment that default_toast_compression is an int as far as
guc.c is concerned, but storing one of the char value of TOAST_*_COMPRESSION
Shouldn't varlena.c pg_column_compression() call GetCompressionMethodName () ?
I guess it should already have done that.
Maybe pg_dump.c can't use those constants, though (?)
--
Justin
Re: [HACKERS] Custom compression methods
Robert Haas <robertmhaas@gmail.com> writes:
I think this is significantly cleaner than what we have now, and I
also prefer it to your proposal.
+1 in general. However, I suspect that you did not try to compile
this without --with-lz4, because if you had you'd have noticed the
other uses of NO_LZ4_SUPPORT() that you broke. I think you need
to leave that macro where it is. Also, it's not nice for GUC check
functions to throw ereport(ERROR); we prefer the caller to be able
to decide if it's a hard error or not. That usage should be using
GUC_check_errdetail() or a cousin, so it can't share the macro anyway.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 2:10 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
I think this is significantly cleaner than what we have now, and I
also prefer it to your proposal.+1 in general. However, I suspect that you did not try to compile
this without --with-lz4, because if you had you'd have noticed the
other uses of NO_LZ4_SUPPORT() that you broke. I think you need
to leave that macro where it is.
You're correct that I hadn't tried this without --with-lz4, but I did
grep for other uses of NO_LZ4_SUPPORT() and found none. I also just
tried it without --with-lz4 just now, and it worked fine.
Also, it's not nice for GUC check
functions to throw ereport(ERROR); we prefer the caller to be able
to decide if it's a hard error or not. That usage should be using
GUC_check_errdetail() or a cousin, so it can't share the macro anyway.
I agree that these are valid points about GUC check functions in
general, but the patch I sent adds 0 GUC check functions and removes
1, and it didn't do the stuff you describe here anyway.
Are you sure you're looking at the patch I sent,
toast-compression-guc-rmh.patch? I can't help wondering if you applied
it to a dirty source tree or got the wrong file or something, because
otherwise I don't understand why you're seeing things that I'm not
seeing.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 1:58 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
guc.c should not longer define this as extern:
default_toast_compression_options
Fixed.
I think you should comment that default_toast_compression is an int as far as
guc.c is concerned, but storing one of the char value of TOAST_*_COMPRESSION
Done.
Shouldn't varlena.c pg_column_compression() call GetCompressionMethodName () ?
I guess it should already have done that.
It has a 0-3 integer, not a char value.
Maybe pg_dump.c can't use those constants, though (?)
Hmm, toast_compression.h might actually be safe for frontend code now,
or if necessary we could add #ifdef FRONTEND stanzas to make it so. I
don't know if that is really this patch's job, but I guess it could
be.
A couple of other things:
- Upon further reflection, I think the NO_LZ4_SUPPORT() message is
kinda not great. I'm thinking we should change it to say "LZ4 is not
supported by this build" instead of "unsupported LZ4 compression
method" and drop the hint and detail. That seems more like how we've
handled other such cases.
- It is not very nice that the three possible values of attcompression
are TOAST_PGLZ_COMPRESSION, TOAST_LZ4_COMPRESSION, and
InvalidCompressionMethod. One of those three identifiers looks very
little like the other two, and there's no real good reason for that. I
think we should try to standardize on something, but I'm not sure what
it should be. It would also be nice if these names were more visually
distinct from the related but very different enum values
TOAST_PGLZ_COMPRESSION_ID and TOAST_LZ4_COMPRESSION_ID. Really, as the
comments I added explain, we want to minimize the amount of code that
knows about the 0-3 "ID" values, and use the char values whenever we
can.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v2-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patchapplication/octet-stream; name=v2-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patchDownload
From 22271bf6d449b919341033462030d8a8b5c323e7 Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 22 Mar 2021 16:13:43 -0400
Subject: [PATCH v2] Tidy up more loose ends related to configurable TOAST
compression.
Change the default_toast_compression GUC to be an enum rather than
a string. Earlier, uncommitted versions of the patch supported using
CREATE ACCESS METHOD to add new compression methods to a running
system, but that idea was dropped before commit. So, we can simplify
the GUC handling as well, which as the nice side effect of improving
the error messages.
While updating the documentation to reflect the new GUC type, also
move it back to the right place in the list. I moved this while
revising what became commit 24f0e395ac5892cd12e8914646fe921fac5ba23d,
but apparently the intended ordering is "alphabetical" rather than
"whatever Robert thinks looks nice."
Rejigger things to avoid having access/toast_compression.h depend on
utils/guc.h, so that we don't end up with every file that includes
it also depending on something largely unrelated. Move a few
inline functions back into the C source file partly to help reduce
dependencies and partly just to avoid clutter. A few very minor
cosmetic fixes.
Original patch by Justin Pryzby, but very heavily edited by me,
and reverse reviewed by him.
---
doc/src/sgml/config.sgml | 44 +++++-----
src/backend/access/common/toast_compression.c | 74 ++++++++---------
src/backend/utils/misc/guc.c | 31 ++++---
src/include/access/toast_compression.h | 81 +++++--------------
src/test/regress/expected/compression.out | 4 +-
src/test/regress/expected/compression_1.out | 9 +--
6 files changed, 108 insertions(+), 135 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 5679b40dd5..97266b5290 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8108,28 +8108,6 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
- <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
- <term><varname>default_toast_compression</varname> (<type>string</type>)
- <indexterm>
- <primary><varname>default_toast_compression</varname> configuration parameter</primary>
- </indexterm>
- </term>
- <listitem>
- <para>
- This variable sets the default
- <link linkend="storage-toast">TOAST</link>
- compression method for columns of newly-created tables. The
- <command>CREATE TABLE</command> statement can override this default
- by specifying the <literal>COMPRESSION</literal> column option.
-
- The supported compression methods are <literal>pglz</literal> and
- (if configured at the time <productname>PostgreSQL</productname> was
- built) <literal>lz4</literal>.
- The default is <literal>pglz</literal>.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="guc-default-tablespace" xreflabel="default_tablespace">
<term><varname>default_tablespace</varname> (<type>string</type>)
<indexterm>
@@ -8173,6 +8151,28 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
+ <term><varname>default_toast_compression</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>default_toast_compression</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This variable sets the default
+ <link linkend="storage-toast">TOAST</link>
+ compression method for columns of newly-created tables. The
+ <command>CREATE TABLE</command> statement can override this default
+ by specifying the <literal>COMPRESSION</literal> column option.
+
+ The supported compression methods are <literal>pglz</literal> and
+ (if configured at the time <productname>PostgreSQL</productname> was
+ built) <literal>lz4</literal>.
+ The default is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-temp-tablespaces" xreflabel="temp_tablespaces">
<term><varname>temp_tablespaces</varname> (<type>string</type>)
<indexterm>
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 00af1740cf..d772cc0aaa 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -23,8 +23,15 @@
#include "fmgr.h"
#include "utils/builtins.h"
-/* Compile-time default */
-char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* GUC */
+int default_toast_compression = TOAST_PGLZ_COMPRESSION;
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
/*
* Compress a varlena using PGLZ.
@@ -271,46 +278,41 @@ toast_get_compression_id(struct varlena *attr)
}
/*
- * Validate a new value for the default_toast_compression GUC.
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
*/
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
+char
+CompressionNameToMethod(const char *compression)
{
- if (**newval == '\0')
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
{
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
}
- if (strlen(*newval) >= NAMEDATALEN)
- {
- GUC_check_errdetail("%s is too long (maximum %d characters).",
- "default_toast_compression", NAMEDATALEN - 1);
- return false;
- }
+ return InvalidCompressionMethod;
+}
- if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+const char *
+GetCompressionMethodName(char method)
+{
+ switch (method)
{
- /*
- * When source == PGC_S_TEST, don't throw a hard error for a
- * nonexistent compression method, only a NOTICE. See comments in
- * guc.h.
- */
- if (source == PGC_S_TEST)
- {
- ereport(NOTICE,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("compression method \"%s\" does not exist",
- *newval)));
- }
- else
- {
- GUC_check_errdetail("Compression method \"%s\" does not exist.",
- *newval);
- return false;
- }
+ case TOAST_PGLZ_COMPRESSION:
+ return "pglz";
+ case TOAST_LZ4_COMPRESSION:
+ return "lz4";
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ return NULL; /* keep compiler quiet */
}
-
- return true;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 3b36a31a47..3066bffc77 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -509,6 +509,14 @@ static struct config_enum_entry shared_memory_options[] = {
{NULL, 0, false}
};
+static struct config_enum_entry default_toast_compression_options[] = {
+ {"pglz", TOAST_PGLZ_COMPRESSION, false},
+#ifdef USE_LZ4
+ {"lz4", TOAST_LZ4_COMPRESSION, false},
+#endif
+ {NULL, 0, false}
+};
+
/*
* Options for enum values stored in other modules
*/
@@ -517,6 +525,7 @@ extern const struct config_enum_entry archive_mode_options[];
extern const struct config_enum_entry recovery_target_action_options[];
extern const struct config_enum_entry sync_method_options[];
extern const struct config_enum_entry dynamic_shared_memory_options[];
+extern const struct config_enum_entry default_toast_compression_options[];
/*
* GUC option variables that are exported from this module
@@ -3943,17 +3952,6 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
- {
- {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default compression for new columns."),
- NULL,
- GUC_IS_NAME
- },
- &default_toast_compression,
- DEFAULT_TOAST_COMPRESSION,
- check_default_toast_compression, NULL, NULL
- },
-
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
@@ -4595,6 +4593,17 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ TOAST_PGLZ_COMPRESSION,
+ default_toast_compression_options, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 44b73bd57c..46c2544e31 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,18 +13,26 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
-#include "utils/guc.h"
-
-/* GUCs */
-extern char *default_toast_compression;
-
-/* default compression method if not specified. */
-#define DEFAULT_TOAST_COMPRESSION "pglz"
+/*
+ * GUC support.
+ *
+ * default_toast_compression is an integer for purposes of the GUC machinery,
+ * but the value is one of the char values defined below, as they appear in
+ * pg_attribute.attcompression, e.g. TOAST_PGLZ_COMPRESSION.
+ */
+extern int default_toast_compression;
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression methods.
+ *
+ * Don't use these values for anything other than understanding the meaning
+ * of the raw bits from a varlena; in particular, if the goal is to identify
+ * a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc.
+ * below. We might someday support more than 4 compression methods, but
+ * we can never have more than 4 values in this enum, because there are
+ * only 2 bits available in the places where this is used.
*/
typedef enum ToastCompressionId
{
@@ -39,60 +47,13 @@ typedef enum ToastCompressionId
*/
#define TOAST_PGLZ_COMPRESSION 'p'
#define TOAST_LZ4_COMPRESSION 'l'
+#define InvalidCompressionMethod '\0'
-#define InvalidCompressionMethod '\0'
-#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
-
-#define NO_LZ4_SUPPORT() \
- ereport(ERROR, \
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
- errmsg("unsupported LZ4 compression method"), \
- errdetail("This functionality requires the server to be built with lz4 support."), \
- errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+#define CompressionMethodIsValid(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
-/*
- * GetCompressionMethodName - Get compression method name
- */
-static inline const char *
-GetCompressionMethodName(char method)
-{
- switch (method)
- {
- case TOAST_PGLZ_COMPRESSION:
- return "pglz";
- case TOAST_LZ4_COMPRESSION:
- return "lz4";
- default:
- elog(ERROR, "invalid compression method %c", method);
- return NULL; /* keep compiler quiet */
- }
-}
-
-/*
- * CompressionNameToMethod - Get compression method from compression name
- *
- * Search in the available built-in methods. If the compression not found
- * in the built-in methods then return InvalidCompressionMethod.
- */
-static inline char
-CompressionNameToMethod(char *compression)
-{
- if (strcmp(compression, "pglz") == 0)
- return TOAST_PGLZ_COMPRESSION;
- else if (strcmp(compression, "lz4") == 0)
- {
-#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
-#endif
- return TOAST_LZ4_COMPRESSION;
- }
-
- return InvalidCompressionMethod;
-}
-
/*
* GetDefaultToastCompression -- get the default toast compression method
*
@@ -101,7 +62,7 @@ CompressionNameToMethod(char *compression)
static inline char
GetDefaultToastCompression(void)
{
- return CompressionNameToMethod(default_toast_compression);
+ return (char) default_toast_compression;
}
/* pglz compression/decompression routines */
@@ -115,8 +76,10 @@ extern struct varlena *lz4_compress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value,
int32 slicelength);
+
+/* other stuff */
extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
-extern bool check_default_toast_compression(char **newval, void **extra,
- GucSource source);
+extern char CompressionNameToMethod(const char *compression);
+extern const char *GetCompressionMethodName(char method);
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index c2f2e0e230..566a1877ea 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -234,10 +234,10 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6626f8e927..3990933415 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,14 +227,13 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz.
SET default_toast_compression = 'lz4';
-ERROR: unsupported LZ4 compression method
-DETAIL: This functionality requires the server to be built with lz4 support.
-HINT: You need to rebuild PostgreSQL using --with-lz4.
+ERROR: invalid value for parameter "default_toast_compression": "lz4"
+HINT: Available values: pglz.
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
--
2.24.3 (Apple Git-128)
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 4:33 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Mon, Mar 22, 2021 at 1:58 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
guc.c should not longer define this as extern:
default_toast_compression_optionsFixed.
Fixed some more.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
v3-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patchapplication/octet-stream; name=v3-0001-Tidy-up-more-loose-ends-related-to-configurable-T.patchDownload
From 565b174b16d8ad5836ffc2e20d003c1bb0c464ce Mon Sep 17 00:00:00 2001
From: Robert Haas <rhaas@postgresql.org>
Date: Mon, 22 Mar 2021 16:54:26 -0400
Subject: [PATCH v3] Tidy up more loose ends related to configurable TOAST
compression.
Change the default_toast_compression GUC to be an enum rather than
a string. Earlier, uncommitted versions of the patch supported using
CREATE ACCESS METHOD to add new compression methods to a running
system, but that idea was dropped before commit. So, we can simplify
the GUC handling as well, which as the nice side effect of improving
the error messages.
While updating the documentation to reflect the new GUC type, also
move it back to the right place in the list. I moved this while
revising what became commit 24f0e395ac5892cd12e8914646fe921fac5ba23d,
but apparently the intended ordering is "alphabetical" rather than
"whatever Robert thinks looks nice."
Rejigger things to avoid having access/toast_compression.h depend on
utils/guc.h, so that we don't end up with every file that includes
it also depending on something largely unrelated. Move a few
inline functions back into the C source file partly to help reduce
dependencies and partly just to avoid clutter. A few very minor
cosmetic fixes.
Original patch by Justin Pryzby, but very heavily edited by me,
and reverse reviewed by him.
---
doc/src/sgml/config.sgml | 44 +++++-----
src/backend/access/common/toast_compression.c | 74 ++++++++---------
src/backend/utils/misc/guc.c | 30 ++++---
src/include/access/toast_compression.h | 81 +++++--------------
src/test/regress/expected/compression.out | 4 +-
src/test/regress/expected/compression_1.out | 9 +--
6 files changed, 107 insertions(+), 135 deletions(-)
diff --git a/doc/src/sgml/config.sgml b/doc/src/sgml/config.sgml
index 5679b40dd5..97266b5290 100644
--- a/doc/src/sgml/config.sgml
+++ b/doc/src/sgml/config.sgml
@@ -8108,28 +8108,6 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
- <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
- <term><varname>default_toast_compression</varname> (<type>string</type>)
- <indexterm>
- <primary><varname>default_toast_compression</varname> configuration parameter</primary>
- </indexterm>
- </term>
- <listitem>
- <para>
- This variable sets the default
- <link linkend="storage-toast">TOAST</link>
- compression method for columns of newly-created tables. The
- <command>CREATE TABLE</command> statement can override this default
- by specifying the <literal>COMPRESSION</literal> column option.
-
- The supported compression methods are <literal>pglz</literal> and
- (if configured at the time <productname>PostgreSQL</productname> was
- built) <literal>lz4</literal>.
- The default is <literal>pglz</literal>.
- </para>
- </listitem>
- </varlistentry>
-
<varlistentry id="guc-default-tablespace" xreflabel="default_tablespace">
<term><varname>default_tablespace</varname> (<type>string</type>)
<indexterm>
@@ -8173,6 +8151,28 @@ COPY postgres_log FROM '/full/path/to/logfile.csv' WITH csv;
</listitem>
</varlistentry>
+ <varlistentry id="guc-default-toast-compression" xreflabel="default_toast_compression">
+ <term><varname>default_toast_compression</varname> (<type>enum</type>)
+ <indexterm>
+ <primary><varname>default_toast_compression</varname> configuration parameter</primary>
+ </indexterm>
+ </term>
+ <listitem>
+ <para>
+ This variable sets the default
+ <link linkend="storage-toast">TOAST</link>
+ compression method for columns of newly-created tables. The
+ <command>CREATE TABLE</command> statement can override this default
+ by specifying the <literal>COMPRESSION</literal> column option.
+
+ The supported compression methods are <literal>pglz</literal> and
+ (if configured at the time <productname>PostgreSQL</productname> was
+ built) <literal>lz4</literal>.
+ The default is <literal>pglz</literal>.
+ </para>
+ </listitem>
+ </varlistentry>
+
<varlistentry id="guc-temp-tablespaces" xreflabel="temp_tablespaces">
<term><varname>temp_tablespaces</varname> (<type>string</type>)
<indexterm>
diff --git a/src/backend/access/common/toast_compression.c b/src/backend/access/common/toast_compression.c
index 00af1740cf..d772cc0aaa 100644
--- a/src/backend/access/common/toast_compression.c
+++ b/src/backend/access/common/toast_compression.c
@@ -23,8 +23,15 @@
#include "fmgr.h"
#include "utils/builtins.h"
-/* Compile-time default */
-char *default_toast_compression = DEFAULT_TOAST_COMPRESSION;
+/* GUC */
+int default_toast_compression = TOAST_PGLZ_COMPRESSION;
+
+#define NO_LZ4_SUPPORT() \
+ ereport(ERROR, \
+ (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
+ errmsg("unsupported LZ4 compression method"), \
+ errdetail("This functionality requires the server to be built with lz4 support."), \
+ errhint("You need to rebuild PostgreSQL using --with-lz4.")))
/*
* Compress a varlena using PGLZ.
@@ -271,46 +278,41 @@ toast_get_compression_id(struct varlena *attr)
}
/*
- * Validate a new value for the default_toast_compression GUC.
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
*/
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
+char
+CompressionNameToMethod(const char *compression)
{
- if (**newval == '\0')
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
{
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;
}
- if (strlen(*newval) >= NAMEDATALEN)
- {
- GUC_check_errdetail("%s is too long (maximum %d characters).",
- "default_toast_compression", NAMEDATALEN - 1);
- return false;
- }
+ return InvalidCompressionMethod;
+}
- if (!CompressionMethodIsValid(CompressionNameToMethod(*newval)))
+/*
+ * GetCompressionMethodName - Get compression method name
+ */
+const char *
+GetCompressionMethodName(char method)
+{
+ switch (method)
{
- /*
- * When source == PGC_S_TEST, don't throw a hard error for a
- * nonexistent compression method, only a NOTICE. See comments in
- * guc.h.
- */
- if (source == PGC_S_TEST)
- {
- ereport(NOTICE,
- (errcode(ERRCODE_UNDEFINED_OBJECT),
- errmsg("compression method \"%s\" does not exist",
- *newval)));
- }
- else
- {
- GUC_check_errdetail("Compression method \"%s\" does not exist.",
- *newval);
- return false;
- }
+ case TOAST_PGLZ_COMPRESSION:
+ return "pglz";
+ case TOAST_LZ4_COMPRESSION:
+ return "lz4";
+ default:
+ elog(ERROR, "invalid compression method %c", method);
+ return NULL; /* keep compiler quiet */
}
-
- return true;
}
diff --git a/src/backend/utils/misc/guc.c b/src/backend/utils/misc/guc.c
index 3b36a31a47..547b65fcb7 100644
--- a/src/backend/utils/misc/guc.c
+++ b/src/backend/utils/misc/guc.c
@@ -509,6 +509,14 @@ static struct config_enum_entry shared_memory_options[] = {
{NULL, 0, false}
};
+static struct config_enum_entry default_toast_compression_options[] = {
+ {"pglz", TOAST_PGLZ_COMPRESSION, false},
+#ifdef USE_LZ4
+ {"lz4", TOAST_LZ4_COMPRESSION, false},
+#endif
+ {NULL, 0, false}
+};
+
/*
* Options for enum values stored in other modules
*/
@@ -3943,17 +3951,6 @@ static struct config_string ConfigureNamesString[] =
check_default_table_access_method, NULL, NULL
},
- {
- {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
- gettext_noop("Sets the default compression for new columns."),
- NULL,
- GUC_IS_NAME
- },
- &default_toast_compression,
- DEFAULT_TOAST_COMPRESSION,
- check_default_toast_compression, NULL, NULL
- },
-
{
{"default_tablespace", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the default tablespace to create tables and indexes in."),
@@ -4595,6 +4592,17 @@ static struct config_enum ConfigureNamesEnum[] =
NULL, NULL, NULL
},
+ {
+ {"default_toast_compression", PGC_USERSET, CLIENT_CONN_STATEMENT,
+ gettext_noop("Sets the default compression for new columns."),
+ NULL,
+ GUC_IS_NAME
+ },
+ &default_toast_compression,
+ TOAST_PGLZ_COMPRESSION,
+ default_toast_compression_options, NULL, NULL
+ },
+
{
{"default_transaction_isolation", PGC_USERSET, CLIENT_CONN_STATEMENT,
gettext_noop("Sets the transaction isolation level of each new transaction."),
diff --git a/src/include/access/toast_compression.h b/src/include/access/toast_compression.h
index 44b73bd57c..46c2544e31 100644
--- a/src/include/access/toast_compression.h
+++ b/src/include/access/toast_compression.h
@@ -13,18 +13,26 @@
#ifndef TOAST_COMPRESSION_H
#define TOAST_COMPRESSION_H
-#include "utils/guc.h"
-
-/* GUCs */
-extern char *default_toast_compression;
-
-/* default compression method if not specified. */
-#define DEFAULT_TOAST_COMPRESSION "pglz"
+/*
+ * GUC support.
+ *
+ * default_toast_compression is an integer for purposes of the GUC machinery,
+ * but the value is one of the char values defined below, as they appear in
+ * pg_attribute.attcompression, e.g. TOAST_PGLZ_COMPRESSION.
+ */
+extern int default_toast_compression;
/*
* Built-in compression method-id. The toast compression header will store
* this in the first 2 bits of the raw length. These built-in compression
* method-id are directly mapped to the built-in compression methods.
+ *
+ * Don't use these values for anything other than understanding the meaning
+ * of the raw bits from a varlena; in particular, if the goal is to identify
+ * a compression method, use the constants TOAST_PGLZ_COMPRESSION, etc.
+ * below. We might someday support more than 4 compression methods, but
+ * we can never have more than 4 values in this enum, because there are
+ * only 2 bits available in the places where this is used.
*/
typedef enum ToastCompressionId
{
@@ -39,60 +47,13 @@ typedef enum ToastCompressionId
*/
#define TOAST_PGLZ_COMPRESSION 'p'
#define TOAST_LZ4_COMPRESSION 'l'
+#define InvalidCompressionMethod '\0'
-#define InvalidCompressionMethod '\0'
-#define CompressionMethodIsValid(cm) ((bool) ((cm) != InvalidCompressionMethod))
-
-#define NO_LZ4_SUPPORT() \
- ereport(ERROR, \
- (errcode(ERRCODE_FEATURE_NOT_SUPPORTED), \
- errmsg("unsupported LZ4 compression method"), \
- errdetail("This functionality requires the server to be built with lz4 support."), \
- errhint("You need to rebuild PostgreSQL using --with-lz4.")))
+#define CompressionMethodIsValid(cm) ((cm) != InvalidCompressionMethod)
#define IsStorageCompressible(storage) ((storage) != TYPSTORAGE_PLAIN && \
(storage) != TYPSTORAGE_EXTERNAL)
-/*
- * GetCompressionMethodName - Get compression method name
- */
-static inline const char *
-GetCompressionMethodName(char method)
-{
- switch (method)
- {
- case TOAST_PGLZ_COMPRESSION:
- return "pglz";
- case TOAST_LZ4_COMPRESSION:
- return "lz4";
- default:
- elog(ERROR, "invalid compression method %c", method);
- return NULL; /* keep compiler quiet */
- }
-}
-
-/*
- * CompressionNameToMethod - Get compression method from compression name
- *
- * Search in the available built-in methods. If the compression not found
- * in the built-in methods then return InvalidCompressionMethod.
- */
-static inline char
-CompressionNameToMethod(char *compression)
-{
- if (strcmp(compression, "pglz") == 0)
- return TOAST_PGLZ_COMPRESSION;
- else if (strcmp(compression, "lz4") == 0)
- {
-#ifndef USE_LZ4
- NO_LZ4_SUPPORT();
-#endif
- return TOAST_LZ4_COMPRESSION;
- }
-
- return InvalidCompressionMethod;
-}
-
/*
* GetDefaultToastCompression -- get the default toast compression method
*
@@ -101,7 +62,7 @@ CompressionNameToMethod(char *compression)
static inline char
GetDefaultToastCompression(void)
{
- return CompressionNameToMethod(default_toast_compression);
+ return (char) default_toast_compression;
}
/* pglz compression/decompression routines */
@@ -115,8 +76,10 @@ extern struct varlena *lz4_compress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum(const struct varlena *value);
extern struct varlena *lz4_decompress_datum_slice(const struct varlena *value,
int32 slicelength);
+
+/* other stuff */
extern ToastCompressionId toast_get_compression_id(struct varlena *attr);
-extern bool check_default_toast_compression(char **newval, void **extra,
- GucSource source);
+extern char CompressionNameToMethod(const char *compression);
+extern const char *GetCompressionMethodName(char method);
#endif /* TOAST_COMPRESSION_H */
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index c2f2e0e230..566a1877ea 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -234,10 +234,10 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz, lz4.
SET default_toast_compression = 'lz4';
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6626f8e927..3990933415 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -227,14 +227,13 @@ DETAIL: pglz versus lz4
-- test default_toast_compression GUC
SET default_toast_compression = '';
ERROR: invalid value for parameter "default_toast_compression": ""
-DETAIL: default_toast_compression cannot be empty.
+HINT: Available values: pglz.
SET default_toast_compression = 'I do not exist compression';
ERROR: invalid value for parameter "default_toast_compression": "I do not exist compression"
-DETAIL: Compression method "I do not exist compression" does not exist.
+HINT: Available values: pglz.
SET default_toast_compression = 'lz4';
-ERROR: unsupported LZ4 compression method
-DETAIL: This functionality requires the server to be built with lz4 support.
-HINT: You need to rebuild PostgreSQL using --with-lz4.
+ERROR: invalid value for parameter "default_toast_compression": "lz4"
+HINT: Available values: pglz.
DROP TABLE cmdata2;
CREATE TABLE cmdata2 (f1 text);
\d+ cmdata2
--
2.24.3 (Apple Git-128)
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 03:47:58PM -0400, Robert Haas wrote:
On Mon, Mar 22, 2021 at 2:10 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
Robert Haas <robertmhaas@gmail.com> writes:
I think this is significantly cleaner than what we have now, and I
also prefer it to your proposal.+1 in general. However, I suspect that you did not try to compile
this without --with-lz4, because if you had you'd have noticed the
other uses of NO_LZ4_SUPPORT() that you broke. I think you need
to leave that macro where it is.You're correct that I hadn't tried this without --with-lz4, but I did
grep for other uses of NO_LZ4_SUPPORT() and found none. I also just
tried it without --with-lz4 just now, and it worked fine.Also, it's not nice for GUC check
functions to throw ereport(ERROR); we prefer the caller to be able
to decide if it's a hard error or not. That usage should be using
GUC_check_errdetail() or a cousin, so it can't share the macro anyway.I agree that these are valid points about GUC check functions in
general, but the patch I sent adds 0 GUC check functions and removes
1, and it didn't do the stuff you describe here anyway.Are you sure you're looking at the patch I sent,
toast-compression-guc-rmh.patch? I can't help wondering if you applied
it to a dirty source tree or got the wrong file or something, because
otherwise I don't understand why you're seeing things that I'm not
seeing.
I'm guessing Tom read this hunk as being changes to
check_default_toast_compression() rather than removing the function ?
- * Validate a new value for the default_toast_compression GUC.
+ * CompressionNameToMethod - Get compression method from compression name
+ *
+ * Search in the available built-in methods. If the compression not found
+ * in the built-in methods then return InvalidCompressionMethod.
*/
-bool
-check_default_toast_compression(char **newval, void **extra, GucSource source)
+char
+CompressionNameToMethod(const char *compression)
{
- if (**newval == '\0')
+ if (strcmp(compression, "pglz") == 0)
+ return TOAST_PGLZ_COMPRESSION;
+ else if (strcmp(compression, "lz4") == 0)
{
- GUC_check_errdetail("%s cannot be empty.",
- "default_toast_compression");
- return false;
+#ifndef USE_LZ4
+ NO_LZ4_SUPPORT();
+#endif
+ return TOAST_LZ4_COMPRESSION;--
Justin
Re: [HACKERS] Custom compression methods
Justin Pryzby <pryzby@telsasoft.com> writes:
On Mon, Mar 22, 2021 at 03:47:58PM -0400, Robert Haas wrote:
Are you sure you're looking at the patch I sent,
toast-compression-guc-rmh.patch? I can't help wondering if you applied
it to a dirty source tree or got the wrong file or something, because
otherwise I don't understand why you're seeing things that I'm not
seeing.
I'm guessing Tom read this hunk as being changes to
check_default_toast_compression() rather than removing the function ?
Yeah, after looking closer, the diff looks like
check_default_toast_compression is being modified in-place,
whereas actually it's getting replaced by CompressionNameToMethod
which does something entirely different. I'd also not looked
closely enough at where NO_LZ4_SUPPORT() was being moved to.
My apologies --- I can only plead -ENOCAFFEINE.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Fri, Mar 19, 2021 at 2:44 PM Robert Haas <robertmhaas@gmail.com> wrote:
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.
I think this is bbe0a81db69bd10bd166907c3701492a29aca294, right?
This introduced a new assert failure, steps to reproduce:
"""
create table t1 (col1 text, col2 text);
create unique index on t1 ((col1 || col2));
insert into t1 values((select array_agg(md5(g::text))::text from
generate_series(1, 256) g), version());
"""
Attached is a backtrace from current HEAD
--
Jaime Casanova
Director de Servicios Profesionales
SYSTEMGUARDS - Consultores de PostgreSQL
Attachments:
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 1:22 PM Jaime Casanova
<jcasanov@systemguards.com.ec> wrote:
On Fri, Mar 19, 2021 at 2:44 PM Robert Haas <robertmhaas@gmail.com> wrote:
I committed the core patch (0003) with a bit more editing. Let's see
what the buildfarm thinks.I think this is bbe0a81db69bd10bd166907c3701492a29aca294, right?
This introduced a new assert failure, steps to reproduce:"""
create table t1 (col1 text, col2 text);
create unique index on t1 ((col1 || col2));
insert into t1 values((select array_agg(md5(g::text))::text from
generate_series(1, 256) g), version());
"""Attached is a backtrace from current HEAD
Thanks for reporting this issue. Actually, I missed setting the
attcompression for the expression index and that is causing this
assert. I will send a patch in some time.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 1:43 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
"""
create table t1 (col1 text, col2 text);
create unique index on t1 ((col1 || col2));
insert into t1 values((select array_agg(md5(g::text))::text from
generate_series(1, 256) g), version());
"""Attached is a backtrace from current HEAD
Thanks for reporting this issue. Actually, I missed setting the
attcompression for the expression index and that is causing this
assert. I will send a patch in some time.
PFA, patch to fix the issue.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v1-0001-Fix-attcompression-for-index-expression-columns.patchapplication/octet-stream; name=v1-0001-Fix-attcompression-for-index-expression-columns.patchDownload
From de55a6a4d23a58b6d899016fde7adf524c5fe937 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 14:02:06 +0530
Subject: [PATCH v1] Fix attcompression for index expression columns
For expression columns the attcompression was not set. So this patch
set it to the default compression method for compressible types otherwise
to the invalid compression method.
---
src/backend/catalog/index.c | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 397d70d..b5a79ce 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -30,6 +30,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/visibilitymap.h"
#include "access/xact.h"
@@ -379,6 +380,16 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ /*
+ * For expression column, if attribute type storage is compressible
+ * then set the default compression method, otherwise invalid
+ * compression method.
+ */
+ if (IsStorageCompressible(typeTup->typstorage))
+ to->attcompression = GetDefaultToastCompression();
+ else
+ to->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
/*
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 02:24:41PM +0530, Dilip Kumar wrote:
On Wed, Mar 24, 2021 at 1:43 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
create table t1 (col1 text, col2 text);
create unique index on t1 ((col1 || col2));
insert into t1 values((select array_agg(md5(g::text))::text from
generate_series(1, 256) g), version());Attached is a backtrace from current HEAD
Thanks for reporting this issue. Actually, I missed setting the
attcompression for the expression index and that is causing this
assert. I will send a patch in some time.PFA, patch to fix the issue.
Could you include a test case exercizing this code path ?
Like Jaime's reproducer.
--
Justin
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 2:49 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Mar 24, 2021 at 02:24:41PM +0530, Dilip Kumar wrote:
On Wed, Mar 24, 2021 at 1:43 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
create table t1 (col1 text, col2 text);
create unique index on t1 ((col1 || col2));
insert into t1 values((select array_agg(md5(g::text))::text from
generate_series(1, 256) g), version());Attached is a backtrace from current HEAD
Thanks for reporting this issue. Actually, I missed setting the
attcompression for the expression index and that is causing this
assert. I will send a patch in some time.PFA, patch to fix the issue.
Could you include a test case exercizing this code path ?
Like Jaime's reproducer.
I will do that.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 3:10 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
On Wed, Mar 24, 2021 at 2:49 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
On Wed, Mar 24, 2021 at 02:24:41PM +0530, Dilip Kumar wrote:
On Wed, Mar 24, 2021 at 1:43 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
create table t1 (col1 text, col2 text);
create unique index on t1 ((col1 || col2));
insert into t1 values((select array_agg(md5(g::text))::text from
generate_series(1, 256) g), version());Attached is a backtrace from current HEAD
Thanks for reporting this issue. Actually, I missed setting the
attcompression for the expression index and that is causing this
assert. I will send a patch in some time.PFA, patch to fix the issue.
Could you include a test case exercizing this code path ?
Like Jaime's reproducer.I will do that.
0001 ->shows compression method for the index attribute in index describe
0002 -> fix the reported bug (test case included)
Apart from this, I was thinking that currently, we are allowing to
ALTER SET COMPRESSION only for the table and matview, IMHO it makes
sense to allow to alter the compression method for the index column as
well? I mean it is just a one-line change, but just wanted to know
the opinion from others. It is not required for the storage because
indexes can not have a toast table but index attributes can be
compressed so it makes sense to allow to alter the compression method.
Thought?
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v2-0001-Show-compression-method-in-index-describe.patchapplication/x-patch; name=v2-0001-Show-compression-method-in-index-describe.patchDownload
From fa3ee05e21af64af86bedcc919488300c4f9a92d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 15:08:14 +0530
Subject: [PATCH v2 1/2] Show compression method in index describe
---
src/bin/psql/describe.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index eeac0ef..9d15324 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1897,6 +1897,7 @@ describeOneTableDetails(const char *schemaname,
if (pset.sversion >= 140000 &&
!pset.hide_compression &&
(tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
tableinfo.relkind == RELKIND_MATVIEW))
{
--
1.8.3.1
v2-0002-Fix-attcompression-for-index-expression-columns.patchapplication/x-patch; name=v2-0002-Fix-attcompression-for-index-expression-columns.patchDownload
From 54edcec2d44cfac01e1c2d20cc413dac57496ca9 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 14:02:06 +0530
Subject: [PATCH v2 2/2] Fix attcompression for index expression columns
For expression columns the attcompression was not set. So this patch
set it to the default compression method for compressible types otherwise
to the invalid compression method.
---
src/backend/catalog/index.c | 11 +++++++++++
src/test/regress/expected/compression.out | 13 +++++++++++++
src/test/regress/expected/compression_1.out | 14 ++++++++++++++
src/test/regress/sql/compression.sql | 8 ++++++++
4 files changed, 46 insertions(+)
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 397d70d..b5a79ce 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -30,6 +30,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/visibilitymap.h"
#include "access/xact.h"
@@ -379,6 +380,16 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ /*
+ * For expression column, if attribute type storage is compressible
+ * then set the default compression method, otherwise invalid
+ * compression method.
+ */
+ if (IsStorageCompressible(typeTup->typstorage))
+ to->attcompression = GetDefaultToastCompression();
+ else
+ to->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
/*
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index c2f2e0e..19707fb 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -313,6 +313,19 @@ SELECT pg_column_compression(f1) FROM cmdata;
lz4
(2 rows)
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+\d+ idx1
+ Index "public.idx1"
+ Column | Type | Key? | Definition | Storage | Compression | Stats target
+--------+------+------+------------+----------+-------------+--------------
+ expr | text | yes | (f1 || f2) | extended | pglz |
+unique, btree, for table "public.cmdata2"
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6626f8e..84b933d 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -310,6 +310,20 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+ERROR: relation "cmdata2" does not exist
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+ERROR: relation "cmdata2" does not exist
+LINE 1: INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::...
+ ^
+\d+ idx1
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 5e178be..4afd5a2 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -130,6 +130,14 @@ SELECT pg_column_compression(f1) FROM cmdata;
VACUUM FULL cmdata;
SELECT pg_column_compression(f1) FROM cmdata;
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+\d+ idx1
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 3:40 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
0001 ->shows compression method for the index attribute in index describe
0002 -> fix the reported bug (test case included)Apart from this, I was thinking that currently, we are allowing to
ALTER SET COMPRESSION only for the table and matview, IMHO it makes
sense to allow to alter the compression method for the index column as
well? I mean it is just a one-line change, but just wanted to know
the opinion from others. It is not required for the storage because
indexes can not have a toast table but index attributes can be
compressed so it makes sense to allow to alter the compression method.
Thought?
I have anyway created a patch for this as well. Including all three
patches so we don't lose track.
0001 ->shows compression method for the index attribute in index describe
0002 -> fix the reported bug (test case included)
(optional) 0003-> Alter set compression for index column
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v3-0003-ALTER-SET-COMPRESSION-for-index-columns.patchtext/x-patch; charset=US-ASCII; name=v3-0003-ALTER-SET-COMPRESSION-for-index-columns.patchDownload
From 6f83c026ddfe9cda14a9c5e965841b6cd2df0a2a Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 17:08:31 +0530
Subject: [PATCH v3 3/3] ALTER SET COMPRESSION for index columns
---
src/backend/commands/tablecmds.c | 2 +-
src/bin/psql/tab-complete.c | 4 ++--
src/test/regress/expected/compression.out | 16 ++++++++++++++++
src/test/regress/expected/compression_1.out | 6 ++++++
src/test/regress/sql/compression.sql | 4 ++++
5 files changed, 29 insertions(+), 3 deletions(-)
diff --git a/src/backend/commands/tablecmds.c b/src/backend/commands/tablecmds.c
index 3349bcf..6d5b79d 100644
--- a/src/backend/commands/tablecmds.c
+++ b/src/backend/commands/tablecmds.c
@@ -4335,7 +4335,7 @@ ATPrepCmd(List **wqueue, Relation rel, AlterTableCmd *cmd,
pass = AT_PASS_MISC;
break;
case AT_SetCompression: /* ALTER COLUMN SET COMPRESSION */
- ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW);
+ ATSimplePermissions(rel, ATT_TABLE | ATT_MATVIEW | ATT_INDEX);
/* This command never recurses */
/* No command-specific prep needed */
pass = AT_PASS_MISC;
diff --git a/src/bin/psql/tab-complete.c b/src/bin/psql/tab-complete.c
index b67f4ea..305665e 100644
--- a/src/bin/psql/tab-complete.c
+++ b/src/bin/psql/tab-complete.c
@@ -1772,10 +1772,10 @@ psql_completion(const char *text, int start, int end)
}
/* ALTER INDEX <name> ALTER COLUMN <colnum> */
else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLUMN", MatchAny))
- COMPLETE_WITH("SET STATISTICS");
+ COMPLETE_WITH("SET");
/* ALTER INDEX <name> ALTER COLUMN <colnum> SET */
else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLUMN", MatchAny, "SET"))
- COMPLETE_WITH("STATISTICS");
+ COMPLETE_WITH("COMPRESSION", "STATISTICS");
/* ALTER INDEX <name> ALTER COLUMN <colnum> SET STATISTICS */
else if (Matches("ALTER", "INDEX", MatchAny, "ALTER", "COLUMN", MatchAny, "SET", "STATISTICS"))
{
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 19707fb..000983f 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -326,6 +326,22 @@ generate_series(1, 50) g), VERSION());
expr | text | yes | (f1 || f2) | extended | pglz |
unique, btree, for table "public.cmdata2"
+CREATE INDEX idx2 ON cmdata2(f2);
+\d+ idx2
+ Index "public.idx2"
+ Column | Type | Key? | Definition | Storage | Compression | Stats target
+--------+------+------+------------+----------+-------------+--------------
+ f2 | text | yes | f2 | extended | lz4 |
+btree, for table "public.cmdata2"
+
+ALTER INDEX idx2 ALTER COLUMN f2 SET COMPRESSION pglz;
+\d+ idx2
+ Index "public.idx2"
+ Column | Type | Key? | Definition | Storage | Compression | Stats target
+--------+------+------+------------+----------+-------------+--------------
+ f2 | text | yes | f2 | extended | pglz |
+btree, for table "public.cmdata2"
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 84b933d..761fee4 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -324,6 +324,12 @@ ERROR: relation "cmdata2" does not exist
LINE 1: INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::...
^
\d+ idx1
+CREATE INDEX idx2 ON cmdata2(f2);
+ERROR: relation "cmdata2" does not exist
+\d+ idx2
+ALTER INDEX idx2 ALTER COLUMN f2 SET COMPRESSION pglz;
+ERROR: relation "idx2" does not exist
+\d+ idx2
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 4afd5a2..6777f11 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -137,6 +137,10 @@ CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
generate_series(1, 50) g), VERSION());
\d+ idx1
+CREATE INDEX idx2 ON cmdata2(f2);
+\d+ idx2
+ALTER INDEX idx2 ALTER COLUMN f2 SET COMPRESSION pglz;
+\d+ idx2
-- check data is ok
SELECT length(f1) FROM cmdata;
--
1.8.3.1
v3-0001-Show-compression-method-in-index-describe.patchtext/x-patch; charset=US-ASCII; name=v3-0001-Show-compression-method-in-index-describe.patchDownload
From fa3ee05e21af64af86bedcc919488300c4f9a92d Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 15:08:14 +0530
Subject: [PATCH v3 1/3] Show compression method in index describe
---
src/bin/psql/describe.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index eeac0ef..9d15324 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1897,6 +1897,7 @@ describeOneTableDetails(const char *schemaname,
if (pset.sversion >= 140000 &&
!pset.hide_compression &&
(tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
tableinfo.relkind == RELKIND_MATVIEW))
{
--
1.8.3.1
v3-0002-Fix-attcompression-for-index-expression-columns.patchtext/x-patch; charset=US-ASCII; name=v3-0002-Fix-attcompression-for-index-expression-columns.patchDownload
From 54edcec2d44cfac01e1c2d20cc413dac57496ca9 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 14:02:06 +0530
Subject: [PATCH v3 2/3] Fix attcompression for index expression columns
For expression columns the attcompression was not set. So this patch
set it to the default compression method for compressible types otherwise
to the invalid compression method.
---
src/backend/catalog/index.c | 11 +++++++++++
src/test/regress/expected/compression.out | 13 +++++++++++++
src/test/regress/expected/compression_1.out | 14 ++++++++++++++
src/test/regress/sql/compression.sql | 8 ++++++++
4 files changed, 46 insertions(+)
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 397d70d..b5a79ce 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -30,6 +30,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/visibilitymap.h"
#include "access/xact.h"
@@ -379,6 +380,16 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ /*
+ * For expression column, if attribute type storage is compressible
+ * then set the default compression method, otherwise invalid
+ * compression method.
+ */
+ if (IsStorageCompressible(typeTup->typstorage))
+ to->attcompression = GetDefaultToastCompression();
+ else
+ to->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
/*
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index c2f2e0e..19707fb 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -313,6 +313,19 @@ SELECT pg_column_compression(f1) FROM cmdata;
lz4
(2 rows)
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+\d+ idx1
+ Index "public.idx1"
+ Column | Type | Key? | Definition | Storage | Compression | Stats target
+--------+------+------+------------+----------+-------------+--------------
+ expr | text | yes | (f1 || f2) | extended | pglz |
+unique, btree, for table "public.cmdata2"
+
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 6626f8e..84b933d 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -310,6 +310,20 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+ERROR: relation "cmdata2" does not exist
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+ERROR: relation "cmdata2" does not exist
+LINE 1: INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::...
+ ^
+\d+ idx1
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 5e178be..4afd5a2 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -130,6 +130,14 @@ SELECT pg_column_compression(f1) FROM cmdata;
VACUUM FULL cmdata;
SELECT pg_column_compression(f1) FROM cmdata;
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+\d+ idx1
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 7:45 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I have anyway created a patch for this as well. Including all three
patches so we don't lose track.0001 ->shows compression method for the index attribute in index describe
0002 -> fix the reported bug (test case included)
(optional) 0003-> Alter set compression for index column
As I understand it, the design idea here up until now has been that
the index's attcompression values are irrelevant and ignored and that
any compression which happens for index attributes is based either on
the table attribute's assigned attcompression value, or the default.
If that's the idea, then all of these patches are wrong.
Now, a possible alternative design would be that the index's
attcompression controls compression for the index same as a table's
does for the table. But in that case, it seems to me that these
patches are insufficient, because then we'd also need to, for example,
dump and restore the setting, which I don't think anything in these
patches or the existing code will do.
My vote, as of now, is for the first design, in which case you need to
forget about trying to get pg_attribute to have the right contents -
in fact, I think we should set all the values there to
InvalidCompressionMethod to make sure we're not relying on them
anywhere. And then you need to make sure that everything that tries to
compress an index value uses the setting from the table column or the
default, not the setting on the index column.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 8:41 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Mar 24, 2021 at 7:45 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
I have anyway created a patch for this as well. Including all three
patches so we don't lose track.0001 ->shows compression method for the index attribute in index describe
0002 -> fix the reported bug (test case included)
(optional) 0003-> Alter set compression for index columnAs I understand it, the design idea here up until now has been that
the index's attcompression values are irrelevant and ignored and that
any compression which happens for index attributes is based either on
the table attribute's assigned attcompression value, or the default.
If that's the idea, then all of these patches are wrong.
The current design is that whenever we create an index, the index's
attribute copies the attcompression from the table's attribute. And,
while compressing the index tuple we will use the attcompression from
the index attribute.
Now, a possible alternative design would be that the index's
attcompression controls compression for the index same as a table's
does for the table. But in that case, it seems to me that these
patches are insufficient, because then we'd also need to, for example,
dump and restore the setting, which I don't think anything in these
patches or the existing code will do.
Yeah, you are right.
My vote, as of now, is for the first design, in which case you need to
forget about trying to get pg_attribute to have the right contents -
in fact, I think we should set all the values there to
InvalidCompressionMethod to make sure we're not relying on them
anywhere. And then you need to make sure that everything that tries to
compress an index value uses the setting from the table column or the
default, not the setting on the index column.
Okay, that sounds like a reasonable design idea. But the problem is
that in index_form_tuple we only have index tuple descriptor, not the
heap tuple descriptor. Maybe we will have to pass the heap tuple
descriptor as a parameter to index_form_tuple. I will think more
about this that how can we do that.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Andrew Dunstan <andrew@dunslane.net> writes:
On 3/20/21 3:03 PM, Tom Lane wrote:
I fixed up some issues in 0008/0009 (mostly cosmetic, except that
you forgot a server version check in dumpToastCompression) and
pushed that, so we can see if it makes crake happy.
It's still produced a significant amount more difference between the dumps. For now I've increased the fuzz factor a bit like this: - if ( ($oversion ne $this_branch && $difflines < 2000) + if ( ($oversion ne $this_branch && $difflines < 2700) I'll try to come up with something better. Maybe just ignore lines like SET default_toast_compression = 'pglz'; when taking the diff.
I see that some other buildfarm animals besides your own critters
are still failing the xversion tests, presumably because they lack
this hack :-(.
On reflection, though, I wonder if we've made pg_dump do the right
thing anyway. There is a strong case to be made for the idea that
when dumping from a pre-14 server, it should emit
SET default_toast_compression = 'pglz';
rather than omitting any mention of the variable, which is what
I made it do in aa25d1089. If we changed that, I think all these
diffs would go away. Am I right in thinking that what's being
compared here is new pg_dump's dump from old server versus new
pg_dump's dump from new server?
The "strong case" goes like this: initdb a v14 cluster, change
default_toast_compression to lz4 in its postgresql.conf, then
try to pg_upgrade from an old server. If the dump script doesn't
set default_toast_compression = 'pglz' then the upgrade will
do the wrong thing because all the tables will be recreated with
a different behavior than they had before. IIUC, this wouldn't
result in broken data, but it still seems to me to be undesirable.
dump/restore ought to do its best to preserve the old DB state,
unless you explicitly tell it --no-toast-compression or the like.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 11:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Okay, that sounds like a reasonable design idea. But the problem is
that in index_form_tuple we only have index tuple descriptor, not the
heap tuple descriptor. Maybe we will have to pass the heap tuple
descriptor as a parameter to index_form_tuple. I will think more
about this that how can we do that.
Another option might be to decide that the pg_attribute tuples for the
index columns always have to match the corresponding table columns.
So, if you alter with ALTER TABLE, it runs around and updates all of
the indexes to match. For expression index columns, we could store
InvalidCompressionMethod, causing index_form_tuple() to substitute the
run-time default. That kinda sucks, because it's a significant
impediment to ever reducing the lock level for ALTER TABLE .. ALTER
COLUMN .. SET COMPRESSION, but I'm not sure we have the luxury of
worrying about that problem right now.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 9:32 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Mar 24, 2021 at 11:41 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Okay, that sounds like a reasonable design idea. But the problem is
that in index_form_tuple we only have index tuple descriptor, not the
heap tuple descriptor. Maybe we will have to pass the heap tuple
descriptor as a parameter to index_form_tuple. I will think more
about this that how can we do that.Another option might be to decide that the pg_attribute tuples for the
index columns always have to match the corresponding table columns.
So, if you alter with ALTER TABLE, it runs around and updates all of
the indexes to match. For expression index columns, we could store
InvalidCompressionMethod, causing index_form_tuple() to substitute the
run-time default. That kinda sucks, because it's a significant
impediment to ever reducing the lock level for ALTER TABLE .. ALTER
COLUMN .. SET COMPRESSION, but I'm not sure we have the luxury of
worrying about that problem right now.
Actually, we are already doing this, I mean ALTER TABLE .. ALTER
COLUMN .. SET COMPRESSION is already updating the compression method
of the index attribute. So 0003 doesn't make sense, sorry for the
noise. However, 0001 and 0002 are still valid, or do you think that
we don't want 0001 also? If we don't need 0001 also then we need to
update the test output for 0002 slightly.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 11:42 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
On reflection, though, I wonder if we've made pg_dump do the right
thing anyway. There is a strong case to be made for the idea that
when dumping from a pre-14 server, it should emit
SET default_toast_compression = 'pglz';
rather than omitting any mention of the variable, which is what
I made it do in aa25d1089. If we changed that, I think all these
diffs would go away. Am I right in thinking that what's being
compared here is new pg_dump's dump from old server versus new
pg_dump's dump from new server?The "strong case" goes like this: initdb a v14 cluster, change
default_toast_compression to lz4 in its postgresql.conf, then
try to pg_upgrade from an old server. If the dump script doesn't
set default_toast_compression = 'pglz' then the upgrade will
do the wrong thing because all the tables will be recreated with
a different behavior than they had before. IIUC, this wouldn't
result in broken data, but it still seems to me to be undesirable.
dump/restore ought to do its best to preserve the old DB state,
unless you explicitly tell it --no-toast-compression or the like.
This feels a bit like letting the tail wag the dog, because one might
reasonably guess that the user's intention in such a case was to
switch to using LZ4, and we've subverted that intention by deciding
that we know better. I wouldn't blame someone for thinking that using
--no-toast-compression with a pre-v14 server ought to have no effect,
but with your proposal here, it would. Furthermore, IIUC, the user has
no way of passing --no-toast-compression through to pg_upgrade, so
they're just going to have to do the upgrade and then fix everything
manually afterward to the state that they intended to have all along.
Now, on the other hand, if they wanted to make practically any other
kind of change while upgrading, they'd have to do something like that
anyway, so I guess this is no worse.
But also ... aren't we just doing this to work around a test case that
isn't especially good in the first place? Counting the number of lines
in the diff between A and B is an extremely crude proxy for "they're
similar enough that we probably haven't broken anything."
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 4:57 PM Robert Haas <robertmhaas@gmail.com> wrote:
Fixed.
Fixed some more.
Committed.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Mar 24, 2021 at 11:42 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
On reflection, though, I wonder if we've made pg_dump do the right
thing anyway. There is a strong case to be made for the idea that
when dumping from a pre-14 server, it should emit
SET default_toast_compression = 'pglz';
rather than omitting any mention of the variable, which is what
I made it do in aa25d1089.
But also ... aren't we just doing this to work around a test case that
isn't especially good in the first place? Counting the number of lines
in the diff between A and B is an extremely crude proxy for "they're
similar enough that we probably haven't broken anything."
I wouldn't be proposing this if the xversion failures were the only
reason; making them go away is just a nice side-effect. The core
point is that the charter of pg_dump is to reproduce the source
database's state, and as things stand we're failing to ensure we
do that.
(But yeah, we really need a better way of making this check in
the xversion tests. I don't like the arbitrary "n lines of diff
is probably OK" business one bit.)
regards, tom lane
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 12:45 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
I wouldn't be proposing this if the xversion failures were the only
reason; making them go away is just a nice side-effect. The core
point is that the charter of pg_dump is to reproduce the source
database's state, and as things stand we're failing to ensure we
do that.
Well, that state is just a mental construct, right? In reality, there
is no such state stored anywhere in the old database. You're choosing
to attribute to it an implicit state that matches what would need to
be configured in the newer version to get the same behavior, which is
a reasonable thing to do, but it is an interpretive choice rather than
a bare fact.
I don't care very much if you want to change this, but to me it seems
slightly worse than the status quo. It's hard to imagine that someone
is going to create a new cluster, set the default to lz4, run
pg_upgrade, and then complain that the new columns ended up with lz4
as the default. It seems much more likely that they're going to
complain if the new columns *don't* end up with lz4 as the default.
And I also can't see any other scenario where imagining that the TOAST
compression property of the old database simply does not exist, rather
than being pglz implicitly, is worse.
But I could be wrong, and even if I'm right it's not a hill upon which
I wish to die.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 12:24:38PM -0400, Robert Haas wrote:
On Wed, Mar 24, 2021 at 11:42 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
On reflection, though, I wonder if we've made pg_dump do the right
thing anyway. There is a strong case to be made for the idea that
when dumping from a pre-14 server, it should emit
SET default_toast_compression = 'pglz';
rather than omitting any mention of the variable, which is what
I made it do in aa25d1089. If we changed that, I think all these
diffs would go away. Am I right in thinking that what's being
compared here is new pg_dump's dump from old server versus new
pg_dump's dump from new server?The "strong case" goes like this: initdb a v14 cluster, change
default_toast_compression to lz4 in its postgresql.conf, then
try to pg_upgrade from an old server. If the dump script doesn't
set default_toast_compression = 'pglz' then the upgrade will
do the wrong thing because all the tables will be recreated with
a different behavior than they had before. IIUC, this wouldn't
result in broken data, but it still seems to me to be undesirable.
dump/restore ought to do its best to preserve the old DB state,
unless you explicitly tell it --no-toast-compression or the like.This feels a bit like letting the tail wag the dog, because one might
reasonably guess that the user's intention in such a case was to
switch to using LZ4, and we've subverted that intention by deciding
that we know better. I wouldn't blame someone for thinking that using
--no-toast-compression with a pre-v14 server ought to have no effect,
but with your proposal here, it would. Furthermore, IIUC, the user has
no way of passing --no-toast-compression through to pg_upgrade, so
they're just going to have to do the upgrade and then fix everything
manually afterward to the state that they intended to have all along.
Now, on the other hand, if they wanted to make practically any other
kind of change while upgrading, they'd have to do something like that
anyway, so I guess this is no worse.
I think it's not specific to pg_upgrade, but any pg_dump |pg_restore.
The analogy with tablespaces is restoring from a cluster where the tablespace
is named "vast" to one where it's named "huge". I do this by running
PGOPTIONS=-cdefault_tablespace=huge pg_restore --no-tablespaces
So I thinks as long as --no-toast-compression does the corresponding thing, the
"restore with alternate compression" case is handled fine.
--
Justin
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 12:14 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Actually, we are already doing this, I mean ALTER TABLE .. ALTER
COLUMN .. SET COMPRESSION is already updating the compression method
of the index attribute. So 0003 doesn't make sense, sorry for the
noise. However, 0001 and 0002 are still valid, or do you think that
we don't want 0001 also? If we don't need 0001 also then we need to
update the test output for 0002 slightly.
It seems to me that 0002 is still not right. We can't fix the
attcompression to whatever the default is at the time the index is
created, because the default can be changed later, and there's no way
to fix index afterward. I mean, it would be fine to do it that way if
we were going to go with the other model, where the index state is
separate from the table state, either can be changed independently,
and it all gets dumped and restored. But, as it is, I think we should
be deciding how to compress new values for an expression column based
on the default_toast_compression setting at the time of compression,
not the time of index creation.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 1:24 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think it's not specific to pg_upgrade, but any pg_dump |pg_restore.
The analogy with tablespaces is restoring from a cluster where the tablespace
is named "vast" to one where it's named "huge". I do this by running
PGOPTIONS=-cdefault_tablespace=huge pg_restore --no-tablespacesSo I thinks as long as --no-toast-compression does the corresponding thing, the
"restore with alternate compression" case is handled fine.
I think you might be missing the point. If you're using pg_dump and
pg_restore, you can pass --no-toast-compression if you want. But if
you're using pg_upgrade, and it's internally calling pg_dump
--binary-upgrade, then you don't have control over what options get
passed. So --no-toast-compression is just fine for people who are
dumping and restoring, but it's no help at all if you want to switch
TOAST compression methods while doing a pg_upgrade. However, what does
help with that is sticking with what Tom committed before rather than
changing to what he's proposing now.
If you like his current proposal, that's fine with me, as long as
we're on the same page about what happens if we adopt it.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 01:30:26PM -0400, Robert Haas wrote:
On Wed, Mar 24, 2021 at 1:24 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
I think it's not specific to pg_upgrade, but any pg_dump |pg_restore.
The analogy with tablespaces is restoring from a cluster where the tablespace
is named "vast" to one where it's named "huge". I do this by running
PGOPTIONS=-cdefault_tablespace=huge pg_restore --no-tablespacesSo I thinks as long as --no-toast-compression does the corresponding thing, the
"restore with alternate compression" case is handled fine.I think you might be missing the point. If you're using pg_dump and
pg_restore, you can pass --no-toast-compression if you want. But if
Actually, I forgot that pg_restore doesn't (can't) have --no-toast-compression.
So my analogy is broken.
you're using pg_upgrade, and it's internally calling pg_dump
--binary-upgrade, then you don't have control over what options get
passed. So --no-toast-compression is just fine for people who are
dumping and restoring, but it's no help at all if you want to switch
TOAST compression methods while doing a pg_upgrade. However, what does
help with that is sticking with what Tom committed before rather than
changing to what he's proposing now.
I don't know what/any other cases support using pg_upgrade to change stuff like
the example (changing to lz4). The way to do it is to make the changes either
before or after. It seems weird to think that pg_upgrade would handle that.
I'm going to risk making the analogy that it's not supported to pass
--no-tablespaces from pg_upgrade to pg_dump/restore. It certainly can't work
for --link (except in the weird case that the dirs are on the same filesystem).
--
Justin
Re: [HACKERS] Custom compression methods
Justin Pryzby <pryzby@telsasoft.com> writes:
On Wed, Mar 24, 2021 at 01:30:26PM -0400, Robert Haas wrote:
... So --no-toast-compression is just fine for people who are
dumping and restoring, but it's no help at all if you want to switch
TOAST compression methods while doing a pg_upgrade. However, what does
help with that is sticking with what Tom committed before rather than
changing to what he's proposing now.
I don't know what/any other cases support using pg_upgrade to change stuff like
the example (changing to lz4). The way to do it is to make the changes either
before or after. It seems weird to think that pg_upgrade would handle that.
Yeah; I think the charter of pg_upgrade is to reproduce the old database
state. If you try to twiddle the process to incorporate some changes
in that state, maybe it will work, but if it breaks you get to keep both
pieces. I surely don't wish to consider such shenanigans as supported.
But let's ignore the case of pg_upgrade and just consider a dump/restore.
I'd still say that unless you give --no-toast-compression then I would
expect the dump/restore to preserve the tables' old compression behavior.
Robert's argument that the pre-v14 database had no particular compression
behavior seems nonsensical to me. We know exactly which compression
behavior it has.
regards, tom lane
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 10:57 PM Robert Haas <robertmhaas@gmail.com> wrote:
On Wed, Mar 24, 2021 at 12:14 PM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Actually, we are already doing this, I mean ALTER TABLE .. ALTER
COLUMN .. SET COMPRESSION is already updating the compression method
of the index attribute. So 0003 doesn't make sense, sorry for the
noise. However, 0001 and 0002 are still valid, or do you think that
we don't want 0001 also? If we don't need 0001 also then we need to
update the test output for 0002 slightly.It seems to me that 0002 is still not right. We can't fix the
attcompression to whatever the default is at the time the index is
created, because the default can be changed later, and there's no way
to fix index afterward. I mean, it would be fine to do it that way if
we were going to go with the other model, where the index state is
separate from the table state, either can be changed independently,
and it all gets dumped and restored. But, as it is, I think we should
be deciding how to compress new values for an expression column based
on the default_toast_compression setting at the time of compression,
not the time of index creation.
Okay got it. Fixed as suggested.
--
Regards,
Dilip Kumar
EnterpriseDB: http://www.enterprisedb.com
Attachments:
v4-0001-Show-compression-method-in-index-describe.patchtext/x-patch; charset=US-ASCII; name=v4-0001-Show-compression-method-in-index-describe.patchDownload
From c478a311b3410d5360946e953e1bfd53bbef7197 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 15:08:14 +0530
Subject: [PATCH v4 1/2] Show compression method in index describe
---
src/bin/psql/describe.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/src/bin/psql/describe.c b/src/bin/psql/describe.c
index eeac0ef..9d15324 100644
--- a/src/bin/psql/describe.c
+++ b/src/bin/psql/describe.c
@@ -1897,6 +1897,7 @@ describeOneTableDetails(const char *schemaname,
if (pset.sversion >= 140000 &&
!pset.hide_compression &&
(tableinfo.relkind == RELKIND_RELATION ||
+ tableinfo.relkind == RELKIND_INDEX ||
tableinfo.relkind == RELKIND_PARTITIONED_TABLE ||
tableinfo.relkind == RELKIND_MATVIEW))
{
--
1.8.3.1
v4-0002-Fix-attcompression-for-index-expression-columns.patchtext/x-patch; charset=US-ASCII; name=v4-0002-Fix-attcompression-for-index-expression-columns.patchDownload
From 75dbb3094973a4e69dafc213b760f5f13ab33a91 Mon Sep 17 00:00:00 2001
From: Dilip Kumar <dilipkumar@localhost.localdomain>
Date: Wed, 24 Mar 2021 14:02:06 +0530
Subject: [PATCH v4 2/2] Fix attcompression for index expression columns
For the expression columns, set the attcompression to the invalid
compression method and while actually compressing the data if the
attcompression is not valid then use the default compression method.
Basically, attcompression for the index attribute is always in sync with
that of the table attribute but for the expression columns that is not
possible so for them always use the current default method while
compressing the data.
---
src/backend/access/brin/brin_tuple.c | 8 +++++---
src/backend/access/common/indextuple.c | 16 ++++++++++++++--
src/backend/catalog/index.c | 9 +++++++++
src/test/regress/expected/compression.out | 6 ++++++
src/test/regress/expected/compression_1.out | 13 +++++++++++++
src/test/regress/sql/compression.sql | 7 +++++++
6 files changed, 54 insertions(+), 5 deletions(-)
diff --git a/src/backend/access/brin/brin_tuple.c b/src/backend/access/brin/brin_tuple.c
index 8d03e60..32ffd9f 100644
--- a/src/backend/access/brin/brin_tuple.c
+++ b/src/backend/access/brin/brin_tuple.c
@@ -220,10 +220,12 @@ brin_form_tuple(BrinDesc *brdesc, BlockNumber blkno, BrinMemTuple *tuple,
/*
* If the BRIN summary and indexed attribute use the same data
- * type, we can use the same compression method. Otherwise we
- * have to use the default method.
+ * type and it has a valid compression method, we can use the
+ * same compression method. Otherwise we have to use the
+ * default method.
*/
- if (att->atttypid == atttype->type_id)
+ if (att->atttypid == atttype->type_id &&
+ CompressionMethodIsValid(att->attcompression))
compression = att->attcompression;
else
compression = GetDefaultToastCompression();
diff --git a/src/backend/access/common/indextuple.c b/src/backend/access/common/indextuple.c
index 1f6b7b7..4d58644 100644
--- a/src/backend/access/common/indextuple.c
+++ b/src/backend/access/common/indextuple.c
@@ -103,8 +103,20 @@ index_form_tuple(TupleDesc tupleDescriptor,
(att->attstorage == TYPSTORAGE_EXTENDED ||
att->attstorage == TYPSTORAGE_MAIN))
{
- Datum cvalue = toast_compress_datum(untoasted_values[i],
- att->attcompression);
+ Datum cvalue;
+ char compression = att->attcompression;
+
+ /*
+ * If the compression method is not valid then use the default
+ * compression method. This can happen because, for the expression
+ * columns, we set the attcompression to the invalid compression
+ * method, while creating the index so that we can use the current
+ * default compression method when we actually need to compress.
+ */
+ if (!CompressionMethodIsValid(compression))
+ compression = GetDefaultToastCompression();
+
+ cvalue = toast_compress_datum(untoasted_values[i], compression);
if (DatumGetPointer(cvalue) != NULL)
{
diff --git a/src/backend/catalog/index.c b/src/backend/catalog/index.c
index 397d70d..5297d5e 100644
--- a/src/backend/catalog/index.c
+++ b/src/backend/catalog/index.c
@@ -30,6 +30,7 @@
#include "access/relscan.h"
#include "access/sysattr.h"
#include "access/tableam.h"
+#include "access/toast_compression.h"
#include "access/transam.h"
#include "access/visibilitymap.h"
#include "access/xact.h"
@@ -379,6 +380,14 @@ ConstructTupleDescriptor(Relation heapRelation,
to->attalign = typeTup->typalign;
to->atttypmod = exprTypmod(indexkey);
+ /*
+ * For the expression column set attcompression to the invalid
+ * compression. If the value for this attribute needs to be
+ * compressed during insertion then it will be compressed using
+ * default compression.
+ */
+ to->attcompression = InvalidCompressionMethod;
+
ReleaseSysCache(tuple);
/*
diff --git a/src/test/regress/expected/compression.out b/src/test/regress/expected/compression.out
index 566a187..61e97cb 100644
--- a/src/test/regress/expected/compression.out
+++ b/src/test/regress/expected/compression.out
@@ -313,6 +313,12 @@ SELECT pg_column_compression(f1) FROM cmdata;
lz4
(2 rows)
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/expected/compression_1.out b/src/test/regress/expected/compression_1.out
index 3990933..d03d625 100644
--- a/src/test/regress/expected/compression_1.out
+++ b/src/test/regress/expected/compression_1.out
@@ -309,6 +309,19 @@ SELECT pg_column_compression(f1) FROM cmdata;
pglz
(2 rows)
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+ERROR: unsupported LZ4 compression method
+DETAIL: This functionality requires the server to be built with lz4 support.
+HINT: You need to rebuild PostgreSQL using --with-lz4.
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+ERROR: relation "cmdata2" does not exist
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+ERROR: relation "cmdata2" does not exist
+LINE 1: INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::...
+ ^
-- check data is ok
SELECT length(f1) FROM cmdata;
length
diff --git a/src/test/regress/sql/compression.sql b/src/test/regress/sql/compression.sql
index 5e178be..76d1776 100644
--- a/src/test/regress/sql/compression.sql
+++ b/src/test/regress/sql/compression.sql
@@ -130,6 +130,13 @@ SELECT pg_column_compression(f1) FROM cmdata;
VACUUM FULL cmdata;
SELECT pg_column_compression(f1) FROM cmdata;
+-- test expression index
+DROP TABLE cmdata2;
+CREATE TABLE cmdata2 (f1 TEXT COMPRESSION pglz, f2 TEXT COMPRESSION lz4);
+CREATE UNIQUE INDEX idx1 ON cmdata2 ((f1 || f2));
+INSERT INTO cmdata2 VALUES((SELECT array_agg(md5(g::TEXT))::TEXT FROM
+generate_series(1, 50) g), VERSION());
+
-- check data is ok
SELECT length(f1) FROM cmdata;
SELECT length(f1) FROM cmdata1;
--
1.8.3.1
Re: [HACKERS] Custom compression methods
On Wed, Mar 24, 2021 at 2:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
But let's ignore the case of pg_upgrade and just consider a dump/restore.
I'd still say that unless you give --no-toast-compression then I would
expect the dump/restore to preserve the tables' old compression behavior.
Robert's argument that the pre-v14 database had no particular compression
behavior seems nonsensical to me. We know exactly which compression
behavior it has.
I said that it didn't have a state, not that it didn't have a
behavior. That's not exactly the same thing. But I don't want to argue
about it, either. It's a judgement call what's best here, and I don't
pretend to have all the answers. If you're sure you've got it right
... great!
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Thu, Mar 25, 2021 at 5:44 AM Dilip Kumar <dilipbalaut@gmail.com> wrote:
Okay got it. Fixed as suggested.
Committed with a bit of editing of the comments.
--
Robert Haas
EDB: http://www.enterprisedb.com
Re: [HACKERS] Custom compression methods
On Mon, Mar 22, 2021 at 10:41:33AM -0400, Robert Haas wrote:
trying to explain how TOAST works here, I added a link. It looks,
though, like that documentation also needs to be patched for this
change. I'll look into that, and your remaining patches, next.
I added an Opened Item for any necessary updates to the toast docs.
https://wiki.postgresql.org/wiki/PostgreSQL_14_Open_Items
--
Justin
Re: [HACKERS] Custom compression methods
On 3/24/21 12:45 PM, Tom Lane wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Mar 24, 2021 at 11:42 AM Tom Lane <tgl@sss.pgh.pa.us> wrote:
On reflection, though, I wonder if we've made pg_dump do the right
thing anyway. There is a strong case to be made for the idea that
when dumping from a pre-14 server, it should emit
SET default_toast_compression = 'pglz';
rather than omitting any mention of the variable, which is what
I made it do in aa25d1089.But also ... aren't we just doing this to work around a test case that
isn't especially good in the first place? Counting the number of lines
in the diff between A and B is an extremely crude proxy for "they're
similar enough that we probably haven't broken anything."I wouldn't be proposing this if the xversion failures were the only
reason; making them go away is just a nice side-effect. The core
point is that the charter of pg_dump is to reproduce the source
database's state, and as things stand we're failing to ensure we
do that.(But yeah, we really need a better way of making this check in
the xversion tests. I don't like the arbitrary "n lines of diff
is probably OK" business one bit.)
Well, I ran this module for years privately and used to have a matrix of
the exact number of diff lines expected for each combination of source
and target branch. If I didn't get that exact number of lines I reported
an error on stderr. That was fine when we weren't reporting the results
on the server, and I just sent an email to -hackers if I found an error.
I kept this matrix by examining the diffs to make sure they were all
benign. That was a pretty laborious process. So I decided to try a
heuristic approach instead, and by trial and error came up with this
2000 lines measurement. When this appeared to be working and stable the
module was released into the wild for other buildfarm owners to deploy.
Nothing is hidden here - the diffs are reported, see for example
<https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2021-03-28%2015%3A37%3A07&stg=xversion-upgrade-REL9_4_STABLE-HEAD>
What we're comparing here is target pg_dumpall against the original
source vs target pg_dumpall against the upgraded source.
If someone wants to come up with a better rule for detecting that
nothing has gone wrong, I'll be happy to implement it. I don't
particularly like the current rule either, it's there faute de mieux.
cheers
andrew
--
Andrew Dunstan
EDB: https://www.enterprisedb.com
Re: [HACKERS] Custom compression methods (buildfarm xupgrade)
On Sun, Mar 28, 2021 at 04:48:29PM -0400, Andrew Dunstan wrote:
Nothing is hidden here - the diffs are reported, see for example
<https://buildfarm.postgresql.org/cgi-bin/show_stage_log.pl?nm=crake&dt=2021-03-28%2015%3A37%3A07&stg=xversion-upgrade-REL9_4_STABLE-HEAD>
What we're comparing here is target pg_dumpall against the original
source vs target pg_dumpall against the upgraded source.
The command being run is:
https://github.com/PGBuildFarm/client-code/blob/master/PGBuild/Modules/TestUpgradeXversion.pm#L610
system( "diff -I '^-- ' -u $upgrade_loc/origin-$oversion.sql "
. "$upgrade_loc/converted-$oversion-to-$this_branch.sql "
. "> $upgrade_loc/dumpdiff-$oversion 2>&1");
...
my $difflines = `wc -l < $upgrade_loc/dumpdiff-$oversion`;
where -I means: --ignore-matching-lines=RE
I think wc -l should actually be grep -c '^[-+]'
otherwise context lines count for as much as diff lines.
You could write that with diff -U0 |wc -l, except the context is useful to
humans.
With some more effort, the number of lines of diff can be very small, allowing
a smaller fudge factor.
For upgrade from v10:
time make -C src/bin/pg_upgrade check oldsrc=`pwd`/10 oldbindir=`pwd`/10/tmp_install/usr/local/pgsql/bin
$ diff -u src/bin/pg_upgrade/tmp_check/dump1.sql src/bin/pg_upgrade/tmp_check/dump2.sql |wc -l
622
Without context:
$ diff -u src/bin/pg_upgrade/tmp_check/dump1.sql src/bin/pg_upgrade/tmp_check/dump2.sql |grep -c '^[-+]'
142
Without comments:
$ diff -I '^-- ' -u src/bin/pg_upgrade/tmp_check/dump1.sql src/bin/pg_upgrade/tmp_check/dump2.sql |grep -c '^[-+]'
130
Without SET default stuff:
diff -I '^$' -I "SET default_table_access_method = heap;" -I "^SET default_toast_compression = 'pglz';$" -I '^-- ' -u /home/pryzbyj/src/postgres/src/bin/pg_upgrade/tmp_check/dump1.sql /home/pryzbyj/src/postgres/src/bin/pg_upgrade/tmp_check/dump2.sql |less |grep -c '^[-+]'
117
Without trigger function call noise:
diff -I "^CREATE TRIGGER [_[:alnum:]]\+ .* FOR EACH \(ROW\|STATEMENT\) EXECUTE \(PROCEDURE\|FUNCTION\)" -I '^$' -I "SET default_table_access_method = heap;" -I "^SET default_toast_compression = 'pglz';$" -I '^-- ' -u /home/pryzbyj/src/postgres/src/bin/pg_upgrade/tmp_check/dump1.sql /home/pryzbyj/src/postgres/src/bin/pg_upgrade/tmp_check/dump2.sql |grep -c '^[-+]'
11
Maybe it's important not to totally ignore that, and instead perhaps clean up
the known/accepted changes like s/FUNCTION/PROCEDURE/:
</home/pryzbyj/src/postgres/src/bin/pg_upgrade/tmp_check/dump2.sql sed '/^CREATE TRIGGER/s/FUNCTION/PROCEDURE/' |diff -I '^$' -I "SET default_table_access_method = heap;" -I "^SET default_toast_compression = 'pglz';$" -I '^-- ' -u /home/pryzbyj/src/postgres/src/bin/pg_upgrade/tmp_check/dump1.sql - |grep -c '^[-+]'
11
It seems weird that we don't quote "heap" but we quote tablespaces and not
toast compression methods.
--
Justin
Re: [HACKERS] Custom compression methods
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Mar 24, 2021 at 2:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
But let's ignore the case of pg_upgrade and just consider a dump/restore.
I'd still say that unless you give --no-toast-compression then I would
expect the dump/restore to preserve the tables' old compression behavior.
Robert's argument that the pre-v14 database had no particular compression
behavior seems nonsensical to me. We know exactly which compression
behavior it has.
I said that it didn't have a state, not that it didn't have a
behavior. That's not exactly the same thing. But I don't want to argue
about it, either. It's a judgement call what's best here, and I don't
pretend to have all the answers. If you're sure you've got it right
... great!
I've not heard any other comments about this, but I'm pretty sure that
preserving a table's old toast behavior is in line with what we'd normally
expect pg_dump to do --- especially in light of the fact that we did not
provide any --preserve-toast-compression switch to tell it to do so.
So I'm going to go change it.
regards, tom lane
Re: [HACKERS] Custom compression methods
On 3/30/21 10:30 AM, Tom Lane wrote:
Robert Haas <robertmhaas@gmail.com> writes:
On Wed, Mar 24, 2021 at 2:15 PM Tom Lane <tgl@sss.pgh.pa.us> wrote:
But let's ignore the case of pg_upgrade and just consider a dump/restore.
I'd still say that unless you give --no-toast-compression then I would
expect the dump/restore to preserve the tables' old compression behavior.
Robert's argument that the pre-v14 database had no particular compression
behavior seems nonsensical to me. We know exactly which compression
behavior it has.I said that it didn't have a state, not that it didn't have a
behavior. That's not exactly the same thing. But I don't want to argue
about it, either. It's a judgement call what's best here, and I don't
pretend to have all the answers. If you're sure you've got it right
... great!I've not heard any other comments about this, but I'm pretty sure that
preserving a table's old toast behavior is in line with what we'd normally
expect pg_dump to do --- especially in light of the fact that we did not
provide any --preserve-toast-compression switch to tell it to do so.
So I'm going to go change it.
It looks like this CF entry should have been marked as committed so I
did that.
Regards,
--
-David
david@pgmasters.net
Re: [HACKERS] Custom compression methods
On Thu, Apr 8, 2021 at 11:32 AM David Steele <david@pgmasters.net> wrote:
It looks like this CF entry should have been marked as committed so I
did that.
Thanks.
Here's a patch for the doc update which was mentioned as an open item upthread.
--
Robert Haas
EDB: http://www.enterprisedb.com
Attachments:
update-storage-docs.patchapplication/octet-stream; name=update-storage-docs.patchDownload
diff --git a/doc/src/sgml/storage.sgml b/doc/src/sgml/storage.sgml
index 3234adb639..f78fe33eb5 100644
--- a/doc/src/sgml/storage.sgml
+++ b/doc/src/sgml/storage.sgml
@@ -394,9 +394,9 @@ Further details appear in <xref linkend="storage-toast-inmemory"/>.
<para>
The compression technique used for either in-line or out-of-line compressed
-data is a fairly simple and very fast member
-of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+can be selected using the <literal>COMPRESSION</literal> option on a per-column
+basis when creating a table. The default for columns with no explicit setting
+is taken from the value of <xref linkend="guc-default-toast-compression" />.
</para>
<sect2 id="storage-toast-ondisk">
@@ -425,8 +425,9 @@ retrieval of the values. A pointer datum representing an out-of-line on-disk
<acronym>TOAST</acronym>ed value therefore needs to store the OID of the
<acronym>TOAST</acronym> table in which to look and the OID of the specific value
(its <structfield>chunk_id</structfield>). For convenience, pointer datums also store the
-logical datum size (original uncompressed data length) and physical stored size
-(different if compression was applied). Allowing for the varlena header bytes,
+logical datum size (original uncompressed data length), physical stored size
+(different if compression was applied), and the compression method used, if
+any. Allowing for the varlena header bytes,
the total size of an on-disk <acronym>TOAST</acronym> pointer datum is therefore 18
bytes regardless of the actual size of the represented value.
</para>
Re: [HACKERS] Custom compression methods
On Thu, Apr 08, 2021 at 02:58:04PM -0400, Robert Haas wrote:
On Thu, Apr 8, 2021 at 11:32 AM David Steele <david@pgmasters.net> wrote:
It looks like this CF entry should have been marked as committed so I
did that.Thanks.
Here's a patch for the doc update which was mentioned as an open item upthread.
Thanks too.
It looks like this should not remove the word "data" ?
The compression technique used for either in-line or out-of-line compressed
-data is a fairly simple and very fast member
-of the LZ family of compression techniques. See
-<filename>src/common/pg_lzcompress.c</filename> for the details.
+can be selected using the <literal>COMPRESSION</literal> option on a per-column
+basis when creating a table. The default for columns with no explicit setting
+is taken from the value of <xref linkend="guc-default-toast-compression" />.I thought this patch would need to update parts about borrowing 2 spare bits,
but maybe that's the wrong header..
--
Justin
Re: [HACKERS] Custom compression methods
On Thu, Apr 8, 2021 at 3:38 PM Justin Pryzby <pryzby@telsasoft.com> wrote:
It looks like this should not remove the word "data" ?
Oh, yes, right.
The compression technique used for either in-line or out-of-line compressed -data is a fairly simple and very fast member -of the LZ family of compression techniques. See -<filename>src/common/pg_lzcompress.c</filename> for the details. +can be selected using the <literal>COMPRESSION</literal> option on a per-column +basis when creating a table. The default for columns with no explicit setting +is taken from the value of <xref linkend="guc-default-toast-compression" />.I thought this patch would need to update parts about borrowing 2 spare bits,
but maybe that's the wrong header..before.
We're not borrowing any more bits from the places where we were
borrowing 2 bits before. We are borrowing 2 bits from places that
don't seem to be discussed in detail here, where no bits were borrowed
before.
--
Robert Haas
EDB: http://www.enterprisedb.com